The Synergy Between Threat Modeling & Security Champions
Security-smart organizations continue to build security champion programs that empower teams to think security-first. Champions connect the development team with the security team to improve communication. This results in a deeper focus on security throughout the SDLC. But where does threat modeling fit into this structure?
Speakers
Chris Romeo, CEO, Devici
Dustin Lehr, Co-founder and Chief Product and Technology Officer, Katilyst
Transcript
Chris Romeo 00:01
Hey folks, welcome to another webinar from Devici, and this webinar is called The Synergy Between Threat Modeling & Security Champions, which is going to be a pretty insightful conversation. I can't wait to learn more about the synergistic relationships between these two things. My name is Chris Romeo. I'm the CEO of Devici. I've been in security for almost 30 years. At this point, done lots of different things, spent time at Cisco. I started Security Journey, and now I've founded Devici to build a new threat modeling platform for the industry. And as always, I'm excited to be joined by my friend Dustin Lehr, and so Dustin, I'd love for you to just give an introduction. People probably already know who you are, but just in case they don't, maybe set the stage for them.
Dustin Lehr 00:21
Yeah, happy to first of all, thanks for having me. Chris and I always look forward to our conversations full of excitement, potential disagreements, so we'll kind of see how this goes. So yeah, my background is in computer science. I was a software engineer for over 13 years. Kind of worked my way into a app architect position, and then security architect position, and then AppSec leadership. So I've led AppSec at companies like Staples, Fivetran, and then about three years ago, actually started a company Katilyst, and we focus on building security champion programs. So I've sort of went all in, if you will, on this whole idea of security champions, which you know were critical when I rolled out AppSec programs at these companies. So excited to be here, excited to talk about my favorite topic,
Chris Romeo 01:42
Yeah, two of my favorite topics coming together here. It's like worlds collide. I can't wait. But just a quick reminder for folks that are tuning in right now on LinkedIn, please, please, please, put your questions that you have into the comments, because we'll see those pop up on the screen here. And if you have any feedback or any opinions about the things that we're talking about, or you want to just answer the questions that we're asking. Put your answer in there too, because I guarantee you we'll react to those things. We'll bring them up on screen and talk through kind of what your thoughts are on the topic as well. Because one of the great things I could say about Dustin is he's a lifelong learner, just like myself, and so we don't come to a conversation like this and go, "We have all the answers. Everyone sit back and listen." We want to learn from what your experiences are if you're tuning in here. So please feel free to just go into LinkedIn comments on this event where you're watching it and type away a question, a comment, I promise you we will read it, we will react to it, and it may even take our conversation in a different direction. So Dustin, to kind of get started here, and I'm gonna ask the questions, I'm gonna let you answer, and then I'll give my answers as well, and I'm sure there'll be some conversation to come out of it. But, champions, why are they critical to an organization's threat modeling program?
Dustin Lehr 03:05
Yeah, I was actually just talking to someone about this this morning. I think that threat modeling and champions kind of go together, like steak and fine wine or something. You know, there's something about when you're trying to scale and change your culture having a threat modeling practice rolled out across your organization is a good way to really get people to think about security and better security habits, and then ultimately get to a point where they're following proactive security habits of which threat modeling is is obviously an essential one. So, I think, you know, I think champions are critical to rolling out these types of programs, because it helps you scale, essentially. And I'm sure we'll go into this more. But one thing that I always like to say is that people are actually more likely to listen to non-security people talk about security than security professionals. Like you can run around all day and say, Hey, threat modeling is important. A lot of people are just going to be like, okay, sure, the security team says that stuff all the time. It's the security team. Of course, we're supposed to be threat modeling and doing security and stuff. It's not until someone on their team, who they already respect, who they already know, says something about it, where they're going to listen, they're going to listen closer, right? They're going to say, Okay, it's not just crazy. Dustin talking about security again, this is actually someone I respect and work closely with who's helping me understand better practices, which includes threat modeling. So I think that's why champions are critical to rolling out a threat modeling program.
Chris Romeo 04:38
Yeah, I'm finding nothing to disagree with you on that, but I'll add a little bit of context from my perspective. When I think about what champions provide, and you mentioned Staples, Fivetran – in my background, it was Cisco. Cisco was my champion laboratory. That's what I thought of. It was a laboratory. I was doing experiments and things. But, what I saw in the criticality of this connection was really being able to scale. Because at Cisco, I had 25,000 engineers. It was very difficult to do anything to capture their attention. But with the 25 champions I started with, that grew to 500 over a couple of years, I could, to your point, I could use them as connected tissue to the engineers who were building most of the features. So my champions became that pass through, and they were able to provide that pass through connection, to your point, just that we could never reach as the security team, and I've experienced the same thing. You know, there's something to be said about a developer who knows how to code, has this background, and then someone who's maybe a senior developer and also a champion, someone they see as a mentor, they're saying, Yeah, this is, this is cool. This person has this thing – I'm going to take a look at it, versus you or I. We just don't have as much street cred because we haven't been in the trenches alongside them. Now, that's a pitch for people in AppSec to get into the trenches and stop hiding from developers and go spend some time with them, which is something I recommend to people all the time. Get in there, spend some time with them. Right away, almost six minutes into this webinar, I'm gonna go off the script already, because you made me think of another question. Chicken or egg, threat modeling your champions. What comes first? Because you made a comment. Something you said kind of led me into that question. But now I'm curious. I want to know more about your answer.
Dustin Lehr 06:42
Yeah, here we go off the ranch. We're gonna see where this heads. Yeah, threat modeling or champions. So in my view, and I've sort of expressed this a lot, in terms of an overall strategy for rolling out a successful AppSec program or a cybersecurity program as a whole. I actually think you need to start by building relationships before you try to do anything. I think the foundation is connecting with people and learning about challenges at the company – like imagine you come in, you know, since we're talking about chicken or egg, you come in cold to a new company, right? You don't know anything about the landscape. Nobody knows you. How do you go about that? I think the very first step is not, Okay, cool. Hey, everyone, we're going to have a threat modeling program. We're just going to roll this out. I think what you need to do is learn the landscape, you know, connect with folks, build those relationships, start to educate them in terms of why this stuff is important, And then, even get to the point where people understand why proactive approaches are necessary at the company. And I think even building that case takes a while. You almost have to demonstrate, Hey, there are issues that exist in our environment. Look, I found them. I did a pen test. I did scanning. There are things that exist that we need to fix, but then start thinking about, how do we actually prevent those things going forward? And that's where I think the conversation of proactive preventative measures comes into play, even things like training. You know? Why do we need to train people like, if you literally do a pen test walking in the door, this won't happen. This would never happen, unfortunately. I'd love to see this happen. You do a pen test, there are no findings. Everything's great. Everything's perfect, right? What do you need to do, like, what proactive measures exactly do you need to introduce into the environment? In that case, what kind of training is needed? Maybe not a whole lot, right? So you kind of have to build up to that. You have to say, hey, there are issues in our environment. Here's how we can prevent those issues. You know, better, architectural reviews, you know, of new code and so forth, threat modeling activities and all of that, and you don't do any of that without that foundation of connecting with people and building those relationships.
Chris Romeo 09:09
Yeah, and I love your thought experiment there, because there's not going to be a place where you come in, you did a pen test, there's no findings. They've already they've already implemented security perfectly, and they've considered everything from a privacy perspective, and they're secure by design and default, and they don't need a champions program, because everybody's just already doing everything, and everybody just talks about security at lunch. They won't be quiet. It's pie in the sky, though, right? Like, it's not a real thing. In the real world, we're dealing with software that was written five years ago, 10 years ago, if I'm lucky, that we're trying to then add all these things to later in the game.
Dustin Lehr 09:48
But why are you there? Why did they even hire you? You know,
Chris Romeo 09:52
If there's nothing, you mean, if there's there's nothing?
Dustin Lehr 09:54
Exactly. So, that that's the chicken and egg. You know?
Chris Romeo 09:57
That is the chicken and egg. Yeah. I want to react to a couple of these. So this was Stephen Clark, in reference to our conversation about, you know, the credibility that security may not have. Stephen's agreeing with us exactly, no one else sees them as software engineers, and to their credibility. And then he said, the need to be both an engineer and a security pro, like, that's, something that I've been spouting off about for a long time now, this need for AppSec people to be better engineers, like, if you're an app sec, you're not a security person, you're a security person, but you should have an engineering mindset so that you can practice that empathy and know what people are doing there.
Dustin Lehr 10:40
100%. And we said something earlier that I want to touch on, that I think is in line with this too. Since we're already off the ranch, let's just keep going. I think you know, the beauty of that champion model is it's a two way model, right? It's not just, Hey, we have representatives on the dev team that are going to push out, roll out security best practices. Awesome. You know, security pushes those to the champions. Champions push them to their team. Great, but that's only half the story, right? The other half is that those champions are also a liaison between their dev team and the security team. So that two way relationship needs to happen, and the security team, frankly, needs to be open to learning from the dev teams just as much as they're sharing with them, so that it's this two way conversation you talked about. Hey, we're lifelong learners. I think that growth mindset is exactly what both sides need to make this type of program work.
Chris Romeo 11:43
Yeah, yeah. So, Ron said security champion should come first. I think I agree with Ron there. I mean, I think they're both – threat modeling programs and champions are both crucial to the success of any application security program. And it's the things that I would start with – normally, the activity I start with is threat modeling just because I believe I can get the most bang for the buck quickly. But now, in the Cisco laboratory experiment, I had its group of champions already. Though they already existed, I didn't have to build it from scratch. So I was able to then work with them as my first constituents, help them understand threat modeling and help them roll it out further.
Dustin Lehr 12:22
Yeah, and I think that's a really good grassroots type approach, where threat modeling comes in very quickly. So the way I'm thinking about this is, yeah, you want to create relationships with folks, but I think it's also about finding allies, right? It's about finding people who respond really well to the message that you're saying. And those sort of become your champions. You know, you could start recruiting those people. You could start having regular meetings, you know, with them. And I think one of the first topics, if you sort of do this correctly, and we'll, I'm sure, get into this more, could be threat modeling, because it helps them understand the landscape. You know the idea that there are threats and there are people actively, potentially working against you in what you're building in your environment. Now I think you need to introduce that slowly, and I was going to share this later, but maybe I'll share this now that I think you can overwhelm people pretty easily when it comes to the topic of threat modeling, if you don't sort of lead them to water – like lead them down a path, right, think about where they're starting from. Don't go straight into STRIDE and how to do hardcore threat modeling the first time you meet with everybody, I think you need to introduce it in terms of, Look, this concept of threat modeling, it's not that hard to grasp. You're already doing it. You actually do it every day when you lock your front door at your house, right? When you think about, you know, the security features you might have at your house. You're thinking about, well, I have these because I don't want, you know, bad guys to get in and be able to have access to, you know, the important things in my house. So, you know, starting with that, I think, is a really good initial conversation with with these champions. And then you can obviously go from there.
Chris Romeo 14:20
Yeah, I have a slide in my threat modeling training that at the top. It says, "Everybody threat models." And so, I love doing that in a live scenario. It doesn't work so well in virtual but in a live scenario, I'll say, Okay, everybody that knows how to threat model, raise your hand. And like, a handful of hands will go up in the room in an intro to threat modeling class, for example. I'm like, "Okay, what I'm going to do now is make an argument for why everybody's hand should have gone up in here." And then I go through an example. I say, "Okay, you know, you walk out the front door and you need to walk a small child to school." And then I take them through. I say, "Imagine like all of a sudden, you hear cars driving past, and they're obviously going way too fast for the street. You reach down. You grab the child's hand. What did you do? Threat identified - child could get hit by car. Mitigation - grab the child's hand. Physical control of the situation." But it's definitely something that is relatable. But I love your idea here about let's give small bites. Let's not try to just dump this big, giant pile of information and stuff on top.
Dustin Lehr 15:30
Well, and let me, let me actually share something. Two things. One, I love that analogy that's actually similar to another analogy I like to use, and that's while you're driving you're always threat modeling as well, like, especially if you're driving defensively, which you probably should, you know. That car that's about to do something stupid. You know, you're going through the scenarios, you know? Are they going to pull in front of me? What's going to happen? Do they see me? Those are all great questions that, I think, prepare you for that threat modeling mindset. And then just to solidify what we were just saying, I want to share actually a mistake that I made, and that was the very first champion meeting that we had at a company that I was leading the AppSec team in. The very first session we had with all of our champions was literally hardcore threat modeling. You know, it was not a slow introduction, it was, here's all the details, here's, you know, how you o a data flow diagram and trust boundaries and each STRIDE thing. And it was way too much. We basically lost everyone, right? Like, attendance after that was just like, very low, which we had to sort of recover later. So that, to me, really solidified it in my head, like we need to, we need to bring them with us. Don't just overwhelm them with all the cool things we know.
Chris Romeo 16:57
Yeah, it's about reading your audience too, right? That's something that's a lesson you took away from that, because I've heard you talk about that example before. And that's something I think we all do in running any type of a program, you have to understand your audience, their capabilities, where they are, where you want them to go to, but realize they're not where you want them to get to. yet. That's why we're on this, this journey, right? Threat modeling is a journey. Security champions is a journey. It's not something where you're ever going to get to the point where, like, Hey, we've arrived. We don't have to do anything else with our champions or our threat modeling. We're always going to be making improvements to that. I wanted to highlight a couple more comments, because we got people that are giving us some good thoughts here as we go forward. So Barry Green's a name I remember from my Cisco days, and so definitely Barry agree that we did have good leadership who was helping to push forth the champions and what we were trying to do. And so I always appreciated having people like John Stewart at Cisco, who was the chief security officer and in charge of our security kind of group. He was always very supportive and helped to push with his peers, at the executive level, the importance of what we were doing and that was a big ecosystem too, right? Like that was a 75,000 person company, but I'm certainly fond of the time I spent growing up in the champions and threat modeling world both.
Dustin Lehr 18:11
That brings up a really good thing too to talk about really quick. And that is, you know, what does that leadership support look like? It sounds like you had actual advocates at the leadership level who were helping push the message and talk to people about the whole thing–evangelize it, if you will, as opposed to a head nod, which, you know, I think a lot of people, at least in my personal experience, that might be all the leadership support you get. Like you do this amazing proposal, I want to do this champion thing, and it's going to change the culture and all that, you know, in the CTO, or somebody at the C level goes, Great. Go for it. I support you. It's like, is that enough? You know what I mean? Or should we be asking more from senior leaders in terms of their involvement in the entire thing, front to back? You know, do you mind mentioning it at an all hands? Because, you know, it kind of goes to reporting chains to some degree too. It's one thing for the security team to be saying it as this external team to a dev team. It's something else for their leadership chain to be supporting the entire effort. You know, they're the ones that ultimately they're accountable to who fill out their performance reviews and so forth. So, yeah, and it's not easy to get that level of leadership support, but I think that leadership support that you're describing is is totally the ideal situation.
Chris Romeo 19:56
Yeah, we were very lucky in that situation to have an SVP. For anybody who doesn't know the name John Stewart, John Stewart is very famous from within the walls of Cisco. He's since retired now, but when John Chambers was CEO, he was John's security guy. So anything cybersecurity related, Chambers went right to John Stewart. So like he had that level of street cred inside of Cisco, but yet he was also very supportive of what we were doing, from Cisco security development lifecycle perspective, from a threat modeling perspective, from champions like he knew – we briefed him on what we were doing, and we didn't have to ask him to do things, to go promote. He just started promoting them for us. He would mention things at all hands. It was always top of mind for him. But to your point, Dustin like, there may be situations where, whether you're a threat modeling program leader or a champions program leader, don't be afraid to ask for your senior leadership to do things because it's so easy for us to sit back and say they'll know what to do. They're senior leaders. They'll know what to do. Give them a roadmap and say, Hey, could you help us promote this by sending this email, sending this other message, pinging some of your peers – like giving them a list of things, and don't just assume that they have it all figured out and know what to do, because, you know, that's, that's just, you're not setting yourself up for success.
Dustin Lehr 21:28
Yeah. And also, they probably want to help. I think some of it might be, you know, they don't have time for this, or there's other things that they're thinking about, or whatever. And I think not being afraid to ask, you know, and just bring them in, I think that they would really appreciate that, like as I have sort of climbed up the leadership chain to a large degree, I really appreciate it when folks ask for my help, for specific things that you know, only I can help with. I'm here for that, you know? And I think a lot of senior leaders have the same mindset. They're just never asked, you know? It's like the prettiest girls never asked out on a date kind of thing. You know? It's like people are intimidated when they really should be just getting out there and trying to build as much support as possible.
Chris Romeo 22:18
And that's a mindset that we grow right as we mature in our careers, we get to the point where we're not afraid, but I could see, you know, if your early career, it's intimidating to be leading a big program and, Oh, I can't ask this VP, I can't ask this Senior VP or Executive VP to do something. I'm just way down. I'm 10 layers away from them in the management chain. But I think the big takeaway there is, don't be afraid to ask, just like when you're buying something, if you don't ask for a discount, nobody's ever going to say, Justin, would you like $10,000 less for this car? Because I really feel like you're a nice guy and I want to take care of you. If you ask for $10,000 then there's a chance they're going to give it to you, but if you don't ask, you know, you can't kind of be concerned about it. I want to address a question here we just had in the comments, because there's some conversation going on the comments too. But I'm going to adapt this a little bit. I'm going to add champions to this as well. Because I'm curious about, Dustin from your perspective, both from a threat modeling perspective, but also from a champions perspective, where do product managers fit into both of these things?
Dustin Lehr 23:29
Oh, yeah, this is a great topic. So I think when it comes to product managers –assuming the definition of these are the folks who are in charge of basically writing the requirements for the product itself, functionality and ideally non-functional requirements. I think that's where the opportunity for them to write in those non-functional requirements around security, around quality, you know, quality assurance, that sort of stuff. That's where the conversation of threat modeling sort of comes in. And my own personal experience, is as like an app architect, bringing those product managers into the conversation about, hey, I know you want to do all these features, but from a technical perspective, we also need to account for these things, tech debt and so forth, I was actually able to proudly build in like, a 10% focus on tech debt each sprint, which I thought was amazing, and took months and months. But that was an ongoing conversation, with the product managers. And I think that relationship needs to be strong. I think the other thing that I've learned here, sort of as a product manager like I've been a product manager now for Katilyst for a few years, and what I'm learning on the other side is that I can't think of all these different scenarios. I actually need the engineers and the security folks to, you know, help me understand what I'm missing when I'm building out this product, because they're in the code, they have to deal with the architecture every day. So again, it's about building that strong relationship. And then that's where you can talk about threat modeling, and have the product manager sort of understand what that is for the betterment of the product. And I think that that's what this is all about, right, building a high quality product together. So to that end, and to the second part of your question, having product managers as security champions, beautiful, absolutely, find a way for your champion program to include them. And this kind of goes into some of my philosophies around how to build a champion program. I don't think it's just for technical people in general. I think it's really anybody who has that spark that you like to talk about, Chris, of interest in security and invite them in and find a way to design content that's tailored not just for technical people, but for product managers and Scrum Masters and all of those folks too. I think you're going to have overall better chance of success if you do that.
Chris Romeo 26:09
Yeah, and I think product managers being part of the threat modeling process is just priceless, because, as we wrote in the threat modeling manifesto I had a chance to be a part, an author, of that, there is great value in a diversified thought and experience in the threat modeling process. So having product managers come to the threat modeling session, product managers always point out things differently than developers do. Like a product manager will say, Well, what about this? And sometimes I'll look at it and go, I never thought about it like that, because I tend to be in the security track. I mean, I'm a terrible developer, but I can fake it a little bit, right? And I can, I guess I can fake it as a product manager too, but they have a perspective. Like pure product managers have a perspective on things. They just think about the customer, they think about the business, they think about the challenges that the customer is going to have. It's just a different perspective and mindset. And so, I think it's a great question and great thing to be thinking about here. I wanted to highlight one of Barry's answers. Barry's just killing it in the comments here. By the way, Barry great job here. This is a great point Barry makes here. It's hard for the product manager when you pull in threat models halfway through the product cycle and expose the risk that would force a redesign, because a product manager is ultimately a business person. If you're telling them they have to go back and start over again, that's a pretty costly thing, and that's a tough thing to move. So to Barry's point here, by having the product manager be part of that threat modeling process with you early, then they can be making changes and adjustments from the very beginning, versus trying to do this, you know, try to disrupt and then causing them a lot of additional pain.
Dustin Lehr 27:56
That's exactly right. It's a little bit more of like an agile type approach in general too. It's like, product manager goes well, here's kind of what I'm thinking, and the dev team goes, Well, did you consider these things? And then they can, like, revise the entire feature set together. I want to go to something that you started to touch on, and that's, you know, having product managers be part of that threat modeling process offers a different perspective. So, the most beautiful threat modeling sessions that I've seen are those that have multiple people who have different positions, right? Like, maybe you've got an app architect and then an engineer, and then product manager and so forth, and what starts to happen during that conversation is people are learning from each other. So, to a large degree, you know, security is essentially kicking off an exercise that reinforces tighter communication between these parties, so that the threat modeling session itself is not so much, you know, the security team working with the dev team only trying to figure out, you know, what the threats are and that sort of stuff, but to actually have these folks work with each other even better. So there's been so many sessions where people are like, Oh, it works like that. I didn't know it worked like that. I thought it worked like this. Well, no, it doesn't work like that. Here's how it works. And they're actually learning from each other in a conversation that wouldn't have otherwise happened, which I think is, again, one of the most beautiful things that I see during threat modeling sessions.
Chris Romeo 29:35
And that's a concept that you and I have both been passionate about for a long time now, and that is security culture at its core. Security culture is enabling those conversations. Yes, threat modeling was the vehicle, but those relationships are going to last longer than that threat modeling session, because that product manager is going to go, Hey, I got an architect that I can bounce this off from, but that developer's going to go, let me bounce this off my friend and product and see what think about that idea of us changing this thing. And once again, it's just building a kinder, gentler world within the development organization, but we're using, we're using threat modeling as a vehicle to get us there.
Dustin Lehr 30:14
And I think in a lot of cases, your AppSec team are playing the role of connectors across the dev teams like I can't tell you how many times you know the teams are split up in a way that's very siloed at the companies that I've worked with. But the AppSec team, or even a specific member might have visibility across those silos. So in some cases, we will identify, hey, you guys are working on something that's very similar to something that I've seen over here. Have you guys chatted yet? Because you should probably talk about that. Maybe there's like, a common library you can build, or some sort of best practices you can learn from each other about this stuff. And then, when they do, like you have helped unify and sort of cross boundaries that wouldn't have otherwise been crossed. So I love to see that. I love to see when AppSec teams are playing that role.
Chris Romeo 31:10
Yeah. We used to do that in our champions programs, champions meetings towards the end of my time leading the Cisco program. We would have a segment where developers, engineer, whatever, would would share something that they were working on that there was a chance could provide common value to other products and things like – it might have been a library – somebody created a library to do a particular function, and then we were like, hey, we want you to showcase this, because we want people to if somebody else could use it – why would we not make that connection? And it's, once again, it's that cultural change here. All right, we got to talk about BJs question here, because I've been looking at this for a little while going, this is a good one. So, I'll read it just to catch everybody up, in case people are, I don't know, maybe driving their car, don't stare at the screen in the middle of traffic. We've had both programs in place. So BJ is talking about threat modeling and champions. AppSec program does SAST, DAST, SCA – it's quite mature. However, incorporating threat modeling into this suite might raise concerns about potentially diluting the focus of our champions, especially since devs often perceive security as an overhead. That is definitely the case. So Dustin, starting with you, and then I'll give an answer, what would be the most crucial step to effectively integrate both programs and ensure their success? BJ, great question.
Dustin Lehr 32:30
So this goes back to what I was saying before, I think originally, in this and that's okay. Start by building relationships. Work up to finding issues, fixing issues, and then making a case for preventing issues, ultimately, based on the issues that you're finding. So I think part of what you're pointing out and how to address it is to help them connect those dots. You know, there's a lot of things you're fixing after the fact they've already been incorporated into the code base. They were found by a scanner. Now they're going to have to run off and fix those things. You know, what is cheaper overall for the business? Is it that cycle, or is it somehow introducing the right training and so forth to prevent those things from happening to begin with? So making the case that you know, if you're tired of running around fixing these things while they're already implemented. How do we be more proactive in preventing those things makes good business sense, and you could make the business case for that as well. It's cheaper overall from a development standpoint. Now, threat modeling is a little bit more at the architectural level, but I think the same thing applies, especially, I don't know how many times I've gone down this road, and maybe you have gone down this road too. Okay, there's a specific vulnerability, but it's actually indicating more of an architectural issue. There's a general architectural problem here that was implemented years ago, and now we're sort of spinning our wheels, fixing individual vulnerabilities when really what should be addressed is the overall architectural issue. Those architectural issues are hard to fix and they're very expensive to fix. You know, I always, and I heard this from a conference a long time ago, what makes it an architectural issue is basically the time it takes to correct it. It's not just a code change. You know, there's a lot more effort that has to be put into fixing those architectural issues that ties directly to threat modeling. You know, in order to prevent those larger architectural issues, can we have conversations earlier in our cycle to identify those potential architectural issues, to prevent those from going live in the future, we're going to need a threat modeling program to do that. Or, again, if you like, fixing all these things while they're live, let's do that. But I think it's much better to get ahead of it and have more of a, you know, more of a culture that's focused on preventing those things.
Chris Romeo 34:55
Yeah, so BJ, I'm going to answer your question the way I like to answer this type of question, which is, if I was in your shoes, it's how I like to set this up? So if I was in your shoes with a program that had mature SAST, DAST, SCA, but there was some potential pushback about threat modeling taking up too much champions time, what would I do? And so I think how I would approach this, given the fact that you have maturity with those other scanning tools, is I would spend some time trying to fully mature those tools. Because if I have a program where those things have been running for a while, we kind of have a system going – I want to get to the point where I'm really limiting the results of those tools to real issues that require a ticket and some type of a fix. And so I would invest some more time in maturing those things to the maximum level that I could get them to, but then also balancing that and saying, Okay, we're going to change the scope of our champions program here, to some degree, to say threat modeling is something that's going to need to be important. And I would do that in past champions programs using goals. I used to have a yearly goal set of goals that champions and their managers would re-up each year. And so I would change some of the scope of the champion to migrate away from dealing with SAST, DAST, SCA, and even look for ways to push those to the end developers, versus having the champions deal with those issues. Because I want the champions to be able to sink their teeth into the things they're going to have the biggest impact for the business. And in your case, I think threat modeling would have a bigger impact now than those other tools do, because you've matured them. And so it's time to get Threat Modeling moving forward and working towards maturing that. And then you can add the next thing along the way as you go. Great question, though. Lots of great comments going on here from Barry and Steven about kind of the value of these things. Dustin, I want to make sure we catch two more questions that we originally wanted to talk about, because I want to hear your answers to them. That's selfish on my part, I know, but it was what we thought of as our fourth question. How do you measure champions' contributions to threat modeling? Because I'm struggling with this, and I really want to know your your answer.
Dustin Lehr 37:14
This is a good one, so I think some of it ties back to what we were just talking about. You know, how do you demonstrate that, overall, your threat modeling practice is having an impact on the actual issues that are reaching production, you know, the escape rate, if you will. Because I think, I think you need to establish that to make a case to continue doing threat modeling.
Chris Romeo 37:42
What do you mean by escape rate? In case somebody doesn't know that term
Dustin Lehr 37:44
Sure, sure, sure, yeah, I heard this term years ago, which I really like. So think of, think of the steps of the SDLC. Escape rate would be essentially measuring how many vulnerabilities end up in each of those stages. You know, how many things are in production, versus pre prod, versus a dev environment, and so forth. So it's kind of like, think of it as, you know, how many vulnerabilities are escaping from the developer's keyboard into the next stage, and how many are escaping from there to the next stage and so forth. So I bring that up in terms of, you know, trying to measure success of a threat modeling program, because that escape rate, ideally, you'd be catching things earlier in the process, right? You would be seeing overall, less of those issues escape all the way into later stages, Is kind of the way that you think about it. So, measuring things like, you know, the vulnerabilities that you're finding overall with your SAST and DAST, and SCA tools, and then seeing how those are changing over time, I think that this is where we start talking about correlation versus causation. The fact that it's very difficult to directly measure the impact that threat modeling is having, which is the same with champions, and which is, frankly, the same as training. How do you prove that, like, I'll just take a training example, right? How do you prove that someone was about to introduce a vulnerability, but they remembered their training so they didn't right? You could never measure that. Okay? So what you can measure, though, is a correlation, and I always talk about it in terms of two different ways to measure correlation, before and after. Okay, for our threat modeling practice, here was our escape rate. We saw these vulnerabilities being introduced in these stages of our development lifecycle versus after introducing threat modeling. Here's what we see now. Okay, so threat modeling is likely having an impact, and you can actually measure the strength of that correlation as well, especially if you do it across different teams, different timelines and so forth. So the before and after is a good way to measure it. The other great way to measure it is more like an A/B approach, where you can say, hey, maybe certain teams who have, you know, adopted the threat modeling practice, how are they compared to other teams that don't do that, you know? And can you draw a correlation in that way and and thus make a case for for threat modeling at your company.
Chris Romeo 40:25
Yeah, so I love what you said there. I don't disagree with any of that. I would say that I've had success early on in just using bulk metrics to measure things. So how many threat models are we doing? Well, you might say, well, that doesn't tell you how good they are in the beginning, I don't care how good they are, because a threat model that was done is better than a threat model that wasn't done. Like you're never going to show me a model and go that has less value than not doing it at all. I mean, maybe if you made it so complicated that nobody... but who's going to invest 100 hours to make the most complicated model possible, right? Like, as long as somebody learned something in the process, then that was a success. And so in the early days, I would say, I like bulk metrics, just to say, how many models are we doing? How many models did we do, this month versus last month? This quarter versus last quarter? Can you show me trend lines and things? Let's see, and then you can start to think about, you know, this escape rate, but I'm going to try to add that into my thinking as well, because it's a great way to to consider the impact of change, especially if you have that data for two years, and then you change something, being able to measure the next quarter and say, Okay, well, we made this tweak. How did that change our historical numbers? That's how you run a business, right? And security and champions and threat modeling, if you're running this program, it's a business. And metrics are how you run your business and how you understand how it's performing. And so these are things that definitely have to be applied.
Dustin Lehr 42:04
Yeah, let me expand on this a little bit too, because I think the nature of your question, I sort of answered the first part, and that's, how do you make a case for threat modeling. But more specifically, how do you measure champions contributions to threat modeling? So I think to step back, which we didn't really cover yet. You know, what should champions contributions to threat modeling be? We should probably state a little bit more clearly, and that's, I don't really expect champions, at least not at first, to be able to run a threat modeling session or lead a threat modeling session at the same level that a security professional would. Okay, so there's actually unique contributions that champions can make. And we've been kind of talking about this, but I think it's good to call it out. You know, they're more familiar with the systems themselves, maybe even more familiar than what the security team knows. Right? The Security Team sort of brings their expertise, and the developers and champions bring their expertise, and it's the combination of those that I think makes for a really good threat modeling exercise. So how do you measure that? You know that your champion program is actually having an impact on threat modeling? I think it comes down to measuring those unique contributions and figuring out how they're affecting your threat modeling practice overall, with things like, you know, time to threat model. As an example, all right, we used to do threat modeling sessions, and we had to do the entire thing top to bottom, you know, from scratch, in an entire day, because we couldn't get anyone's time. So the first part of the day was learning the architecture and doing the data flow diagram and drawing the trust boundaries and all that stuff. Then we got to identifying threats, and then we got to results and all that stuff. Lots of work, right? Having champions there could shorten that quite a bit, especially if those champions are proactively putting together a lot of that stuff. They create an architecture diagram to begin with, or they even push their teams over time talking about culture change to just do that as part of their development cycle. You know, now all of your teams have an architecture diagram that's up to date, mostly, and all that stuff. It's a better foundation, you know, that you can use to perform threat modeling on top of that. And I think that sort of stuff is how you can measure champions specific contribution to the threat modeling practice.
Chris Romeo 44:31
Yeah, I like what you said that about champions are going to make it easier, because they are that bridge. If they're operating as they were designed and intended to they are that bridge, and they can shorten that time it takes, or, like you said, they can bring the architectural diagram. Oh, don't worry, we've already been working on one, so we're ready to use this as a starting point. And so maybe that cuts your threat modeling time in half from what you were currently planning to spend on it. So I want to get this last question about incentivization for champions to perform threat modeling. Because I know that's a incentivizing people and behaviors is a thing you spend a lot of time noodling on, so how do we do this? How do we incentivize a champion to perform threat modeling? What is the champion going to see this as value? Like, what are they going to want to get for value.
Dustin Lehr 45:22
Yeah, this is a whole rabbit hole we can go down, and I, at this point, have spent a lot of time looking into behavioral science and gamification and all these sorts of techniques to find out how to effectively motivate people. You know, I think where you should start is trying to understand your people and understand your culture. What are things that are being currently incentivized? How do leaders judge the performance of the individuals at the company? You know, what types of things culturally are generally reinforced positively, what things are punished, like learning all that stuff I think is really crucial, because what you want to do is then design an incentivization system that's based on that. Here's the culture we're in. Here are the types of things that are going to incentivize folks. So doing all of that, I think, you know, it comes down to a handful of things that I've personally observed, especially around champions and technical champions or developers. You know, I think a big, very effective way to incentivize people is actually through things like status and recognition as an example. And let's get into more detail here. But you know, people want to be recognized for the good work that they do ultimately. So someone's going above and beyond, like we were just describing, you know, rolling out a better architectural design program. Let's say they did that specifically for their team, even just calling them out, like you don't have to spend money on challenge coins and sweatshirts and stuff and pizza, but saying, hey, this person made a significant contribution to the security practice overall. Hey, CTO, since you are actually supporting us, you know, in line with that other conversation we were having, can you call that out during some sort of all hands, right? And, you know, tell the manager, hey, this person that reports to you made a significant contribution when it comes to security, because then they can build that into performance reviews and just sort of get a general better sense of the contributions that their employees making. I know Chris, you want me to mention SAPS, which I appreciate the invitation to do so, because this is a gamification acronym that I learned years ago. SAPS, okay, Status, Access, Power and Stuff is what it stands for. And I like to talk about this because it's a good way to brainstorm other ways of delivering rewards than I think people initially think of. I think when people hear incentivization, they go right to stuff very quickly. Okay, bonus, salary increase, you know, how do I incentivize them with stuff? But there's three other dimensions of rewards. Status, like we were just talking about labeling somebody as a guru in something, or giving them recognition for the work that they're doing. Access, maybe there's something that people have to earn their way into. Maybe they're not able to use certain tools or have access to certain environments and so forth, until they demonstrate that they're going to make contributions from a security standpoint, that would be an Access reward. And this kind of goes into the way that we're wired, you know, the second we can't have something as humans, whether you think this is logical or not, it's true, we want it more, you know. So how can you tap into that to make it more attractive to folks? And then the P stands for Power. That's really the ability to make decisions, be part of the decision making process, have opportunities to provide feedback, that sort of stuff. That's also very motivating for folks, too. So that's the start. There's a lot of depth there, but I would offer that as a start.
Chris Romeo 49:20
Yeah, and this is an area that you've written about a lot, spoken about a lot, so folks can find other talks and things where you're going to go a lot deeper. And I'm glad you mentioned it, though, because this is a concept that I learned from you. I think I was using pieces of it that I had stumbled upon, you know, a lot more on the Stuff and and Access, were the two kind of areas of it that I had tapped into, just because I just saw an opportunity to people like having Access to senior leaders, for example, like a senior leader hosting a dinner with four or five people like you can that's worth $1,000 salary increase to some people because they want to be able to mold their career with the insight of somebody who they want to be in their shoes, for example. And so that's worth more than money to them, because it's part of their their career trajectory. All right, we're running out of time here, Dustin, where can folks find you, and how can they see Katilyst? I've had a chance to follow what you guys are doing. It's pretty cool from a champions perspective. And so what? How do folks find you, and how do they find Katilyst?
Dustin Lehr 50:32
Yeah, absolutely. So I'm around. I'm on LinkedIn. I do these types of events all the time as well. I'm pretty consistently posting. Feel free to send me a note over LinkedIn. You can also reach me over email, dustin(at)katilyst.com. Don't hesitate to reach out. I love talking about this stuff. If you've got questions, there are no stupid questions, you know, any sort of struggles or challenges that you're having in this area, I'm happy to help. And then, in terms of finding Katilyst, we do have a website, katilyst.com you can sort of check out the four different things we do. We offer security champion services. We have a product as well that helps to gamify your security champion program. So yeah, and feel free to reach out over the website. There's a there's a form there you can fill out, or, again, you can feel free to just contact me directly. I also wrote a guide. I don't know that I've mentioned this before in this conversation, but I wrote a guide to help you build successful security champion programs. It's called The Security Champion Program Success Guide. So maybe we can paste the URL or something for folks, somehow, Chris, but that's a good, free resource. It's actually got most of what I know and all these lessons, I've tried to incorporate that into the guide as well. So I hope that helps you, as well, build out these types of programs.
Chris Romeo 51:52
Yeah, definitely. And I've read that guide and studied it, and it's got lots of good insight in it. So check that out from a champions perspective about Devici. You can reach us Devici.com we have a free forever plan for threat modeling. I mean, I get out of bed in the morning. I want more people to threat model. It's one of my goal, one of my life goals. So go to devici.com sign up for a free account, try out this threat modeling platform, see how it works. And I think you're going to find it will help you solve some of your threat modeling, if not all of your threat modeling, problems that you're struggling with right now. Thank you to this audience for engaging so actively in the comments. It was great. We got to a couple of the questions we planned, because we took your questions and that made this so special as an event. So Dustin as always, it's great to spend time with you. You're brilliant. Have to tell very quickly, I had a chance at RSA a couple years ago to have lunch with Dustin, Dr. Kim Wuyts, and Brooke Schoenfeld at the same time. I mean, it was mind blowing, mind blowing conversation. So it's always a treat to have a chance to chat with you, pick your brain and just talk about these things, because you think about them so deeply. And so I'm so excited for what you guys are doing, a Katilyst, where you're going. I think you're going to be a giant success. So thanks for going on this journey and this ride with us, and we got to do this again sometime the next couple of months, because it's just fun to chat with you.
Dustin Lehr 53:16
For sure. Agree, Chris, thanks for having me. Always fun to chat with you as well. The food was awesome at that place too. Let's definitely try to find a way to do that again. But yeah, thanks for having me. Thanks for everyone to show up today, and yeah till next time.
Chris Romeo 53:32
Yeah, thanks, everybody.
