March 24, 2020
Ep. #17, Leveling Up Software Delivery with Pete Hodgson
In episode 17 of O11ycast, Charity and Liz are joined by Pete Hodgson, an independent software delivery consultant. They discuss the DORA re...
In the latest episode of The Secure Developer, Francois Raynaud joins Guy to discuss the current state of IT security.
Francois explains why a cultural shift is needed to make security more inclusive, with security professionals taking on a greater mentoring and guiding role. Francois discusses why he created DevSecCon, a Development Security Conference aimed at fostering inclusion. He also shares approaches for DevOps and Security teams to better understand what other teams are trying to achieve so they can work collaboratively and improve business security.
About the Guests
Francois Raynaud is a Security and DevSecOps specialist with over 15 years international experience in FTSE 100 and Fortune 500 companies. Francois has developed products with innovative security capabilities across a variety of industry verticals including cutting edge SOC and CERT solutions.
Most recently, Francois founded a company called DevSecCon, which provides an annual conference dedicated to bringing DevOps and SecOps together, in order to create and deliver continuously secure solutions.
Guy Podjarny: Hello, everybody, and welcome back to The Secure Developer. Today we have with us Francois Raynaud. Welcome to the show, Francois.
Francois Raynaud: Thank you very much for having me.
Guy: And before we deep dive, can I ask you, Francois, just to explain a little bit about what is it that you do in the world of security, and how you got into it in the first place?
Francois: Okay, so I started working security 17 years ago, did all different kinds of work. I started as a pen tester, incident response, a great emphasis on building security operating centers and consultancy. So I spent about eight years in Verizon, which was a really good company to learn from. I learned a lot from a network security point of view, which led me to become a security architect, which I am now, dealing with a global online retailer, which comes with its own problems as well.
Guy: Cool, and when you say you're a security architect, you consult companies in that space or you work with a specific one?
Francois: So I consult companies and I've got different customers. All my consultancy is based on SABSA. The idea behind SABSA is using all open-source and the inclusion of security within the business. There is nothing worse than having technical controls, which are in place but we haven't got a direct lineage to a business requirement, and that's what we try to do as a security architect is just like go back to basics, and make sure companies are implementing the right controls without too much or too little as it is most of the time.
Guy: Yeah, that makes sense so basically you have to align the activity in the security space with certain needs of the business and how it operates, otherwise, you will never really catch or connect correctly.
Francois: Yes, definitely so it's always been the hardship of security people, which is like, "Show me what is the RI on your security product. It's about, I can't tell you about the RI but I can tell you what is going to happen if you don't actually implement those ones". The CFO is always going to ask you like, "Give me a price. Give me a value. Can you cost me an incident?" It's just about every incident is impossible to actually cost.
You've got some great mathematical formulas, you've got some white papers everywhere but nobodys ever been able to actually tell you from the start, I need to put that amount of incident responses in a company to protect, and to arrive to a certain RI. That's the main issue we've got.
Guy: Yeah, it's basically easy to make up a formula but between that and that problem of being accurate, that's a whole different story.
Francois: Yeah, basically for you, it's just like getting a realistic number, and just adding another zero to it. It's going to be cut off at the end so don't worry.
Guy: Yep, precisely. Cool. Okay, it sounds like you sort of jump around, and work with a lot of these different companies to set up security architecture, consulting, and kind of working with them closely, and alongside that, the way you and I got to meet was as part of an event that you organized called DevSecCon. Do you want to tell us a little bit about that one?
Francois: Yeah, that's correct. DevSecCon, which stands development security conference, the premise of DevSecCon was to promote inclusion.
Security from the beginning has promoted exclusion, which has meant a need-to-know basis of security clearance.
Don't talk to this person about it, don't tell them that, et cetera, et cetera. Don't tell them what the actual problem is. We're doing this investigation that used to be hush-hush but at the same time if you're actually being inclusive to your developers, to your business, you understand what does the business wants to do, what do the developer want to do, and how we can help them achieve this in a secure manner. So the aim of it was to demystify security, and get everybody in the room and have a good time. So far, that works so that's the third edition of what we're doing.
DevSecCon is basically myself and my wife. Luckily my wife is a graphic designer so she's doing all the design.
Guy: That explains a lot of things.
Francois: Yes, that's why it looks so good.
Francois: So I just do the content. I do all the promotion, and all the business side of it, and that's how we came to meet, and the aim was to bring people like you, who understand the problem, and actually want to work with development teams. There's nothing more frustrating than security trying to fix a problem in isolation without understanding the tools, which are being used by development, DevOps, and how they can benefit us so if you take, for example, DevOps, all the agile methodology, all the iterative processing, et cetera, et cetera, this is really beneficial for security.
Security used to be, well, if you go back to PCI DSS 1.0, which is you need to have a firewall somewhere. It didn't even say you need to have powers for the firewall. The dev guys will have understood that. They will tell you, "But yeah, you've got your firewall on the site." You need to put some power in it. But from a security point of view, we've been really good. We just defended ourself using compliancy, and just say, "Oh guys, you need to be compliant."
You need to have all these things in place, without understanding what does the business want to achieve, and that's one of the first question I was asked, which is, "What are you trying to achieve?" The other way to look at it is people will ask you, Oh that doesn't match my risk capital. I was like, "Okay. Define your risk capital. You want to lose one record, two records, 10 records. Does it matter? Does your cost to a data matter at all?" So that's what we've been trying to do is explain how security impacts the rest of the business, how many companies are just going completely under.
If you look at Code Spaces, for example, which was a training company all hosted on AWS, perfect from a DevOps point of view that was really clever. Unfortunately they left all the encryption keys onto the public packet of the S3 so somebody came in, they deleted everything, deleted the backups of the backups. The company went bust. That's a perfect example of just how we need to work together.
Security, it's not a blackout really. There are just a bunch of guys that just grow some beards, grow their hair long, and actually trying to do something better but it's time for us to come out of the dark rooms that we have been building around each other, and explain what we're doing, explain how we find things, explain how security can benefit a company.
Guy: Yeah, and I guess kind of learn the process. There's a lot in what you said that I'd love to unpack. I definitely relate first of all to the cultural aspect of it.
I worked in the field of security for about 13 or 14 years, and then I spent six years in the world of performance, and the delta between just the attitude, and embracing the community, and working with others versus against others, both the people that are a part of your community, and the people that are around your community was night and day.
That was actually one of the question marks I had in my mind when I was wondering whether I want to get back to security, and I feel like events like DevSecCon, which are still few and far between, and hopefully we'll have more and more of those, help bring some of that more positive attitude.
The one that is inclusive, that both embraces the other entities for the purpose of teaching them by making security everybody's job and everybody's responsibility but also everybody's understanding so you're not just following instructions. You understand that you know why it matters and how to do it but also the other way around.
I love some of those comments you made around learning from developers. There is a decent amount of ego in the world of security. There's no shortage of ego in the world of DevOps and dev but I think those worlds tend to be much more open for criticism, for constructive criticism than maybe some of the behaviors we see entrenched in the security space.
So it'll be good to kind of put that off to the side, and embrace some of the methodologies both because it'll make security better but also because there's really no other way. It's the only way to sort of keep security sustainable.
Francois: Definitely so another part of the work that I do is advice on product for certain companies. I can't divulge them unfortunately but the aim is just making them understand how they can build security in for the customers so those companies are DevOps companies. I don't need to explain the name but they're sort of a deployment mechanism, and they've got the ability to actually capture all the data that from a security point of view is really beneficial, and from a DevOps, at the same time, is really beneficial so it's just making them work together.
It's just opening the book and being more transparent so security has been great. We've had the security policy, which is a 500-pages document, and everybody has it printed on the desk, and the only reason why they printed, that's just basically to bash people around the head with it, which is not really beneficial. It's just like let's build it together. Let's actually transform the security policy as code. There are just no developers who will understand it. I don't need to hide anything, which is in security policy.
You need to bake it in your security product from the beginning. If you want to fix a security problem or implement security later on, its going to cost you a fortune.
If you just build it part of your requirements, your job is done, not completely. You need to make sure that it gets actually implemented.
Guy: Of course but the cost is dramatically different, and I guess the other beauty of that statement is you can swap security for many other aspects of software quality, and that statement would hold so it's something that people should be able to relate to.
Francois: Agreed. Totally agreed. So I was speaking to a security vendor yesterday, and they were just, "Oh I've got this technology, its a silver bullet", and it's just like, "No, it's not. How is it? How do you anticipate your customer's need? How do you know the financial data? How do you know the flows, the data flows, what regulation, et cetera, et cetera?"
A tool is good but it's just a tool. You can actually build tools. I can't build tools so that's why I rely on people like you to actually just make this conference great, and make the security better but it's just simple ideas, which is from an incident response point of view, I need data, all the data. I need to know everything in order to be able to understand how did this incident, how did this attack actually occur? How can I prevent it?
So if you look at incident response, everybody is talking about those five steps you identified to the point you reiterate, et cetera but the sixth step is really important, is key, which is your lessons learned, which is what did I learn, and how can I ensure that it's not going to repeat itself?
And don't just publicize for yourself as security. You're just, Oh wow, I've just given a really good root cause analysis report to my CSO. It doesn't help him. The person who would benefit from it is going to be the developer, which is like change your application, change your processes, change your performance monitoring, change your monitoring to actually just get this feedback loop as quickly as possible.
People are talking about the shortage of security people but actually what is lacking are security people who are willing to be transparent.
There is nothing special about security. It's just the only thing we've done is read a book, plenty of them.
Some of us actually have written some books, which is really good, which is a way to actually promote this inclusion whereas we're talking a bit earlier, which is security has always been about exclusion, which is the need-to-know, security clearance, and now DevOps and development is more about the inclusion, which is how can we work together, how can I make you build a better secure product from the start.
Guy: So in this event like this all resonate with me very much, how do you look at the two different audiences, right? You're talking about developer and security, DevSecCon, and inevitably you're probably pulling in some people from the security space that want to broaden their ability to reach out too, and better understand, and better interact with dev.
I think that's maybe the given. That's probably the starting point here because security people really have no choice here. You live in this world. There are developers around you. They will create software, and there are new and modern ways, and you sort of need to accept that. How much did you seek sort of uptake from the dev side coming into DevSecCon? How much does that play a role, and how do you see that progressing forward?
Francois: So the first DevSecCon, I was working at the time with a really DevOps team into a gambling company, believe it or not, and I was really blessed by the ability to communicate, identify a problem, and willingness to fix it as quickly as possible. We're security. We'll put it on the backlog. We'll put it on risk register. It's in a risk register. It's fine. Who's looking at the risk register? Nobody.
Guy: Yeah, ever.
Francois: Because you haven't got a risk officer, which is actually just driving this in, so the first DevSecCon, we had really a 50% DevOps, 50% security but the common denominator was operations. What we realized, it's just we've done a show of hands, and it was just like, DevOps, half of it. Security, half of it. Operation, all the hands were up. And that was the main problem we were trying to fix, which is from an operational point of view, how we can work together. So the point of security or my point of security is that from the inception in your pipeline to your delivery, it's not that it shouldn't be zero defect.
Defects are going to be found. Security people have got enough time on their hand during the night to find some new vulnerabilities.
What is important is zero variation from your immutable object to your product.
That's what we're looking for, which is like how can I measure that I've got a consistent product without adding new functionality part of it where I shouldn't be looking for defects. Defects are going to be found somehow so there is going to be people like you, me, which are going to be looking at can I exploit this function, and that's the main issue so I think DevOps is now embracing security.
We've got to be thankful for Gene Kim to actually just started this trend but one of the things that I think was lacking in the last book was more emphasis and now emphasis in security, which is how can we work together? It's not hard really, and especially a person like this that used to be one of the founder of Tripwire totally understood from the beginning, which is security people needs this data, and from a DevOps, I've got this data so let's exchange this data. It's pure and simple. It's just about inclusion.
Guy: Yep. I definitely can relate somewhat to Gene Kim's work. There's a quote from him or from Josh Cormaney, I keep mixing it up, talk about how to fix security, you have to leave security. There is a lot of ways to interpret that statement but it includes expanding security outside of security but it includes you as a security person leaving security for a moment, seeing some other things.
I think I relate to it just having done that a little bit, having spent these years in the world of perf. There is a lot to bring in. I'd like to maybe drill a little bit more like you mentioned a few things here. One is this notion of DevOps versus SecOps. I already mentioned SecOps in your job description earlier on, and also now talking about how the audience is all operations and working together, and a little bit about the evolution of which path is the right one, or how would you put the emphasis.
Do you think the bulk of the work right now as an industry is around getting the DevOps teams, the dev, the ops teams to embrace some of these security responsibilities and do them, or do you think the bigger opportunity is actually to get security to open up to do that type of sort of education and outreach, and that the ground is already kind of ripe for absorbing that information, and it's more about getting it out there.
Francois: It's funny so I've got two different customers, which I've taken two completely different approaches on this so one of them, they actually just disbanded the old security team with the hope that DevOps will be able to learn security as written on the back of a fat packet. Good luck. Not really going to work but like all the time, you just need to wait for an incident for things to get corrected. One way to do it is actually, so DevOps is here to stay.
Security was here to stay but I think there is a way where we can change the attitude of security, and go to a devolved model where we're embedding security part of every function so as a security architect, I work mostly with all the enterprise architect, which are not security-related. But it's about me inputting everything security-related within the enterprise architecture to help the product, the company develop and be secure. What security needs to change is its attitude.
Security has been the echo chamber of security. We just love talking to each other.
We say "hey look at this new tool. I can break everything. Can you fix it? Not really." But that's not the problem, then you end up into your responsible disclosure, et cetera, which in my opinion is really important. You need to disclose responsibly every security issue that you found, and even more, you need to work with all your DevOps and all your developers to actually fix it from the start.
It's not hard to actually talk with people. It's not hard to actually identify what is the security debt that you're carrying all the time so the pen tester, typical pen testers will arrive, identify a bunch of vulnerabilities, then go back home. Been there, done that, and it's fantastic. You haven't got to fix it, and then slowly, slowly, they ask you but what is the fix, what is the remediation, and we point them out to an RFC like totally obscure numbers. Read this RFC at home. It's all written and explained in there. That doesn't work.
It's all about translating those security problems into the world of DevOps, the world of developers, which is I need to learn every day about developers. I need to learn there are new technologies during your talk at DevSecCon. Actually I had to google quite a few of the words because I was just like, Wow, I didn't know. I'm just like I stop with C++, and that I think security has been used by the rest of the business as the people monitoring for things that can't be fixed, or that people don't want to be fixed. We're going to call them the security debt.
Coming from a SecOps, which is security operations, we were used to create intrusion detection rules to actually monitor for those vulnerabilities that nobody wanted to fix. You've got to deal with this security debt because yes, you're going to write on your risk register. You're going to encrypt your risk register, and suddenly this employee, they actually left, and maybe you were not really nice when you gave him the last cheque, and the redundancy was not enough. He's going to come back, and he's got this data, and he knows that two years, three years down the line, you still haven't fixed these vulnerabilities.
It's not going to take a lot for this person to actually go and publicize it so treat them well, make them work together. From a security point of view, you've got to use all the tools that developers are using. You've got to use the backlog ability.
Now we'll identify vulnerabilities actually directly input them into their backlog so that could be Rally, and that could be any type of bug tracker that you're using so you're actually using the tool, and you can actually predict your security post-job of how it's going to be in three, four, five weeks of course if everybody is respecting the process but that's all down to that. Security is not hard, it's about understanding your processes, understanding your requirements, and implementing them securely.
Guy: And as a part of the existing processes. Yeah, I think I agree. I do feel like in my mind there is a little bit of a bias around maybe where the biggest gap is but you need it to play together so security and I think this is also what you're saying is a security's job needs to become much, much more of a mentor and a guider.
You shouldn't have dedicated security-related tools in the company. You want the tools to be part of the existing stack. They could be serving a security purpose but they shouldn't be exclusively used by the security team because you actually want them embraced and used by the rest of the team.
And then you want security to be sharing and exposing the work that is done with the rest of the organization.
Francois: Definitely so one of the best achievement was actually just to relinquish ownership of security tooling within the security team, and actually give the tools to DevOps. Yes, you can run vulnerability management scanners. You can run all these because you're the one actually looking for the feedback of those.
Yes, I'm going to be able to tell you that's bad, that's actually really bad, but you're the one who's going to fix it so my rule as a security professional is to teach you how to decipher those security coding works that we've put in place.
I like to do the same when by being French, I had to learn English. Now I wouldn't be able to give you a technical conversation in French, and security people ended up into this inability to explain what they're doing by creating those buzz words, and people don't understand it so you arrive, which is now we're trying to make it sexy with those POODLE, et cetera, et cetera but if you actually underline what it means, nobody understands it at all.
Guy: So I think all this conversation goes back, and explain what the role of the security professional or the dev or DevOps person needs to be in the space. Maybe lets sort of backtrack, and talk specifically around the activities in the community, and sort of back to DevSecCon, and just some of the work that you're doing with OWASP.
I find in one of those spaces, right, I'm a security person that wants to sort of get more engaged with dev or the other way around. What's coming up from a DevSecCon, from OWASP that I can get involved with?
Francois: It's quite finished.
DevSecCon is the only conference which is actually looking at this inclusion of security in DevOps so if you want to be part of the change, that's the place to be.
You've done it last year as a presenter, and that's really good, which is really just giving away your knowledge so what we're doing is we've been able to keep the prices really low to make it affordable to everybody to come. The aim is not a money-making machine, the aim is providing as much education as what we can so we started in London.
Personal reason, I'm moving to Singapore, and what we're doing, we're actually creating DevSecCon Asia so I've got two partners over there, which are working on this so at the moment, we're going to be looking at two events a year, one in London, one in Singapore working with different colleagues on DevSecCon's community to actually bring up another event in California. You might say that what I'm trying to do here is to do an endless summer, and you're going to be perfectly all right so that's the aim.
Guy: It's not a bad goal.
Francois: No no no. I spent 15 years in the UK, and looking for a bit of sun now. It's just completely white. It's starting to be a bit transparent but we're doing those events so it's going to be three, four a year. I'm trying to make it as low cost as possible so everybody can access.
The other work that we're doing so I'm really lucky to be working with OWASP, which are really good, and what we're trying to do, what we brought up is what is called the OWASP Summit. The OWASP Summit occurred in 2008, 2011, and this is five days of interactive work so it's not a conference by itself, it's basically we're spending five days in the villa but there will be plenty of villa because there would be about 150 of us.
This year, we're doing it in Center Park actually just outside of London. That's from the 12th to the 16th of June, and it's five days of just inclusive working on workshops so we've got about 10, 12 different kinds of topics, and you're coming here to actually give your knowledge away so type of things we're going to be looking at is going to be responsible disclosures, threat modeling but also lambda security, redoing all the OWASP cheat sheets, doing a bit of cleanup on the type of project that OWASP has been doing.
OWASP is a really great community, and without them, I don't think security would be where we are at the moment.
What we need to do now is instead of this one person talking to everybody and everybody listening like we've all done at school, it's just let's all talk together so we're bringing all the best people from around the world, and we're putting them in a room. There's going to be some foods, there's going to be some drinks, there's going to be little sleep but there's going to be lots of really good information, lots of resources, which we're going to make available to everybody for free.
Guy: That sounds great, yeah, and I think to an extent, that's also again appeared out of the DevOps world 'cause DevOps is also like a few too many men versus women that sort of let their beards grow, and I sort of saw how that evolves but I think a lot of good practices came out of that in this type of anywhere from sort of summits to unconference to sort of collaborative work and output. It's definitely a key there.
Guy: And a good evolution for OWASP.
Francois: Definitely so if you look at Amazon re:Invent so it started, it was really good but if you go to it now, basically.
Francois: Yeah, it's Amazon inviting some people to talk about Amazon so you couldn't be more narcissistic, nonrealistic as you will say in French. It's really this person looking at himself, and that's what security used to be so DevCon, Black Hat, we all done it, and it was just fantastic. It's just like five days of lots of fun especially in Vegas but did we teach anything to anybody? Not really.
The only thing we taught them is just like, Be careful because your car is going to get hacked. It's just like, Oh brilliant. What is the solution?
We haven't got one. We didn't think about that. That's not what we're getting paid for, and the DevOps guys are doing things differently so I'm trying not to mention the companies here but some of them, which are part of this continuous delivery, they are providing more open-sourced connectors than any other companies are doing.
Sometimes you're actually wondering, Where do you make your money? How did it work? But they're giving to the community so one good person I'm working with is Gareth Rushgrove. He's been coming to talk at DevSecCon, and he's been really explaining, "Just look, guys, security have got to be transparent". He made it really clear as in like, "Security, please come and talk to me. I can build the tools for you". Because security is not necessarily there to build the tool to make things better, we're just here to break things. That's got to change.
We've got to really look at the problem in a different manner because if we did enough compliancy, and all those different regulations, I don't think the security industry would still be here today.
Francois: We kind of failed, and the premise of DevSecCon was to completely change this. It was to give the information to people so we can work together. Ideally, I will make myself redundant because I've taught so much to DevOps, and that happened in a different company.
Guy: I think we also know that sort of in this world, that's never entirely going to happen.
This ridiculous talent shortage we have right now in security exists because we assume a certain sort of distribution of work which should be different.
Now I'm not saying redistribute the wealth here but I am saying redistribute the work, and ensure that just one more of the security knowledge and the security activity gets disseminated to be a part of sort of built-in application.
The remaining still very large amount of security guidance, security analysis, security understanding is probably still going to remain in the hands of expert but definitely the ops part of it, the DevOps, the SecOps needs to be a part of the regular ongoing, and the consideration of it needs to be sort of embedded in every action.
Francois: Yeah, so from a SecOps point of view as we're touching on it is the only thing you're doing is like DevOps, you're responding to an incident. The incident might be a server that came down but in that case, that's an attack.
Okay, so the first analysis can be done by DevOps. How you gather this data, what data you're looking for, you just need to tell your DevOps what they need to capture, then okay, maybe you've got a forensic investigator. That's a specialty but is it really security-related? Not really because it's.
Guy: It's analysis.
Francois: Exactly. It's analysis, and anybody can do it. I've done it. I'm sure everybody can do it.
Guy: Cool. I think this was a really interesting conversation. We talked about the role of security operations versus ops as a whole or sort of DevOps, and how those are, in many ways, sort of two sides of the same coin.
We talked about DevSecCon and the upcoming OWASP events, and all of these great spaces that hopefully will help all of the people listening to this participate, and kind of play a role here, and help contribute some of their security knowledge or development knowledge to be a part of this ecosystem, and I think also some interesting observations around the role of the security person as a mentor versus an operator, if you will.
How does that work as a food for thought. I guess before we part, can I just ask you a question that I ask every guest on this show is if you're talking to a dev team that is looking to sort of uplevel in terms of their security posture, what's your sort of current pet peeve? What's sort of the one practice you recommend they look into to get better at this space?
Francois: Sit down together with Security, and that's the same thing, security needs to sit down with DevOps. We both have got to learn something from each other so that's one of the first thing I do is I bring those two teams together. I make them sit together. Go and have a drink. Go outside, party. Just go and have some dinners together. Understand what the other side wants, and understand what you can bring to them, and I think it works both ways so if we want to make security and dev and the overall security area better, we need to promote this inclusion.
Guy: Yeah, I think that's really good advice, and again, kind of has demonstrated the work in the DevOps world, break down the barrier, fundamentally it starts with people.
Guy: Perfect. Thanks a lot, Francois, for coming in the show.
Francois: Thank you very much.