In episode 38 of The Secure Developer, Guy speaks with Andy Ellis, CSO of Akamai. They discuss streamlining customer assurance, the role of an incidents coordinator, and the value of transparency between a security company and their associates.
Guy Podjarny: Hello, everyone. Welcome back to The Secure Developer.
Today we have a good friend and a CISO that I've had the pleasure of being a colleague to, Andy Ellis, who is the chief security officer at Akamai.
Welcome to the show, and thanks for joining us.
Andy Ellis: Thanks for having me, Guy. I really appreciate it.
Guy: Andy, before we dive into the deep details over here, can you just tell us a little bit about yourself, the history, and how you made your way into security in the first place?
Andy: Sure. I'm the chief security officer of Akamai and I have been here for just over 19 years now. I got my way into security by a very roundabout method.
I was at MIT working on my degree in theoretical computer science and I was in the Air Force ROTC program, and for those who aren't familiar with that, that's an officer training program where you get your commission to go into the US armed services once you have your degree.
I really wanted to be what's called the "Weapons systems officer."
You might think of it like the navigator on a large plane, it's the person who sits in the back seat and provides guidance to the pilot in the front, maybe you get to drop some bombs once in a while.
I couldn't be a pilot because I didn't have the vision for it, and while I was on my summer you get to go do these exciting jobs in the summer.
I was down at Luke Air Force Base sitting in the back seat of an F - 16 and I got a phone call from this major down in South Carolina, and he's telling me about this new squadron that does information warfare.
I'm like, "That sounds really cool."
Recognize this is 1996 and this sounds almost like a job interview, which if you've never been in the military you have to understand you don't really get job interviews, you just get assigned to somewhere.
I was being assigned and he was just calling to make sure I wasn't completely crazy, I didn't have a choice about whether I took this job or not, they'd already decided.
Guy: A different world.
Andy: A very different world. They were told they could have anybody they wanted in the Air Force, and they said "We want everybody graduating from MIT with a computer science degree."
That was me, so they had to fill in with other folks.
I showed up and this was working with a commercial intrusion detection system, for people who remember the net range that Cisco bought and border guard that was run by Network Systems Group, that we were deploying those systems all across the Air Force.
I was responsible for building and configuring and designing the defenses and working with our ops team on how we're going to make this work.
I learned an awful lot about security in those years, and then came to Akamai a few years later and I've been doing security here ever since.
Guy: Interesting. You basically got assigned into security?
Andy: I did.
Guy: It worked out well.
Andy: It has. When I came to Akamai we weren't a security company at the time, we were at a CDN and we were just barely starting to talk about performance instead of just offload.
So the transition over these 20 years to now a company that is security first, where performance and offload is still part of the value proposition, but from a go-to-market perspective we lead with security now.
Guy: Yeah. That definitely is a transition. Also the size of the company, how many people are roughly in the early days, if you go 18-19 years back?
Andy: When I came in it was around 500 people or so, we grew to about 1,200 right before the dotcom bubble crashed, and we came down to 550.
That was not an exciting time, I was in charge of doing all the technical coordination for every layoff and now we're back up to about 7,000 people.
Guy: OK, that's also quite a difference. Not the same job when you're running a CISO for 550 vs. 7,000 people.
Andy: It's amazing, there are things that happen now that I'm like "I used to be the person who did that," and I'd be pulling out my hair to get it done.
Now I don't even see it happen and it's running smoothly with folks in different organizations, and to see that maturity grow it's been actually really nice when I notice it.
The trick is to go back and actually notice it.
Guy: Yes, see the change. So let's do that a little bit, let's talk a little bit about the security organization and maybe a little bit about its evolution so people get a glance at what it looks like in different sizes.
Can you give us a bit of a rundown of how your-- You're the CISO, how does your security organization get divided? Who does what?
Andy: In Akamai we have a very different structure than I think a lot of other organizations do.
Principal operational security responsibility we embed into operational teams, our enterprise security are the folks who are securing our corporate network actually works with the CIO.
We actually spun that off to the CIO organization about 10 years ago on the premise that if you're responsible for securing someone's infrastructure and dealing with the alerts coming off of a system, you should be part of that operational organization.
Guy: This is an incident response element, or more you mean the Enterprise? Like, the employee security and the likes?
Andy: Yeah. The endpoint detection and response, so selecting what tool is going on to that device and then dealing with the incidents that come out of that, that's happening in our enterprise security team inside the CIO organization.
They have a dotted line to me, but their solid line is directly up to the CIO who's really responsible for ensuring he's staffing that correctly.
What's amazing is that it actually gets more money now that it's not under me than it did when it was under me, because the CIO really does care about that and now has direct control over it.
It's not just in this nebulous global infosec pot. For our production services, we do very similar things, we often don't have dedicated security teams as you alluded to.
We have a GNOCC that is our global network operations command center and they do Tier 1 and Tier 2 security work.
If there is an alert that says "There's a problem with the machine," they're the ones we're going to try to take care of it first and they'll do the preliminary investigation if they discover something interesting.
Of course we're going to escalate that into an incident, and incidents are all now coordinated again out of the operations team.
That's actually a change for us in the last few years. The incident management program at Akamai historically was very engineer-driven.
When there was an incident we would tap a relevant engineer, usually an architect or maybe a interim manager and say, "Look. You're the incident manager, build your incident team, and you're responsible for managing and running the incident end to end."
That worked really well in our early days. Small organization, it was the same 30 people that you'd tap all the time, but where we have so many different products and technologies now you'd have people who'd be tapped to run an incident maybe once or twice a year.
We found that they didn't know how to run an incident, like they'd been through the training but it was their first time, so you get things wrong.
What we did was we created a new role in our operations team called "The incident coordinators." All that they do is run incidents. Now, they're not supposed to be the technical expert and they're not the decision maker to say, "Should we do X or should we do Y?" But they're the ones who make sure that when that question comes up it's being made by the right person, that they're thinking about the consequences, they know who to coordinate with.
Giving us the professionalism, and then my team backstops them.
The incident coordinators manage almost all of the incidents, my team still handles a few, but we're the ones who are responsible for governing that whole incident process.
Whether it's a security incident or an incident for anything else, my team still does the governance but more and more we've pushed into the operations teams for the follow up work, because that's really operational work.
Our job is to make sure the safety of the platform is robust, and as the incident process has become less ad hoc and more robust we're doing less and less of it.
Which makes my team very happy, frankly.
Guy: Yeah. I'm sensing a theme over here. Enterprise security want people running the enterprise infrastructure, people that handle the incidents and operations also handle the security incidents and operations.
You overlay and support them over it?
Andy: Right. My organization now really has split once and then one of those split a second time.
So one split is the product and system security split vs. the go to market side of the house.
We have a small team that does research on attacks against customers, publishes those through our state of the internet report-- I'll do a small plug for that.
We finally got it out from behind a registration wall so you can just read it directly.
There you go out and write a number of different things, press engagement, and the goal there is to say "As a security business, we are part of the go to market function of the business."
Think of it as the product marketing of Akamai, just the security company, not associated to a specific product. So do research, publish, sell, etc.
Guy: In many cases, there is more of an advocacy and an education and a reach-- A group that's under you.
Andy: Under me.
Guy: Because that home for security expertise in the company, as opposed to sitting in an advocacy or a simple marketing organization, or the likes.
Andy: Right, but we partner closely with the marketing and the sales teams and we backstop our sales organization if a customer says, "Why should I trust you?"
If I send somebody to talk to them and say, "Here's what our controls are."
They just believed more than if a sales engineer tries to tell them the exact same words, and partly because they do know more, they can go more deeply.
If somebody says, "We need special contract terms," I have my own lawyer that works for me that partners with our legal team.
The legal team are the ones who can approve language, but we're the ones who can translate between what reality is, what a customer is looking for, and what that language will be.
It's been fascinating and so successful. What we find is customers just want the assurance and they often don't know how to ask for it.
If we put somebody who is in the same role on our side in the room with them, we can often get them to assurance and then they're comfortable, and now we're just negotiating language.
Whereas if you try to get assurance through a language negotiation, you end up in some really dark, ugly corners.
That's one piece of art, and the other piece is really the product and system safety teams.
Their job is ensuring our systems are robust and safe, that they operate in the way that we want them to.
We split that into one team that focuses on assurance, so they're responsible for a lot of our compliance activities and they'll partner with specific engineering teams on bringing systems up to standards, whether it's PCI or Fed Ramp or SOC 2
It's security with the focus on the compliance. The other piece we'll call system safety and resilience.
The job of that organization is really to focus much more deeply on given technologies, and say, "OK. We're launching a new system, how do we know that it's safe?"
And the evolution that we've done there that is fascinating, because we can't scale our safety and security architecture as fast as the rest of the organization, that's gone.
While we'll still do very deep partnering on specific products, we've actually built a system for development teams to self certify.
For them to come in and say, "I need to a security review."
What we do is say, "Look. Give us one architect from your organization, we'll hand them a guide that says, 'Here's how you do a security and safety review.' They'll do the review with you, you write down your results--"
I'll come back to that in a moment, "And we'll just vet what you wrote down, so when you go to product launch and everybody looks to us and says, 'Was a security review done?' We're just going to answer based on the review you did."
Here's the dirty reality that I didn't really understand until much later in my career, because in the early days as a security person you're told "You want to be the gatekeeper. Nobody should be able to launch a product without your approval." That's horrible advice, because only one person can choose to launch a product or not. It's the CEO of the company whom I delegated to a product, vice president or president. But it's not really delegated much below that, and you can't split that authority.
I had it for a while and I remember there was a product that was launching that used MD-5 as a checksum on one of their messaging things.
Now this was at a time when nobody should have been using MD-5 any more, at least not for anything new.
It was right when the deprecation notice comes out, but we've had better things for quite a while.
We weren't quite to the Poly1305, ChaCha20 world yet, but we were close.
I was like, "This shouldn't launch because they didn't know enough to even use the right library here, so I'm worried about other things."
And literally, the product can say "This is maybe $1 billion dollar product." It wasn't, but that always is what a product engineering team will decide.
They're saying "Andy, are you really going to hold it up based on this one thing?" And I'm sitting here and everybody is focused on me.
So afterwards I sat down with the president of products, I said, "Look. Here's what I'd rather do, I'd rather we tell you if we think they're doing good risk management or not and you have to decide if the risks are OK, but they have to write them down for you."
And he was like, "OK. We can try this."
The very next review came in and they had done what every other review had done, which was shown up and told us three days before that the review is happening and "Can you please review the product?"
Before we've made this change I would say, the launch media would say, "I don't know enough to tell you if this is safe or not."
And everybody would jump on you, "How fast can you do your review? We really need to launch this."
And it became my fault, and afterwards I said, "Look. I'm going to fail this because they don't know what their risks are."
It was this instant changed. The person making the decision said, "Great, they're not launching."
Guy: Because they haven't really done the review on it.
Andy: They haven't done the review, and they were like "Wait, what?"
And people don't come back now unless they've done this review.
Now we let them do the review and there's a fox guarding the henhouse problem, is what everybody who do I tell this to thinks.
They say "Oh my God. Of course they'll hide things."
The reality is developers aren't really interested in hiding things.
If I tell you, "Tell me the unacceptable losses that your system could incur and the hazards," almost every developer is like, "Yes. Let me write these down, because I've been wanting to tell somebody what the problems are."
And it's not saying you can't launch with risk, we're just saying you have to write down what you know your risk is.
Then all of a sudden people are like "That was really bad. I didn't like writing those words down. How do I change this in the next release?"
This iterated engagement around making people accept their own understanding of risk, that "It's not my problem, it's yours. You're the one who wrote it down."
All of a sudden it has dramatically shifted how people operate.
Guy: This is very much in line with "You own it, you secure it," type elements.
How do you overcome the continuous nature, or even more in line security controls? Or maybe a level of expertise, because this is the security review.
It's a new system, there is a maybe an opportunity for more comprehensive review?
But many changes happen that way, many changes happen day to day after that system inversion and 33.3. How did they get embedded over there?
Andy: A piece of the goal is by being part of the original launch process, and major changes will go through that, by forcing people to write down their own risks it means that when they're the ones who are doing that 33.3 release of course they're not going to tell us everything in it.
But they have now gotten practice itself with reviewing, and even if the normal remodel is "I write it, you review it even though you're my partner architect."
But I'm doing that for you, when you write something I review it, and now when I write something in 33.3 I'm like, "Maybe I can do a little bit better."
Because the goal here is to get a little bit better every day, not to be perfect. I think that's where many security professionals lose sight of how to operate.
Our goal is not perfection, our goal is not risk elimination, our goal is to enable our business partners to make wise risk choices because that's what we're in the business to do.
We spend money, that's a risk, we might not make any money on the other side of it. That's like the very definition of risk. If I can see that you're spending the money badly, like you just grabbed the wrong open source library, let me help you grab a better one even if it's not perfect.
We do restrict some things around crypto, we do have rules that say "We don't care if you think you can review it yourself. You don't get to write your own crypto."
There's some code lines there that we're very careful about who's allowed to do things, because we're also contributing a lot of our stuff back into open source in that area.
So it's not that we're completely "Go run it yourself," but our goal is to empower the organization.
Because first of all, we just can't afford as a company to keep trying to buy specialist security architects.
I obviously need more than I have, but that's a true statement every CISO will make.
Guy: Across the organization, across the industry.
Andy: Yeah, and they don't exist.
But if I can take every software architect and enabled them to do a better review than would have happened if we tried to centralize it, because the thing we have to remember is that my architects have to build up a model for your system before they can review it.
Your architects already have that model, I just had to teach them how to apply security and safety thinking against the model they already have.
Guy: Yeah, absolutely. I think that's the advantage of any security embedded into an owner of a system above and beyond the scale and the natural scale element of it.
There's just an intrinsic understanding of the thing that is being protected, be it an app or an organization or a set of servers that have an incident on them.
Drilling one step deeper, you have that dev team and they now understand they need to build the competencies.
Oftentimes there is at least two recurring components over here, which is tooling and training. Who runs those?
Who owns ensuring that people have the right tools at their fingertips and know how to use them?
Above and beyond the questionnaire or the review of knowing what question to ask?
Andy: We own that strategically, but very rarely do we own the implementation.
I'm responsible for identifying "We need a better tool around vulnerability management tracking," to pick a thing that's right now a hot topic.
I'm pushing the data across the business, I'm partnering with our development tools team, we have a team inside engineering that builds tools for the rest of the developers.
They're going to own whatever we settle on, but I'm the driver for them to say "This is important. We're making this a strategic priority so that we can support all of the development team across the organization in that one central spot."
So I'll develop that for training, and we actually went through this because I had to do secure coding training across the business.
We went and bought from a couple of different vendors, different training packages, and I will tell you they were universally panned by our engineers.
They were not happy with the training, so what I did was one of the people who works for me Eric Cobryn who runs our go to market functions. I think you know Eric.
Guy: Yeah, he's awesome. You've got quite an asset inside that can do a lot of those types of educations.
Andy: Right, so Eric put together a secure coding training and then we've augmented with different people in the team.
We built our own secure coding training and our engineers love it. We deliver it through our engineering learning organization, so it's not this huge resource drain on my team.
They don't have to worry about doing individualized training and scheduling things, we have another org that does it, we just became basically the internal vendor for that training.
Not only does it save the company some money but it's actually better training that targets our development practices instead of something that was more generic from an outside company.
Guy: Yeah, definitely got onto your settings.
I like making the most out of a different type of skill set that you might have, because maybe unlike some security teams you have people in the company that look to educate the world about security.
So you can use that and aim them internally.
Guy: Got it, so the tooling is done-- It's run by the dev teams and it's somewhat central still, but not necessarily the security teams.
Guy: So it's still aligned with dev tooling, but dev tooling is not team by team, it is somewhat central so those teams run the security.
But still rolls along the principal of "You own it, you secure it."
Andy: Right, and we'll help you do that. Now I do have my own tooling team, but that is mostly for things that we operate ourselves.
Our compliance system actually runs on a dashboard we built ourselves, because we went and looked at the whole industry of compliance and we did not like any of the tools that were there.
Most of them are not designed for people who build their own software as a service platforms where it's not about "I'm just integrating other people's technology and then I have to turn around and support it."
That just really wasn't the model of many of these other programs, so we just built our own.
It's fantastic because every document has owners, so we can just say "You as an interim manager, here's the five documents that describe what your system does and how your organization operates. Once a year, please review these."
Then we plug them into all of the different compliance frameworks that we support, so when our auditor comes in for PCI maybe they get four of those five documents, but when they come in for SOC -2 they get a different point based on what those regimes wanted.
Guy: A lot of it is about managing the data and the compliance bits on it.
A lot of the action is still handled by the rest of that org, but the consolidation of the status is done by that internal--.
Guy: So let's switch a little bit to the side and talk about visibility.
I think you're well known and I've experienced that firsthand with being at Akamai that you're quite transparent.
You oftentimes tout visibility and transparency in security, which is not the default case for many people in security.
There was a lot of "Don't tell them."
Can you share a little bit of your perspective?
How do you balance transparency and what value do you see there with maybe the risks associated with maybe showing the weaknesses or showing where are the holes?
It's a really fascinating topic, because you have to understand how humans make decisions to really grasp why transparency is so valuable.
It's not about trust, although trust is a wonderful thing that you get when you're open and transparent.
Humans have a set point of risk that they're willing to take, that's sometimes called "Risk compensation" or the Peltzman Effect named after an economist, Sam Peltzman.
It basically says "If you give someone more risk, they will automatically retract from it and do things to compensate and reduce their risk. If you take a risk away from them, they'll engage in more risky behaviors because there's an aggregate amount of risk that they want to take."
This really came into the popular imagination in the 80s in the United States when we were debating national seat belt laws, and Sam Peltzman said "Look, if you make everybody wear a seat belt it's a safety device that will make them feel safer, so they will drive more dangerously, which means that while they'll be safer in an accident, accidents will happen at higher speeds and they'll kill more pedestrians."
If you want to have fun, go look up the National Highway Transportation Safety Administration data for the last 40 years and look at the fatality rates per mile driven for drivers, passengers, pedestrians and motorcyclists.
It turns out Sam Peltzman was wrong, motorcyclists bear the brunt of increased safety systems because they can't get safer but the cards have shifted from driving 60 miles an hour normally to driving 90 miles an hour.
Once you really grok that, that humans react to knowledge and awareness of risk, it explains why you want to be transparent.
If you have a system and you say to me "Andy, is my system safe to launch?"
First of all, this is a setup because you're launching no matter whether I say yes or no.
But let's say you hand it to me and I go and I can spend however much time and create a laundry list of a thousand things that are wrong with your application, and they'll range from minor details to massive things like "There's data breaches waiting to happen," or worse "Lives that you might actually just lose if this gets tickled."
But I know about them and you don't, and you're the decision maker so I've taken all of this risk that you should know about.
Maybe it would be simple if you said, "Look. I'm just not going to sell this to health care because I can't afford the life safety issues."
If you knew all these risks, you might say "This is a great app that I'm going to try out with my startup, but I'm going to target whatever my niche market is where if I fail nobody's life is lost."
I would tell my sales reps, "You are not allowed to sell to a medical provider or to critical services," or whatever it is.
But if I know those risks and you don't, you'll just go ahead and sell to them, and it's not that you're being reckless.
You don't know, so my being transparent is all about enabling the people who are day to day making these decisions, choices about risk, enabling them to make those in the context of the world they live in.
That they shouldn't walk through the world with blinders, and security professionals who are not transparent are literally putting blinders on their business partners and then getting angry when the business partner walks into a dangerous situation.
I love when the human philosophy really is what drives activities and drives those reasonings. You see this a lot in the dev ops scene.
A lot of being around incidents and around responses really boils down to "How do you expect humans to react when you put them in this?"
Andy: The thing I love about dev ops, and I argue that Akamai was dev ops before dev ops was a cool thing.
The most important thing is that when a system breaks, the person who built it is on the hook to do the incident and fix it.
The value that gives you is that they now have a personal incentive to reduce the safety risk of the system, they don't want to be called in an incident.
If you have somebody who can push a change at 5PM and walk out the door and somebody else has to absorb the cost of that breaking, what's ever going to stop them from pushing a change at 5PM?
But if they push it, they walk out the door and I call him and say "Get back in the building, you have to fix this or get on your laptop. I don't care that you're having dinner, you broke it you have to help fix it."
They will self-decide to stop pushing changes at 5PM and walking out the door, or they'll figure out how to make those more safely, or they'll implement controls that will automatically roll back.
I don't have to tell them how to solve the problem, I just have to expose them to the real risk.
Guy: Yeah, absolutely. I think again, aligned very much with the perspective of "Own the risk, see the risk, own the remediation. Be able to do something about that risk."
Andy: I think you're really tagging with "Own the remediation."
This is something I learned from one of the people who works with me Kathryn Kun, who runs our adversarial resilience team.
She pointed out a long time ago that there is a never ending list of good work available to be done and that we shouldn't pick for someone.
If I have twelve things that I would love for you to do, and I know you can only do one of them, I shouldn't pick the one that matters the most.
Maybe I trimmed it down and said "Guy, here's these four big problems that your organization is wrestling with. You really need to solve one of these.
You can pick any one of the four and I am happy." Now it's your project, not my project.
Whereas if I walk in and I picked one you are just going to resist it naturally, no matter how much you care about the organization, because you're not part of the decision.
You didn't own it, I owned it. So I give you four, you pick one, now look how transparent I am.
I'll tell you about a whole bunch of problems in your organization in a way that is non-confrontational. I'm not saying "Here's four things. You have to fix them all right now."
Guy: You give me a choice.
Andy: I give you some choice.
Guy: Not to belittle it, but I'm always astounded sometimes on how the same philosophies apply to a 7,000 person organization and your kids because I feel like basically it's the same.
It boils down to the fact that it's human principles, but my daughter's is awesome and amazing and quite opinionated.
That same strategy works wonders, saying "Here are four options. Pick one."
But it boils down to that there is just an epitome of the same humans that we are 40-50 years later.
We think we're a little bit different, and maybe we are a little bit more rational about it, but the same incentives are at the core.
Andy: The way that I think about it is-- And it's not a belittling, I think people often do this.
They say, "Look. I learned how to manage people from parenting, raising my children."
What people often hear is, "I'm treating my employees like children."
No, it's actually if you're a great parent you recognize that your job is to create amazing adults.
It's treating your children like children that causes problems, and it's treating your employees like children that causes problems.
If you say to your kids-- I get it all the time when my kids will be hungry, and I'll fall into the trap of saying, "Why don't you eat X?" "I don't want X."
"Why don't you eat Y?" "I don't like Y." Each thing, there's a reason they don't like it.
But if I say "You're hungry? You could have X, Y, Z or W or go pick something else, I don't care." All of the sudden now they're like, "Y sounds great. I'll just go do that."
That's the exact same philosophy. I actually treated the child like an adult.
I said, "You have to feed yourself. Here's your list of choices, make your decision."
Whereas only give them one at a time, treat them like a child if they reject it, why are we surprised that adults reject that if even a 13 year old is smart enough to reject it?
Guy: Indeed. This is an awesome conversation, I think we're going long, which is not bad or atypical.
Let me try to squeeze just a short answer on something that I think is a specific perspective on Akamai.
As I run this podcast in general I have conversations, a lot of good practice that emerges is this desire to turn good security into a business value proposition to promote this.
We've had in Slack for instance, they somewhat recently launched some good security features and they talked about how they made a lot of business noise around it, around it's commercial.
For Akamai, that's at its core. Being secure is very much a key selling feature at its roots. How well aligned do you feel?
Just in high level, the internal security, the things that you see and you want to prioritize from a risk reduction internally versus maybe the commercially friendly, promote the business value of being secure?
Andy: I think they are really well aligned, and I think I'll talk about the business value as two different things.
There is being secure, and there is selling security, and I think we often mix those up.
There are things that we do that the only reason that we do this is to protect the customer or to protect the customer's end user, or to protect the whole platform.
There is never really significant arguments about the importance of doing that work, like any important work you obviously always face tradeoffs.
I'm not saying we do it perfectly, but I very rarely see people say "That's not important."
In fact, it's really easy to say "Look. We have customers who care about PCI or SOC-2 or I could keep listing them. That's why we're even having this conversation."
It used to be that people would say, "Maybe I'll sell this to the customers who don't care about PCI."
That conversation used to happen, it very rarely happens anymore. Now we talk about when we're doing an acquisition, "How fast can we bring it into that secure envelope?"
I don't love using that "Compliance" is the conversation starter every time, but it's a great conversation starter and it works, it gets people in the door.
Now there's a pivot, which is "How do you sell security?" At the end of the day, the fact that we're PCI compliant makes certain sales processes easier, but it's not really bringing in new revenue. It might give us a small bump in the annual retail price per unit, or our average.
But then there's products that we sell that are about security and what's interesting to look at there is, if I look back at some of our most successful ones often are ones that we originally thought about to protect ourselves.
We said, "We have customers under DDos attack. How do we make sure we never go down from a DDos?"
At some point we looked and said, "We have a better DDos defense platform than anybody who's selling it commercially, why don't we just sell it commercially?"
That now lets you take that and sell a security product, or our VPN replacement that we built for ourselves.
We were breached in 2009, built a VPN replacement service over the next 10 years and cleaned up a lot of our authentication.
Now we're selling that on the market, this was a technology built on our platform because we had it available that when we first started building it and we had no intention of selling it.
It wasn't even a dream, but it was lined up to our own needs and it turns out many companies have the same problem.
Guy: I think that's great. It's like you know what's tried and true and it works.
Potentially, I don't know if it's how much of this is urban legend, but presumably AWS and maybe the inception of the cloud has come from that same premise.
Guy: Amazon using its own underlying systems, and just adjusting.
I'm sure there was some significant work involved in that, as there is in converting internal tools into commercial ones, but still you know the premise works.
You're not guessing.
Guy: Fascinating. I love this positive conversation starter perspective on compliance and regulations.
There's a lot of goodness, there's a lot of badness, and there's a lot of complexity over there.
But what you can be sure is that it starts conversations, especially GDPR recently.
Some people hate it and some people love it, but what you can't argue is that it rocked the boat.
It triggered a whole bunch of conversations that would not have happened otherwise and for that, I love it. I love the fact that it happened.
Andy: I think we're going to see the same thing with CCPA.
It's going to drive a lot of those conversations too.
Guy: Absolutely. So before I let you go, just one quick last question I like to ask every guest on the show.
If you had one bit of advice or one tip, it could just be a pet peeve that's currently annoying you that people do, to give a team that wants to level up their security.
What would that be?
Andy: I have one simple rule, which is nobody is the villain in their own story.
If you're having an encounter with a business partner, whether you're the development team or the security team and you're telling a story that says this person is bad, they have bad motivations, they're trying to hurt me, whatever it is.
Just stop. They have different motivations than you, they have a different model of the world, but odds are they have the same ultimate goal you have which is if you work in a for profit company or in a startup that would like to be for profit, it's to make money.
That's their goal. They might see different risks than you do, but you can't learn from somebody who's a villain.
If you tell the story that they're a villain, you have just prevented yourself from learning what matters to them, and once you start learning what matters to them you can channel them.
I was just talking to one of my staff who was in a meeting, and he said there was this really weird thing that the person on the business side was making these continuous jokes about the places we were going to disagree.
He would make a statement and then he'd say, "Here's where so-and-so is going to chime in and say 'This is unsafe.'"
I said, "This is perfect. It means they have a mental model of you that's accurate, that they actually saw the unsafe thing before you pointed it out.
That's great. It means they don't see you as a villain, they might see you as somebody they're struggling with but you're struggling together. You both care, they want to be part of it."
Don't tell the story in which other person's a villain, because it's too easy to do. I can take almost any interaction and tell a story about the other person being a villain. But as soon as I do, I lose the opportunity for my own improvement, because instead I can say, "How do I act like they do? How would other people see me in that same light? Maybe I can improve based on my own gut reaction that was negative."
Guy: Brilliant advice. Love both the catch phrase and the whole substance behind it and around it. I think it's useful probably well beyond security.
This has been spectacular, Andy. Thanks a lot for coming on the show.
Andy: No problem. Thanks for having me.
Guy: Thanks to everybody who tuned in, and I hope you join us for the next one.
Subscribe to Heavybit Updates
Subscribe for regular updates about our developer-first content and events, job openings, and advisory opportunities.
Content from the Library
Incident Response and DevOps in the Age of Generative AI
How Does Generative AI Work With Incident Response? Software continues to eat the world, as more dev teams depend on third-party...
Getting There Ep. #7, The March 2023 Datadog Outage with Laura de Vesine
In episode 7 of Getting There, Nora and Niall speak with Laura de Vesine of Datadog. Laura shares a unique perspective on the...
O11ycast Ep. #59, Learning From Incidents with Laura Maguire of Jeli
In episode 59 of o11ycast, Jess and Martin speak with Laura Maguire of Jeli and Nick Travaglini of Honeycomb. They unpack...