April 4, 2019
Ep. #27, Open Source Security with Jeff McAffer of Microsoft
In episode 27 of The Secure Developer, Guy is joined by Jeff McAffer, director of Microsoft’s Open Source Programs Office, who shares his ...
Thanks, everybody. It's a pleasure to be here. I titled my talk "Live to Fight Another Day," really because, in volatile times like this, it's really the primary focus of what we as startup founders and startup companies should be thinking about.
Another way to put it, as you probably all heard is, "It's better to be a cockroach than a unicorn" in this time because you really just want to survive. I heard that quote from Paul Buchheit. He's credited with another quote, and that is, "Advice is basically limited life experiences plus over-generalization."
I can come up here and talk about things and give you some of my limited life experiences, but it's really up to you to apply them to your own situations, because it's probably not applicable to your path forward and your situation.
I guess I'll start with one of those limited life experiences that I had, and that was in 2008. My career actually started in finance and ended in finance in 2008 when this happened. I'll share a couple stories of my career in finance so we can talk about maybe some of the similarities, some of the differences.
I worked in credit derivatives, structuring credit derivatives for about 10 years at a company formerly known as Bear Stearns, and I worked doing that in one of the biggest bull markets in credit ever. I'll share a couple stories to show you how crazy it got.
Has anybody seen "The Big Short"? Yeah, I was at that conference in Vegas, and it's about right. It was about that crazy. When I was there I was structuring a CDO squared, which is basically you take mortgages, you put them in a CDO, and you put those CDOs into another CDO. And it's like this mishmash where you create this leverage that effectively, eventually, blew up the world.
But at any rate, I was there trying to sell one of these deals, and in that situation there's a portfolio manager that's supposed to be presenting with you, and the portfolio manager is the guy that actually picks all of the bonds that go in the deal.
We're sitting, the night before, and I'm like, "Look, we've got a 9 o'clock meeting. I'll see you there at 8:45." I show up at 8:45, and the portfolio manager is nowhere to be found. I'm sitting there. The investors walk in, and the investors are usually a hedge fund or insurance company or a pension fund, and so they're wondering where this portfolio manager is.
I start to get nervous. I walk out to the casino floor, and there's the portfolio manager playing blackjack. I walk up to him, and I say, "Bob," and names have been changed to protect the guilty here, but I say, "Bob, we have this meeting." And it's clear that he'd been playing blackjack all night, was drunk and forgot about the meeting. That's when I started thinking about, maybe I'm not in the right industry, where this happens.
After that, we actually got that deal done, which is even more remarkable and kind of a sign of the times. But after that, I took a vacation, and I got back to the office, and the head of my department brought me in and said, "Hey Taylor, we'd like to talk to you about something."
I thought I was going to get fired because of this debacle, and instead he's like, "We want to promote you to managing director." Which, at that time, was a pretty senior title. And I don't say that to brag, but mainly because I felt fully unqualified to actually do that.
That was another flag in my head when I knew this was a huge bubble market and there were problems, and I think we can draw some similarities.
We were getting a lot of recruiting calls, which is why they eventually promoted me to managing director for no reason. It was a very frothy environment, and I think there's some similarities that we've seen in tech.
I'll say, with Bear Stearns, I think what happened was there was a lot of hubris and just kind of a sense of almost immortality that was built up through a very long bull market. I think it's important for us to keep a level head in these times, and go back and think about where we're at and how we can actually sustain ourselves when times change.
Let's talk a little bit about the current market. Here I have a chart, this is from Thomas Tunguz's blog, which is a great blog if you're not already reading it. But you'll see in Q4 2015 and Q1 2016, there's been kind of a reset in total venture money invested.
While that has been a little bit of a reset, if you look back to 2010 there, it's still above the average or the median of that time period. And so really the question is, is this going to keep going down, or is this just a blip in the market?
I really don't think we know yet. In my mind there hasn't been something super fundamental like what happened in 2008 to change the market, but you never know. Let's dig in a little bit here.
The seed market is down around 30%. This says Series A were flat, and the Series B was down 25%. These are percentages kind of off the high water mark. These are fairly big numbers. I'm surprised that this says that Series A is flat, to be honest.
This is number of deals done versus dollars. This is number of deals, and you'll see the seed market is down 50%. That's pretty shocking. That's kind of like black-swan-type numbers there, but again, you see that Series A is fairly flat.
Let's look at some of their data sources just to corroborate this. This is from Mattermark. They did a recent blog post which talked about, basically year over year, what has happened. And so you'll see from Q1 2015 to Q1 2016, there's been again a downward trend in Series C and Series B, and the Series A is actually up a little bit.
What's happening here? I think basically what's happening is the amount of money being invested is going down. What's going down even further is the number of deals, and so what that says to me is, all this money is getting concentrated into fewer deals, which probably means that there's more discipline in the market from an investing perspective.
That means that they're trying to pick winners and not spread out their money as much as they used to. In reality, this data is probably not 100% great data, because I get it for free. But there are some signs we can get out of there.
Then you go and talk to people in the street, or you look at tweets, and you have a completely different perspective. Here Sam Altman is talking about the recent YC batch, and he said he's never seen higher interest in any batch. To me, that's very confusing when we look at the data of these tweets, and then if you talk to people that are trying to do a Series A, they say it's really hard to actually get done.
What does this mean? I think what it means is that we're in a really volatile time. There's a lot of uncertainty, and so we really need to protect ourselves. The one thing that I did take away from Bear Stearns is that in times of volatility and uncertainty, option value goes up.
Really what we want to do is have options, and optionality is king in these times.
Now, in the financial markets you can very quickly buy options or hedges on where you're at, and startups is much tougher. You're in a very illiquid position. You're obviously very committed to your idea and what you're doing. So how do we get optionality in the startup market?
Really, I think it's all about controlling your own destiny, and what does that mean? That really means not relying on investors or really any third parties to sustain your future. This quote I put up here is from Dan Siroker who's CEO of Optimizely, and this was from a blog post he wrote where he laid off 10% of his workforce.
What he said is, "What we're really trying to do is control the path we are on as a company without having to depend on anyone but ourselves." That's true optionality, when you don't have to rely on other people.
What's very interesting about this quote is he said it right after they raised $58 million, I want to say, and so right after they raised $58 million, they laid off 10% of their workforce. That takes a lot of discipline.
I'm sure Dan doesn't like laying off people, and usually at a time where you raise a bunch of money, you go hire more people. I think that says something about Optimizely's discipline, and it also says that there could be something scary ahead of us.
How do you get optionality? And how do you decide whether you're going to be sustainable or not? Has anybody seen this calculator before? It was created by Trevor Blackwell, who's a YC partner, and basically what you do is you can put in your expenses, your revenue and your growth rate, and it will tell you when you are going to run out of money and how much money you need to get to that point.
It could be that you are three years out or five years out, or it could be that you're six months out on how much money you need. This seems very intuitive to me, but I'm shocked how many people actually don't know this.
I've heard a bunch of stories of founders basically waking up and in two weeks, they don't have any money. They have two weeks left, and they didn't realize it, which is kind of astounding.
If there's one thing I could recommend you do is figure out what your runway is and get a plan in place if it's less than a year.
If it's less than six months, then you're in real trouble, and you're going to probably have to make some tough decisions. Really, you should know this at all times. How do you get to being default alive? It's pretty obvious. You either have to increase your revenue, your growth rate, or reduce your expenses.
We'll start with expenses or costs. When we were at Mailgun, we raised $1.2 million in 2011. Sixteen months later, we were acquired by Rackspace, and we still had over $800,000 in the bank. That's a $25,000-per-month burn rate with six employees, which is really low.
I think, looking back, we were a little too conservative, we were petrified of running out of money all the time, and I think we probably could have been a little more aggressive. But what I will say is it gave us a lot of optionality during our acquisition talks, because any time you're in a negotiation of acquisition or fundraising, you really have to be willing to walk away from the counterparty. Otherwise, they're just going to sense that, and create a bad deal for you.
It gave us a lot of optionality during that time to say, "Look, we don't need you. We can actually operate independently, indefinitely, and be okay without the acquisition."
Why do I have a coffee grinder up here? One of the things that we do generally is create a lean culture in our company. I started a new company called Gravitational about a year ago, and when we were starting we needed something to grind up the coffee because we drink a lot of coffee, and I don't know if you know this, but automatic burr grinders are super expensive.
We decided we were going to buy one of these hand grinders, and these things kind of suck. You basically sit there and grind coffee for what seems like forever, but we made the decision that we're going to do this until we hit a certain revenue target, and then we can get the fancy automatic grinder.
Basically what people have to do is they have to sit in the kitchen and just grind coffee and think about how we are going to hit those revenue targets.
This really sucks. How are we going to get better? I think these little things, obviously a coffee grinder is not going to make or break your company, but these little things that you can do to create this culture. I guess grinding is a good example, it really helps.
This is one thing you can do, is just basically don't spend a lot of money. I don't think if you're starting your own office you need fancy meals or ping pong tables or anything like that. It's tough to compete without those things, but I'm not sure that it really is.
There was a guy, Bill Campbell, I don't know if you know him. He is called The Coach, and he had a great quote. He basically said culture is not about ping pong tables or fancy food. It's really about how you treat your employees and how they treat each other.
I think it's more important that the people that work for you or that work with you are working on interesting stuff, and you can get away without a lot of the fancy stuff in the beginning. This is a big thing. If you want to reduce costs, first of all, you probably shouldn't be in the most expensive city in the world, but we're all here.
What can we do now? I think you should really consider outsourcing. One of the reasons we were so frugal at Mailgun was almost all of our R&D was based out of Russia. It's hard to do that, and
you may not get the development velocity that you would get if everyone was in the same room, but it's a hell of a lot cheaper, and it makes you have a lot more runway.
There are ways to do this properly, and there are ways not to do it. You want to have someone in the remote location that you trust that hopefully is a full-time employee that knows the market. You also want to be able to go to a place where you basically want to optimize your workforce to go where you have those relationships.
The other thing you can do here is with sales, and I don't know how many of you have a sales force yet, but if you don't, there actually is a lot of flexibility with sales.
There's a guy I know who's in a similar spot as us. He basically said he's hiring sales that are 100% commission-based, and they get paid a hefty commission when they do sell a deal, but it's a variable cost. They don't have these big, fixed costs, which is really important.
One more thing I'll say about the remote work is if you start earlier, it becomes a lot easier later, versus if you have everybody in a room and all of a sudden you just decide to hire remote workers, you don't really have the discipline or the communication patterns to make those remote people successful. I recommend doing this early rather than later.
Still more, a radical idea is move everybody out of San Francisco. And I'm pretty conflicted on this. I think San Francisco is the best place to start a company, or the Bay Area, at least. I think when we started Mailgun, it was very hard to raise money, even in New York, and then we moved to San Francisco, and it got to be a lot easier.
But this is some interesting data that again was on Mattermark. It shows that this is again Q1 2015 divided by, or over, Q1 2016, and you can see that actually some of the smaller cities, the funding has increased, and the Bay Area and New York has all gone down a lot.
It's interesting if you actually pull Bay Area out of these numbers; the funding market has actually gone up year over year. It's something to consider. I'm actually pretty happy that there's more options for all of us to move to other cities.
What are some of the other big fixed costs that you have? Obviously office space, and I guess I'm kind of preaching to the choir here. But you know, when you look at starting an office independently, you look at the square footage, and you say, "Wow, $55 or $60 a square foot is a lot, but there's all these other costs that you don't think about immediately.
You have to get food. You have to get cleaners. You have to get utilities, and it all adds up, so you're to the point where you're at $2,000 per employee, which is a lot of money.
The other hidden cost that you don't think about is your time, and so you have to actually spend a lot of time getting your environment to a place where people actually want to work. You have to make sure there's enough pretzels or whatever, and it's time that's probably better spent doing what you're good at, which is probably product or development or sales, or whatever it is. Something with more value added than doing this stuff. That's kind of some of the quick easy costs to reduce, I would say.
Let's talk a little bit about revenue growth. There are really two ways to grow revenue. There's basically: grow revenue at all costs, which I think has been something that people have been focused on recently, and so this is basically let's just grow revenue. Let's get users. Let's do growth hacking, or whatever the common term is today, and not worry about how we're doing that.
I'm sure a lot of you have already seen some of these metrics. People have probably stood up here and talked about them, but really it's focused on communal costs. How are you growing that revenue? Is the LTV of the customer you just acquired more than the cost to acquire the customer? What's the payback period? How long until you get your acquisition cost back? Basically, what is your sales efficiency?
There's another metric here that's relatively new, which probably needs a new name, but it's CRGPD, which I'll talk about in a little bit, which I really like. One thing I will say is you can have these metrics, but you really need to celebrate them with your office or your employees.
I've been in offices, and you go up and ring a bell when you've made a million-dollar sale, and everybody claps and all this stuff, and everybody cheers, and then you go talk to finance guys, and, well, this deal just lost us money. Why is this happening? Why are you celebrating this silly stuff when it's actually losing the company money?
I would recommend you pick the metrics you want, make them for responsible growth, and then celebrate them in your office.
Have them on a wall, base performance on them. Do whatever it takes to get everybody behind you when you pick these metrics.
The one thing for responsible and efficient revenue growth is to focus on customer success.
First of all, it's really a lot cheaper to retain a customer than it is to acquire a customer. And second, those customers, if they are successful, will actually be your evangelists. Here's a little example that shows if your customers become evangelists, and they define evangelists as recommending five other customers.
If you grow that 30%, you'll effectively more than double your sales efficiency. There's really no better way to grow revenue efficiently than have successful customers. You really need to be focused on that.
There's a couple other stats I put up here just to kind of reiterate this. A 5% increase in customer retention will grow your profitability 75%, and attracting new customers will cost your company 4 to 10 times more than an existing customer, than keeping an existing customer.
These are big numbers, and by the way, I just googled these. I have no idea if they're right or wrong, but they kind of reiterate my point. I think we kind of get the general idea here, you really want to focus on retention and success.
If you're going to focus on success, you have to put that into your metrics as a cost. There's a lot of the metrics I went over before that don't actually do that. This is a relatively, I haven't heard about this metric until recently, but it's called "cost per recurring gross profit dollar," and what it does is it helps you figure out, how much does it cost to get one dollar of recurring gross profit?
It factors in your support costs, your gross margin, your CLTV, and the example I put up here shows you the power of reducing churn through success, investing more in success.
Even though you're spending more in support costs, the monthly churn has gone down, and your CRGPD has gone below one. You generally want to keep this below one, but really you want to go as low as you possibly can.
Just some more things I found recently as I was thinking about this talk. Does everybody know Patrick McKenzie, @patio11? He's kind of a guru with this stuff. He basically created a company based on a bingo-card creator, which that's the true definition of grinding it out, is you build a business on basically a random-number generator.
He went to MicroConf, which is a great source of information if you want to get lean, because it's all really boot strappers, and so he tweeted after the conference, "Here are the two highest impact fixes that I heard about. One is improving onboarding, making sure your customers can get onboarded really quickly and easily with minimal friction, and then make sure that you follow up with them on a continual basis to make sure they're successful with using your product."
And then two, reduce churn. Your acquisition costs, if you have what they call a leaky bucket, where you have churn, will never make up for that. If you don't focus on churn first, you're effectively just lighting your acquisition costs on fire and wasting money.
I guess I'll kind of wrap up with some easy wins, as far as revenue, that I've seen, and hopefully most of you are already doing this. This is a website from Aircall. I was looking for a phone system for our support, and I notice that they actually make the annual prepay the default option, which is a little bit of a dark pattern.
It's a little sketchy, but I think the point is that if you can actually get more annual prepay, your runway will last much longer. It's something we should all be doing, and I also think you should probably reevaluate your pricing.
Generally what I've seen is that most startups don't charge enough for their products, and there's a lot of resources available.
Patrick McKenzie's blog is a good one. There's priceintelligently.com. There's SaaStr. It's kind of an easy win to just change some numbers in your website and see your revenue go up, instead of building features and having a six-month pipeline and those features may not even be wanted or desired.
Another thing, I haven't really talked about acquisition that much because it's not really my forte. I'm sure there's a lot of people that have stood up here and talked about acquisition that know it better than me, but one thing that I will share that's been super successful for us, both at Mailgun and our new company, is content marketing and open sourcing.
Especially if you're selling to developers, I think open sourcing comes across as a very authentic way to market to them. You're providing something of real value and contributing back to the community.
It's not free. You have to be thoughtful about what you're open sourcing. You don't want to open source your core IP if that's what you're building your business on, but we open sourced something recently that was fairly core to what we're doing, and for about a week we were up there with Facebook and Google, and no one really knows who we are otherwise.
This is pretty powerful stuff. It's also great for recruiting, and you also get a lot of feedback from the community. It's a good way to test what you're doing in the market to see if there's any interest for it.
This is a bit of a tangent, but one thing that really concerns me about the market we're in is when you look at that reset period, we all start out selling to each other. I don't think that's a bad way to start, but it does create this issue which there could be this knock-on effect where if a company goes out of business, and they're your customer, that hurts your business, so on and so forth. It kind of creates this startup house of cards, I called it, and I think we need to be worried about diversifying our customer base away from just selling to startups.
By the way, it's really hard to find these images, because it's just like Kevin Spacey and Robin Wright. If you need a "House Of Cards" image, I can share it with you.
I think diversifying your customer base away from startups is important, especially when there's signs of volatility in the market.
And so what do you do? You can kind of branch out to different geographic markets that may not be as affected, so international. Another way is you could sell to enterprise, and I think enterprise sales is a little scary to all of us. It's much easier to put up a pricing page and have people sign up, but you'll be surprised at the reception you'll get.
One, enterprise software, the software that they use generally sucks, and so your products are probably much better already. Two, they're actually looking for new things to use, and if you've already built a SaaS product that people are using, you've proven that you're not vapor ware, which is probably the biggest thing they're worried about.
You can kind of negotiate around the other objections, which is, what if you go out of business or what if you get acquired? You can always negotiate around those with a proper contract. This is what we did at Mailgun. We built a fairly successful pure SaaS business where we literally never talked to anybody before they used our product, and we did that for maybe two years, two and a half years, and then we decided to put this link up here that said managed email service.
It was literally just a link, and it led to a landing page, which, I don't know, it took maybe a week to design, but there were no additional features, no development time. We did put an SLA around it so there was better service, priority service, but we were offering priority service to everybody. There's really no difference. All we were doing was kind of capturing the marginal benefit that we were offering these enterprise companies, and we grew revenue I think 25% within a few months by doing no feature development, anything like that.
The other thing that, and this will lead to my one self promotion slide, but the one thing that we kept getting requests for is, "I want to use Mailgun, but I want to use it on my own infrastructure. I want to self-host it". It was something that we always shied away from for obvious reasons.
It will slow down your development pipeline, perhaps. It's hard to do this, but it led us to this idea of, what if we could make this easier for companies to actually either ship their SaaS to multiple sites or go on premise, and how would we do that? That's why we started our new company, which is Gravitational.
Effectively what we do is we help customers sell to enterprise by giving them an on-premise version, and I think with the advent of containers, container orchestration, this is becoming a little easier. I'll say it's a good way to sell into very large, high-value accounts. They'll pay you 10X, 100X. They don't really care what they pay you. They just want a solution, and so it's definitely something to consider.
I think with that I'll just wrap up with the key takeaways here. First of all, figure out your runway. If you haven't done this already, please do it tonight or tomorrow.
Control your own destiny by becoming profitable as soon as you can, and so you do that by reducing revenue, by increasing revenue or reducing your costs, and build a culture that celebrates profit. I think in a San Francisco environment, most recently we've really built a culture that celebrates growth & revenue at all costs, and I think that's going to change gradually.
So, you want to be ahead of the game when you're starting your company. And then finally, diversify your customer base. Whether that's getting outside of San Francisco, going to China, or going somewhere that's not as tied to the local ecosystem, or moving up market, going to enterprise. That's really it. Thanks everybody.
I'm a big fan of the founding team basically doing everything in the beginning. That's obviously product development, product management, but also sales, support. And so you get to a point where you're so busy that you just can't do those things anymore and offer a good product, and so that's when you hire.
I think what happens when people raise money is they immediately hire a bunch of people, but they're not even sure what those people are going to do. And so I think it's okay, especially in this interest-rate market, for your investors to wait a little bit and say, "Look, we're going to be responsible. I think investors actually value that."
They value today that responsibility, and yeah, basically do all the work yourself until you can't sleep anymore. Yeah, till you don't get any sleep, then hire someone to help you out. If you do everything yourself, it'll help you pass on that knowledge, at least initially, to someone who's hopefully better at whatever you're hiring them for.
As far as selling to enterprise, it's more of a value proposition, at least that's what I have heard. I think you have to kind of back off the infrastructure costs as well, but there's also remote support costs that are built in.
You probably wrap it with a better SLA. It's really, "Is this thing going to go down or not?"And if you can provide that assurance, then it's really about the value rather than the cost of the infrastructure.
What's that? The point of not being on the cloud. It's not really so much in my mind, it's not about location. Yes, you can be in Amazon or wherever. It's more about having a private installation.
It's really you're selling into a situation where they literally cannot be on a multi-tenant environment. And so that could do some leverage to say, "Well, yes it could be cheaper to be on the cloud, but you can't be on the cloud, or you can't be in a multi-tenant environment."
You're in a position of a better negotiating position, I think, then. I think you're right. You have to back out, or at least have an answer about the infrastructure. What we do sometimes, we actually manage the infrastructure for them. We'll say, "All right, we're going to give you your own private cloud." Or, "You give us your keys to your private cloud account. We'll actually manage everything for you."
And so that basically reduces your operational burden. Part of our thesis is that, operationally, it costs a lot more to keep software running than the actual development over time. If we can take some of that cost away from you, it's helpful.
Yeah, so we look at the obvious stuff like GitHub stars, commits to the repo, issues created. We also will generally build a separate page from our blog where we will launch open source.
Just like a landing page, that is on our own domain. So that we can see what the flow is from that page to our website and into a purchasing decision or at least a sign-up.
I think it's easier to just put up a GitHub page or a GitHub repo, but you won't be able to measure the results as effectively as if you put something on your own domain. You'll obviously have a GitHub repo as well, but you really want to capture the flow to the initial point to measure the effectiveness.
You obviously want to structure it so that you have a very quick payment period. So like a 30-day net or something smaller. That's hard to do sometimes. I think when you're going into the sales process, you really need to make sure you have a champion that will help you internally navigate that system, and you need to evaluate how legitimate is this actual company in paying their vendors.
What we do is we will generally sell ahead of product. So we tell them that this is what we do when everything is completed. There will be some integration that we need to do, and you're going to pay us during that period so that we actually know they're going to pay us down the road.
I worked at Rackspace and saw it from the other side, it's not that a lot of these companies are purposely not trying to pay you. They're just organizationally screwed up. They literally can't get a piece of paper through 10 people fast enough, and so you kind of need that internal champion to get that through.
But the good thing is, once you're in and you're kind of an approved vendor, you can then upsell tremendously, because you kind of pass through this big gate that you got through.
I don't know if that helps. I think having someone on the inside that you can rely on to help you is important. Just show up and start knocking on their door or something, I don't know. Yeah, it's something definitely to consider.
That's kind of the upfront idea, is you basically reduce, usually it's like 10% where you reduce your yearly cost for an upfront payment. I think you need to be flexible in any sales negotiation with enterprise. We will basically say. "Look, if you pay us X amount up front, we'll have a free back end," or something like that.
You really have to get the customers' view of what's important before you start offering those things. I think it's highly specific. All right, thanks everybody. Appreciate it.