January 27, 2017
Ep. #29, What’s The Future of Continuous Delivery?
In this episode Edith and Paul are joined by Marianna Tessel, VP of Engineering at Docker to discuss the evolution of software delivery from...
Interested in learning more about building developer communities that can power billion dollar businesses? You can now rewatch the rest of the DevGuild: Software-Defined Movements talks here.
There’s no doubt that developer love can translate into long-term profits and market share. But the path between developer adoption and meaningful revenue is tough and it’s ever-evolving. In this session from DevGuild: Software-Defined Movements, our panelists discuss how the best bottom-up companies have reconciled and navigated developer community-building and exceedingly larger enterprise deals.
Ashley: I’m very excited to bring up to stage, Luke Tucker, who is the VP of Community at HackerOne, Marc Holmes, who is the CMO at HashiCorp and Margaret Francis who’s COO at Armory. We are so lucky to have all of these people on stage today. They have been at companies like Heroku, Chef, Docker, Visual Studio, Hortonworks, and we’re going to be having a very, very good talk about connecting developer community to commercial success. Everyone welcome. So good to have you here.
Ashley: It’s not just one department that connects all the dots from developer community to commercial success. There’s a huge effort across all teams. I’d love to hear from each of you, talk to me about your backgrounds, talk about what part of the organization and community side that you’ve worked on. And just give us an overview of how you think about pricing at your current role and then just tell us about you. Margaret, can we start with you?
Margaret: Sure. I’m currently the COO at Armory, and I run prod and engineering and the go-to-market side as well. Before that, I was at Heroku, where I had a very similar function, ran all the different functions in the organization. I think that one of the things that is really a compare and a contrast between the two different roles is that Heroku, I was there for I don’t even know, 6-8 years, and it was a very grassroots product that grew up into the enterprise.
It followed its users, it added enterprise and security features. Sometimes we were even struggling to get corporate and enterprise fast enough to meet our user needs. At Armory, it’s very much, I would say a team and enterprise focused product today, and the exercise is more, how do you package it down so that enterprise developers who’ve loved using the Armory distribution of Spinnaker, can use it in a smaller team setting and still get the benefit? It’s magical to be able to compare and contrast those two experiences.
Ashley: Awesome. Thank you for that context. Luke, your monetization strategy, I didn’t talk about [in my presentation] because it’s so different than any other ones that I’ve ever worked on. I was going to save that one for you to give us an overview of HackerOne. It’s so interesting how you’re monetized, but it is not something that everyone can do.
Luke: It’s actually interesting the way HackerOne started. A lot of today’s listeners are earlier on in their journey. At HackerOne the first several years, it was all about the marketplace model. We were approaching it and similar to eBay as a platform, you have buyers and sellers, we bring together companies and organizations, and we have the reputational graph and index of all these security researchers or what we call hackers on the other side. We connect them and it’s an incentive-driven crowdsourced environment.
If they find a security issue in the attack surface of a company, they get paid and the company’s happy because they’re more secure and HackerOne’s in the middle and we would take a 20% cut. As we’ve evolved the business over time, we figured out that actually, there’s more services and going up market in the enterprise as well. There’s more we can do and our customers are asking for more.
They love HackerOne when you provide the creative intellectual humans on the other end of the screen digitally, and you bring them to define these amazing issues. Let’s have more services, let’s help have you be triaged. Now we can use the platform data. As we’ve become more of a platform, I think Bill Gates had a great quote, it was like, “When the value from everybody else on the platform is greater than the money you get, now you’re going to be a true platform.”
That’s HackerOne’s journey, is initially marketplace. And we grew into a SaaS hybrid. We still have consumption pricing within our SaaS product but it is a fee plus bounties and we prioritize the rewards to the community, so we’re very proud to have paid over $150 million to hackers over the last seven years or so.
Margaret: Luke, I just have to say, is not every day that I get to be on one of these panels where I’m an active customer of the panelist, whether it’s Marc’s products or whether it’s HackerOne, or whether it’s GitHub. Even a lot of your ex products and corporations, so I just have incredible admiration for a lot of the products that have been built by people on this call.
Ashley: Same. I got so excited to talk to everybody, because I’m excited about all of these products and I watch people build some of those incredible developer communities and a lot of you are sitting right here. This has been a very, very good day for me. Marc, talk to us about HashiCorp, but also other monetization strategies you’ve done at all of your previous roles.
Marc: Sure. I was fascinated by Luke’s business model as well. I think it’s super interesting. I think my background actually, I was originally an engineer, and so my background, I cut my teeth as a marketer at Microsoft. The areas that I spent my time have been monetization at that scale. And then I spent a lot of time in smaller startups, but generally speaking, sitting around just post-initial monetization, that’s where I specialized.
If you think about HashiCorp’s position in the market right now, and it’s taken a bunch of years. As a company it’s been around for eight plus years. Monetization came a lot later. Although I think most people became aware of HashiCorp probably in the last couple of years in terms of mass market, it’s been around for a long time. The open source products, whether that’s Packer or Vagrant in the early days, and through Terraform and whatnot, it takes a long time to get those to monetization.
I think that the main lesson in trying to think about how you commercialize is don’t do it too early. Do it very deliberately. And repeating that pattern over a number of years has really been where I would identify most of the success. I would also say generally speaking, as an engineer, or ex-engineer, I try to do everything very thoughtfully. I think the right thing for anything is to have a framework for any decision you’re making in terms of community and commercial activity.
Ashley: Absolutely. I agree, don’t monetize so early. Really focus on building up the community and focus on making sure that you have people that want to use your product and love your product and you take that feedback. Anytime someone’s coming to me and they’re like, “We’re monetizing it right now.” I’m like, “No, no, no, build your community, take a break, think about growth from a community standpoint and then think about monetization.”
That brings me to my next question. Obviously, when you’re going into a board meeting or you’re going into end of year budgeting, you’re thinking about doubling or tripling revenue every year at a certain point. But that ties back to community metrics in a pretty interesting way. How does everyone on this call think about… beyond revenue, how are you thinking about tying that back to community metrics?
Marc: Well, I wake up every morning and I think about two things. I think about MAU [Monthly Active Users] on the one hand and I think about ARR [Annual Recurring Revenue] on the other. So how do we accrue to those two metrics? Not everything is directly connected. But I like to quantify as much as you can. I read a book a few years back called How to Measure Anything. I think the basic advice in the book is, just start.
Once you get that mentality into the team, you can really think about, what does it mean to do softer work such as community work, but how can that really be quantified? Sometimes it can’t be directly connected to, is it precisely going to drive the end metric? but you can get a sense of the quality that you’re trying to deliver through that process. As for instance, even amongst our developer relations team, we would look at how many hours on a monthly basis that they are spending in front of the community.
And there’s different ways to think about that because there’s some very non-scalable work, there’s a lot of writing to be done but there also scalable work which might get a hit on YouTube.
But in aggregate, you get this sense of, is that mix correct? Are we spending enough time? Are we getting the right outputs, anything stalling, are we seeing the right feedback from community?
Just a lot of those measurements of internal processes on the community side.
On the ARR side, we do the same, it’s much more typical marketing metrics. You can imagine leads and conversion ratios and so on. How do we tie these things together as we correlate between what we see in community, what we see in terms of that monthly active user, whatever proxies we’re using for those things, what we expect to see as commercial business?
On the other side, we don’t tie them together truly. We’re not trying to pin down a member of the community, and say, let’s sell you some software. We’re trying to get the general sense as we see growth in certain areas, and so now’s the time to put the pedal down on commercial activities around those things.
Ashley: Got it. Margaret, we’d love to hear from you on this one.
Margaret: I smiled, Marc, when you said MAU and ARR, because it’s MAU and ARR. Now, you can characterize MAU in any number of different ways. It could be paid or unpaid U. It could be applications deployed, deployments that have happened in… I think about those metrics in slightly different ways, in slightly different situations, but it basically boils down to some variety of MAU and ARR.
I think that there’s a little bit of an assumption in this conversation that community means free, and I actually don’t necessarily think about it that way. Neither at Heroku, nor at Armory has community been as clean as is, like there’s free and there’s paid. There’s corporate developers and the free spirits of the community. I think they’re all individuals and they’re moving into different contexts.
Very often you’ll see a champion of your free or entry level product move to a different place, or take a new gig and they want to use the same great tools that they’ve used in their private hobbyist life, in their work setting, and they become great advocates for you inside the company. People move around. They take different jobs and the community ends up being something that’s very fluid that way. Sometimes paid, sometimes unpaid.
You just want people to love your product and appreciate the value that it gives them to accomplish certain kinds of tasks is how I think about it.
Ashley: I totally agree. The community not being only a free user, I agree with that too. For me, I’ve always worked in models where we’re trying to get the free product to be the community edition and then it shifts back a few times. I think it’s been back and forth for me. At GitHub, at GitLab, I’ve seen it move so many times but I agree with you there.
From a land and expand perspective, when you’re thinking about revenue and community, Luke, I’d love to get your feel on when you think about land and expanding, how does that work at HackerOne and how does that tie directly to revenue or to community growth?
Luke: Definitely, a lot of these questions will be unique for the HackerOne scenario, but I think there’s actually things that can be portable. I wanted to tap on, you had a slide that talked about your Indie Hackers and your enterprise developers, and this is one community and that builds off of, Margaret, your point too.
I see this really anecdotally as well as very specific examples in our community, where many of our registered users are moonlighting as bug bounty hunters on our platform and their day job is usually an AppSec engineer or security team lead at companies that we would want to sell into or that want to use our products, and maybe they already do.
That relationship of, if a program manager or a security team lead is communicating with a hacker on our platform, and they don’t have a good experience, then that’s a platform challenge. That’s an opportunity we have. They still have to communicate and connect. The more they have synergy around identification, around hackers, and how they think and feel and what they want to be able to receive in that experience, the better.
When we think of the actual community, it’s all of the above, and I have a joke with our CEO, everybody in our company, every employee is on the community team.
Because if sales sells a deal that maybe is under budgeted or it doesn’t have the right opportunities, then hackers aren’t going to have a good experience. If the triage team that’s reviewing the support doesn’t handle it great, hackers have a bad experience. And it goes down the line. I can name every department in our company.
When I think of all those elements, it’s just this virtual loop and that word of mouth initially in our network effect model, was super valuable. We had air cover to say, “let’s just start and go focus on getting the top hackers on a platform, and they’ll come when we get the best programs.” We got some high paying programs and then we created this wonderful network effect.
That’s been HackerOne’s journey and how I look at it today when I talk to other leaders in the company about how they can take care of our community. It’s the customers as well.
Ashley: Awesome. Margaret, do you have anything to add there?
Margaret: I was just going to add, I think that once you’ve experienced the benefit of a product or a community in one place, you are absolutely going to spread that knowledge to your communities and you’re going to take it wherever you go. I imagine in that HackerOne context, once you’ve actually used that program and you’ve succeeded with it, you’re never going to go someplace else and not have it. Why would you skip that?
You’re like, “This is just a really sensible way and a really sensible set of products to use to do this thing, and so I’m going to continue to disseminate that knowledge.” And every function of the company really does build that community, to your point, Luke, it’s not just DevRel’s job. It’s sales’ job, it’s marketing’s job, it’s engineering’s job to maintain respect for everyone’s time and participation.
If you don’t care about your users, you don’t care about your community and it comes back to bite you.
Marc: Just to build on that, I think that’s totally true. I get a lot of phone calls which is, “Hey, can you come talk to us about how we’d set up a community with business x, y, z?” It’s like if you’re asking that question, you’re already haven’t won here. If you genuinely understand why a community would be useful, you’ve got a leg up, you’ve already got an advantage because you can’t just graft on the idea of community to some inauthentic position.
The same is true of marketing generally. There’s always a conversation, developers don’t like marketing. That’s not true. They don’t like BS, and so if you don’t feed them BS, it’s fine. They like to be told what things do and why we think that matters to you, and they’ll make up their own minds.
That’s true throughout all kinds of developer marketing, whether it’s community activity or commercial activity. Tell them what things you do. Be rational about that. They will make up their own minds and that gives you a lot more strength than trying to sell to them or try to blind them with BS for sure.
Ashley: I totally agree. I’ve always said truth in marketing when you’re working with a developer community. You can’t lie and your product’s messaging has to be true. Whatever is on the webpage, if that’s not exactly what your product does, it is not going to go well.
Marc: It makes life a whole lot easier. It’s because actually at this point, you’re talking about the trends that businesses are actually seeing. They might not like the way you said it, but you’re not trying to hide behind the idea of, “I think I might’ve just made this up.” There’s real issues.
Ashley: This is a very different style of marketing. We’ve talked a lot about monetization and community, and I tried to talk a little bit about getting from free to pay. But I’d love to understand from your perspective and your experience, when you were thinking about the free product and you were thinking about, “okay, what are the paid features that we are going to ship, or what are we going to make paid features,” what did that look like? What were the things that made it into the paid side of the product?
Luke, I think you probably have an interesting take on this. Again. The HackerOne model is so interesting to me.
Luke: Like I mentioned at the outset, we can’t speak as much to the free to paid, although it did start where it’s like, customers only pay us if they get results, so it was a very easy conversation of “Hey, we’ll estimate your bounty budget, you put the money in our HackerOne bank, and we’re going to put the scope out and here’s what you’re expecting.”
Respect is a key element, I think when we’re talking about marketing, to communicate to your audience on all aspects. If you alienate one side who are producing the value, or at least that’s their perception as well and you don’t listen to them and you don’t receive them, that’s not going to be a good experience for anybody and that will bite you down the line.
When we’re looking at free to paid, we do have free versions. Initially when we started HackerOne, like a lot of early-stage startups, because we didn’t have much of a team, it was like, “Let’s just let people manage the programs themselves.” Well, we ran into some issues and some challenges, and so we actually added an apparatus where we’re now the program team in the middle to now charge you a services fee to provide that. That’s some of the evolution.
We have a belief and our co-founder Alex Rice who is Head of Product Security at Facebook is very much into, “Everybody should have a way to freely provide a security vulnerability to any company,” so that’s part of the core mission, is what we call vulnerability disclosure policy. 15 years ago, if a hacker tried to submit a security issue, you could have the FBI knocking on their door. Today, you have platforms like HackerOne, and we want the opportunity for anybody to submit a security vulnerability, so we try and provide that free service.
Thinking about your business beyond, ” let’s just not charge everybody for all these things, let’s get all these areas, let’s do this, make it as easy as possible,” the value of your company has to come to bear in your decision framework. We have a value called default to disclosure. If we don’t default to disclosure and help provide a way for customers to be more secure and live by our mission of making the internet safer together, well then we’re going to make different business decisions, so we always have to have a free component.
Ashley: Marc, HashiCorp is doing pretty well with this right now. Tell us how to monetize, please.
Marc: Well again, I’ll preface this thoughtfully. I guess there’s a couple different dimensions to it. It’s not like it’s always right or anything. But the first is that we build a lot of different products. We have the luxury of having a set of progressions to those products, whether those are just incubating products, emerging products. We have newer things like Waypoint and Boundary, which we’re hoping to gather a minimum viable audience and hopefully add some product market fit and we’ll think about commercialization of those things.
There are others that become more mature and quite clearly, there’s some compelling reasons to want to buy those. We have that feeling of a conveyor belt of those things maturing over time. That’s great for us. We’re a multi-product company. It’s causes a lot of tension because you can imagine things get complicated. But it’s also great because it means we can wait out for a little bit of time to see whether things are going to work and whether they’re not going to work.
On the actual monetization of these things, one thing is important to preface as well, and I think Armory’s the same, Margaret, is, it’s not a bottoms up selling model completely, so we’re not expecting to do all of our deals by working with the community hoping they then buy into it. We see some of that happening, and those people become champions on larger enterprise deals, but we have a dedicated enterprise sales team who are also working down into accounts and it’s worth mentioning, because as you think about monetizing, what you do not particularly want to do generically is penalize the community for their general use of the product.
You try not to limit on scope, you limit on the scale, so no one minds paying for things like team collaboration features, SSO, all of the enterprise capabilities to get you over things like compliance and governance hurdles, but also allow enterprises to actually adopt those things meaningful in a very large-scale way.
But they don’t like it when you say, “well, you can have this much capability but you’re going to have to pay us for a little bit more.” There are some spaces where you can do that, for sure, but generally speaking, I think if you stick to the idea of, scope is not commercial, scale is commercial. There are exceptions to the rules and all that, but that’d be my general framework.
Margaret: I couldn’t agree more about the scale being the thing that you really want to commercialize on. Can your people use the product, can they get the benefit out of it and does the tax get paid when it is used in a commercial setting where scale is important, is probably the best single metric there. I think scale can take a lot of different forms depending on the product, so is it users, is it the size of the database? Is it the security and compliances? Is it the SLA? I’m not going to give away five nines to the community.
Because it’s extraordinarily expensive to deliver in any context. But I can give away a thing that’s like two or three nines and has really good transparency around when things are up or down and that’s fine. Or I can give you something that meets some minimum bar on security, but does not have everything the feds need, because the feds are expensive. They’re expensive for us to service as a business. So I think that paying for scale and trying to figure out how to get it right is a lot of that journey from free to paid that all of us are on all the time.
Marc: And also, you’re looking for those folks to be champions, and so if you’ve limited their ability to use the product, how much can they champion that inside their enterprise? It just doesn’t logically make sense to get the benefits of that from the commercial side as well.
Ashley: It’s a super hard balance. You’re trying to give everyone access to every feature so they can love it and take it to work with them, but then you also need to charge them at work. It’s a very hard thing to get right. This takes me to my next question. This is a hot topic one in my mind because I’ve worked in the open source communities so much and it’s hard to get right.
All of your companies have an aspect of a free community version, open source version. And the community does play a role in product discovery and value creation for your larger customers and the people that you’re charging. How have y’all thought about that at your companies and are you actively using a community and the open source community to help influence what you’re building for your larger corporations or larger companies who are using your product? Margaret, I think this is a good one for you.
Margaret: It’s a great one. The role of the community in driving product development is huge at every place that I have ever been. Either people are talking with you about your product, they’re engaging with you and they’re telling you what the needs are or they’re not. You take something like Spinnaker, which is an open source project that was born out of Netflix. Netflix has a very unique to Netflix operating environment, and there’s certain things that Netflix does by default that are not a common practice at any number of other Fortune 500 Global 2000 companies.
The community says, “Wow, I really want to use this open source technology but I can’t. It’s missing this thing that I need to use it in my context,” and now all of a sudden you have this really clear route to, “That’s a paid thing. We should shift that. We should make it compatible with open source or on top of open source, but we should charge for it because that’s our time and effort to make this technology consumable by the community at large.”
I think those things happen all the time, and sometimes the free community and the corporate community, so to speak, the corporate user communities don’t align about a thing and those can be really tricky situations because the corporate overlords tend to want more control than the free user community wants to support by default and those can become extremely tricky situations to navigate, particularly when some of your code, you are gifting out by open source to the community or you’re working on other open source projects.
That can be tricky, but handled well, it’s the art to building a symbiotic relationship between free users and paid users.
Ashley: Awesome. Luke, we’d love to hear your perspective on this one also.
Luke: I’ll add a couple of ways in which we do it at HackerOne and some warnings that it can be out there in terms of listening to feedback. I think capturing it, we have two high fidelity ways. We have an advisory board of top members of our community that we have active conversations with. We pipe them into our core team, they’re on our Slack, we ask questions, we communicate, if they have a question, there’s a product manager overseeing that. So they’re able to talk through some things. Then we do bi-annual surveys which is under our hacker NPS.
NPS has its drawbacks but it’s something that’s worked at HackerOne to be able to be an artifact that me as community leader, can go to customer success leadership and product and engineering leadership and be like, “Here’s what the community’s asking for at scale and here’s the scientific data that backs it up and here’s what the Hacker Advisory Board is saying. Here’s some user stories and all the things I can bring as a conduit of understanding from my community to say, here’s the land and the facts for us to make a strategic business decision based on the roadmap and everything that you see in your purview and whatnot.”
I do see that as two ways that you can replicate that model and those are done by most customer success organizations that I’m aware of. They run a CAB [Customer Advisory Board] and they run a customer NPS , so we just replicated that on the hacker side. The two warnings are, be ware the sound of one hand clapping. Make sure you don’t just listen to the top 1% of your users. Have a good perspective and take their feedback in.
That’s going to be tough because many, at least HackerOne’s community, we’re a power law distributed marketplace, so the 1% it produces an out-sized percentage of the value for our customers. It might not be the case in other scenarios, but you’re always going to have a minority of the people producing an outsized element of value. You got to make sure you’re listening and responding and definitely managing those conversations, and not just acting and taking that straight to the product leader, straight to people and be like, “Hey, this needs to be done because these two people said so.”
Ashley: Totally agree.
Margaret: Actually, I was looking at my notes and I realized I meant to say something else entirely to that question, which is, how important it is to realize you’re not going to deliver everything that the community wants, so you actually have to enable the ecosystem to help you? Whether that is add-ons and the Heroku Elements marketplace, or whether that’s some of the things Hashi is doing right now. You can’t meet all the community’s needs as one entity. If there are ways that you can actually activate commercial, non-commercial partners to do things the community wants, that’s where you really get to that quote you were talking about from Bill Gates. Right, Luke?
Margaret: Yes. Marc, I would love to hear from you at HashiCorp or in your previous experience, how your enterprise customers actually have interacted with the community or the open source community?
Marc: They are the community to a large extent. I agree with everything everyone’s saying as well, the mechanics, we’re all familiar, we run CABs and PABs and we have various different opportunities for various different community discussions. We’d like to get better at some of them. I think what Margaret is saying about, you can’t promise everything to everyone, you shouldn’t be afraid of that either. And Luke is saying, which is you can’t always listen to the most vocal members of the community.
I orient that back to have a strong thesis. If you know where your product is going and you have a roadmap, and I don’t develop product for HashiCorp so I’m speaking out of scope. But generally speaking, if you have those kinds of frameworks, you can be convicted of the right feedback from the right people. There are those out there who would change the roadmap on your product or would say, “Well, that’s not their accountability. We appreciate the input.” And so on and so forth.
On the other hand as Margaret is saying, you have to generate something that is going to be extensible. This stuff sounds obvious to us because we’re all in that world where we build these kinds of products, but it is non-obvious to the market generally. If you build a provider model for Terraform, what do you know? We ended up with 1,000 providers in a few years time, and so on and so forth, so you’ve got to give people the opportunity to contribute there.
As it goes for enterprise, they’re open source developers by night, they’re enterprise developers by day and those things are much more acceptable inside enterprises now. I’ll skip back 10 years. This was really hard. Open source technology inside an enterprise was not a good thing for either as a developer, you weren’t allowed to do it, but also enterprises couldn’t buy it. The procurement and legal teams did not cope with it. They made sure to indemnify themselves and all that stuff, and that’s gone away largely, thanks to things like the growth of GitHub, the ability to just explore all of these repos, and so open source just became embedded even not at a product level, to the technology level.
As that’s gone away, you find you’re talking to the same people or the people who work at our largest enterprise customers are on stage at an open source conference talking about how they did that. I agree they have different alignments, so sometimes they’ll have very specific enterprise problems and we need to solve that, and that’s off open source community roadmap. But in the end, it’s all the same. We want to serve both of the needs, and enterprise communities in particular offer a whole turn back into open source projects these days.
Ashley: One thing I wanted to add on to everyone’s earlier point about, don’t build your entire product roadmap around that one loud voice. One thing I’ve noticed is, you have to create different kinds of spaces for the same feedback. If you’re trying to get feedback about a product roadmap, and you’re doing a CAB, some people aren’t going to speak up, some people aren’t going to fill your survey out. Do not build your roadmap around Twitter.
You really should think about all the different inputs and different way people like to communicate. Because I won’t end up in a CAB and give my opinion. I will write you an email maybe. But just give everyone a different way to input because not everybody’s is going to be the loud voice in the room and a lot of times like we’ve said, you don’t only want to build for that one loud voice.
Agree 100% percent that the enterprise developer is the person on stage at the open source conference and a lot of that work is going back to open source and vice versa. I’d love to hear examples of a program or feature that came from the community, that then has since been canonized by each of your teams. Margaret, I’m sure you have a lot of these examples from your time at Heroku and also now at Armory.
Margaret: They’re many. I’ll give an example around Postgres data service at Heroku and the commercialization of our Kafka offering. The minute that people start asking for things like, “Hey, we want to do point in time recovery, we want to have these certain kinds of backups.” These are great ideas that come out of the community and they end up being really wonderful things to commercialize around. Now it’s a managed database service that’s truly competitive for some kinds of applications with way over priced technologies from, let’s say, Oracle.
This is a really a big deal. Features that the community needed, because they were used to experiencing them in an Oracle context, they wanted out of a Postgres plan offering and when you’re able to deliver them and then you’re able to commercialize them for use in commercial settings, that is just incredibly, incredibly powerful. Likewise, even in an Armory context, you might see a call for certain kinds of integrations to, let’s say, monitor products that are used in the enterprise.
Well, community doesn’t necessarily care about those as much if you’re able to deliver them, then you’re able to charge for them in a way, and people can use the technology they love, even when they’re using something that a corporation buys versus something that they would use for free in their own settings. I find it very rich to get all of that product feedback from a lot of different channels, but you do have to apply your own thesis, Marc, to use your words, in your own rationale to what you choose to do with that information. That’s the final product.
Ashley: Marc, I’m sure you also have some interesting examples here.
Marc: We also have so many. When you say canonized, any line of code that’s contributed by PRs ends up being canonized in the products and so there’s a ton of that. I’d say actually one of the interesting things back in the Hortonworks days, because it was entirely a platform built around Apache projects, and so the Hortonworks data platform was a effectively a set of Apache project that was robustly tested for the enterprise in the middle. How would I go deploy Hadoop in the broadest sense? Meaning just deploy a set of Apache projects.
There were numerous times where a new project would pop up, so Apache Hive and Ambari, and some of which we would sponsor, some of which the community would sponsor. Then eventually just hit some tipping point, which is “this is the feature” and so therefore it gets integrated into the platform. You see that in the larger and the small all the time.
HashiCorp’s obvious examples, each part has a bunch of extension points. There are a lot of people out there doing serious work on providers, and that stuff is canonized and the path that which you use those tools inside any organization. But again, I think it’s just one of those fundamentals. If you need to have a thesis for how that’s designed into the way in which your products will attach itself to the ecosystem, runtimes and the methods you use inside, where you’re hoping for it to scale and scope expand.
Ashley: Luke, I’m sure you have a lot of examples from HackerOne since your community is really the core of the business. We’d love to hear from you on this one.
Luke: A couple of different points and ideas. I think there’s in software development life cycle, we want to shift left more in HackerOne. Most of what the code that is being reviewed is public end points by hackers out there. We want to help move them earlier on in that cycle, so some of the innovation is, “Hey hackers, what do you think about this?” We have hackers that get really intimately familiar with the customer’s attack surface even more so than some of their employees that are now coming on board.
There’s actually this really interesting and rich dynamic that happens when we connect them more. Our CEO, Martin came from MySQL, and so MySQL being open source, he has a lot of really wonderful things that he published. I’ll reference one that he did on Amazon’s blog about community-led innovation. From his vision, from board members like Bill Gurley that we have at HackerOne, we’re fortunate for these really just amazing thinkers, to help challenge the team to think about how we can crowdsource more.
Bill Gurley’s words ring in my ear when he says, “Distributed and emergent is the name of the game.” How can we think about the problems we’re doing today that were manually just start-ups right? barbed wire and duct tape, trying to figure things out, trying to build the airplane while we’re in free fall.
Well, as we evolve and we’re in our seventh or eighth year, we’re going to start figuring out ways that we can open that up to the community where they can contribute greater, and better and provide this pipeline to our customers so that we’re just in the middle, again. I see us building a lot of machinery around that, a lot of scaffolding still, feels like just like startups, there’s a lot of ups and downs. You got to celebrate the small wins when you can get them, make sure you’d actually look back and realize you’re on an upward trajectory.
But community-led innovation, make sure there’s a difference between invention and innovation. It’s not just speakers and labs and Bunsen burners in that environment, where this is where science happens, this is where things go. Most of today is through this crowdsource, this decentralized, distributed creative intelligence. And so companies and organizations that are harnessing that. Everybody on this call, everybody listening, this is the future.
Ashley: Awesome. We’ve talked a good bit about community, when to monetize, how to monetize, what not to monetize. But the core to all of our startups and all of our businesses has been the team that we work with and the team we’ve been able to build, and I think from all my experience, my favorite thing with any of my startups has been building the team and working with those people.
I’d love to hear from each of you when you think about how you build a team, how do you structurally build the organization support as community grows as well as revenue grows? I’m curious how you all think about that and has that changed as you’ve gotten further along in your careers? Margaret, can we start with you on that one?
Margaret: Sure. I would say I’ve seen a lot of different models, and you have to recognize that all models are going to iterate with stage, and be prepared for the answer that is good right now, to not be the right answer that is good in six or 12 or 18 months. That goes for the team, the way you organize the team, that goes for the way you price the product, and you have to be prepared to thoughtfully iterate for good reasons as you go.
Don’t settle too early into too rigid a form, is probably what I would say. If you’re really thinking about the value that your users and customers get and you organize to continue to deliver on that, you’re okay. Just recognize whatever your answer is now is not your final answer.
Ashley: Very true. Luke, what about you?
Luke: It does change over time, so for our just very specific scenario of HackerOne, community wasn’t a division, before it was nested under marketing and that’s pretty common. That’s pretty typical and now community means everything to different people in different companies too, so defining terms, just with any debate or any decision you want to make is, here’s what we’re defining it as today.
And having that open-mindedness and that iteration to be like, what are the things we’re going to look at to say, “Okay, I need another person here. We need to build, I need to invest in this tool to help connect them.” Coming back to… Marc, I’m going to steal your point: environment, education and experience. If we have these frameworks of how community is approached at the company, then it’s just going to be a matter of when’s the right time to add another leader or add another layer of support, or add another tool.
If you can provide this community a space where they’re getting together with their friends, where they’re contributing in a meaningful way and you’ve listened to them, and you’re responsive to them, then the actual way that’s going to happen is it can not just be a community problem.
It has to be an area where the entire company looks at it, otherwise you’re going to have a fractured experience and your community is very intuitive. They sense it and they’re going to revolt or they’re going to go somewhere else to spend their time because there’s something else out there, so you got to evolve with them and your team.
Ashley: I agree. The community will revolt and they will leave you if you don’t do a good job of managing the community and working really, really well with them.
Marc: 100% of what Margaret said by the way. I think that’s really one of the best pieces of advice is, whatever you’ve decided to do is not going to be true another day. So just bear that in mind. Never be afraid to rethink those things.
Community to me is a bit like customer success. There is community, the team, and we’ll talk about that, but community is an idea. It’s a concept. It’s either in the company or it’s not, and it’s the same as customer success. It’s like there’s a customer success team, but there’s a whole mentality around just the total customer experience that you need to live and breathe every day.
At HashiCorp we have developer relations, which is the overarching title to our community team, sits inside marketing. We try to keep it separate. There’s a lot of commonality between how marketers think and develop relations folk think, because we all just want to do the right thing for the customer. There’s a lot of brands and there’s a lot of events we come together on and all that kind of stuff. But we try to keep the discussions separate, so there’s no bending of developer relations towards commercial concerns and whatnot. We do that percentage-wise.
We have a very heavy product marketing team at HashiCorp because well, we keep building products. But generally speaking, I think 35% of our head count is in community and developer relations and about the same inside of go-to-market marketing, and the rest of the percentages are made up from all the supporting functions. We have engineering, design, brand, communications, things like that.
What you find when you think about investment is, community doesn’t scale super well. You can knock it out of the park. We have our HashiCorp Learn experience. That’s where people go on mass. But generally speaking, it’s a non-scalable activity. People are writing educational content, they put a lot of craft into the things they do, so you have to prepare as a leader over-investing headcount versus other elements of marketing where you can scale out quite a lot. As a marketer, there’s that tension, it’s like, “Wow, how many head count do we have here versus here?” But some things scale, some things don’t.
Luke: Going on scale too, Duolingo has an incredible story of them running thousands upon thousands of these English learning things by community leaders. They build with their community by identifying leaders in these different regions that are passionate about teaching English, and then they just empower them. There’s a community team of like six people that manage 3000 plus meetups every month, which is an incredible thing to strive for from that scale.
Building with your community in that way is going to help address, but make sure we can’t forego the head count and that does take a good leader to be able to make that business case for why they need to be there.
Marc: Those paths of scale are interesting. I remember when I got to Meteor there was one person doing community and they had like 220 chapters of their user groups worldwide. I was like, “How are you doing this?” They basically plugged some political movement software into meetup. They were managing to move a whole bunch of user group activity, and the same way as you’d move people to the polls, and I thought it was fascinating. One of the community actually went, I think to go work on the Obama reelection campaign, as well.
Luke: As you said, there’s so much synergy for politics. Anybody ever been involved in a campaign? Holy smokes, these are intense periods of really just getting people rallied around a common cause to go and do something which is vote at the end of the day. That’s why they got to vote with their code or whatever.
Ashley: We were doing this at GitHub, but when I got there, there were, I think a team of three people with chapters all over the world of people throwing meetups, and we would basically just give them budgets for pizza, mail them boxes of t-shirts and stickers and whatever else we could throw in the box. That was the best way to have all these super fans all over the world and a few people managing it, because it’s really hard to hire a team of 1,000 developer evangelists. It’s much easier to just empower your community.
Luke: Totally. ROI on pizza, swag and beer is pretty darn good.
Margaret: It’s funny you should say that about the election, because we actually had a really interesting usage of Armory by the OpenGov team during the Biden election as well, to bring Kubernetes at much higher scale than I had done before. I think it’s that same point around we’re all familiar with this. We know how to run Kubernetes, workloads in production. But when we need to do it at scale, now it’s time for like a commercial product. We’re already familiar with the technology so we can just make that leap because now it’s time to pay. Whereas it might not have been when it was a hobby project before.
Ashley: I’m humbled and empowered to be surrounded by such incredible leadership today. I have been so excited to sit here with all three of you. You’ve all had such interesting perspectives on what we’ve been talking about, and also, I just respect all of your careers tremendously. I’m excited that I already know some of you but we’ll get to know some of the more, hopefully, and one day we all get to work together. I would love that.
But my final question before we wrap up is, what advice do you have for early founders who are in that phase of their company where we have a community, we’ve raised some money we’ve got to monetize? What are the next steps with commercialization? What’s that one key takeaway you think that everyone should take from this conversation? Marc, let’s start with you.
Marc: A strong framework. Have a strong thesis on what that’s really going to mean to close the first deal. Whether that is limiting projects and products, and how do you think about your skew structure? What’s the progression? What’s the real model? Is it community growth truly bottom-up or is it community growth for a point solution and then you do enterprise sales? Going into it with a strong thesis like that is the right thing.
Terrible advice I’ve seen is getting a million monthly active users and a million dollars revenue at the same time. It’s just not going to work. Make sure that you’ve done those things thoughtfully. Then test those theories. Don’t be afraid to throw them out when it turns out it didn’t work, and just continuous reassessment of those things.
Definitely don’t be afraid of the commercial conversation, and don’t hide from it because the default to hiding from it is you are answering those questions that you haven’t posed yourself through every piece of engineering and every give you put into community. But whether you know it or not, those positions can be hard to unwind from too.
Margaret: Well, in terms of externalizing your thinking and your assumptions to get better data than you can generate in your head, I would say be ware the underprice and the overprice, and be prepared to experiment. I’ll give two anecdotes, one from Heroku, one from Armory. For Heroku, we charged a lot more, basically to run an LXC container and a virtualized private environment than we charged to do it in a multi-tenanted way. We thought that, and it was great products, great premium, Heroku private spaces, but then we were like, “Okay, we will charge quadrillion dollars more to do this compliances on top.”
Well, there was a price premium there, but not as much as we thought, maybe. Because once you’re already operating an enterprise scale and SLAs, security is assumed to be built in. We overpriced there, whereas in the Armory context, we’re like, “Wow, this gigantic financial services institution wants to use our stuff. We should charge $1 million.” Well, turns out, that’s actually an underprice in a way, because those 5,000 developers we got for $1 million, well, they got like 50,000 developers so they would pay a lot more.
There’s going to be some turns of the crank when you’re commercializing that are going to make you hugely uncomfortable and you would screw some of it up. And you have to be able to iterate that model, that pricing model is as you evolve.
Ashley: Great point. Luke?
Luke: Innovation brings a new dimension of performance, to quote Peter Drucker. When you’re looking at build with your users, you got to be able to have the framework to fall back on, but focus on listening and being responsive to them. Communities by nature are exclusionary. People are in your community because they’re passionate about the technology you’re building or some other element. Listen to them, be responsive, build with them, empower them under the framework that you have, and also lastly, we didn’t talk much about, don’t under prioritize go-to market.
Don’t just focus, if you might be so excited about all the amazing things your community’s doing, you can just get into the weeds. But I think if you asked many people who have built some of their companies at scale, they probably, especially if it’s a product-led CEO or leadership, they’re going to say, “I should have prioritized GTM.” Go-to market responsibilities are going to go a long way. That’s my words of advice.
Ashley: Margaret from Armory, thank you so much. Luke, from HackerOne, thank you, Marc from HashiCorp. Really appreciate y’all taking the time. This has been incredible. Really enjoyed this entire conversation and look forward to many more of them. Thank you.