August 9, 2016
Ep. #23, Using Continuous Delivery To Find Product/Market Fit
In this episode of To Be Continuous, Edith and Paul discuss how you should use the principles of Continuous Delivery to find product/market ...
My name is John Sheehan. I am the founder of a company called Runscope. We're not really public yet, but if you use APIs in your applications you can sign up for our preview at runscope.com and in a couple weeks you can check it out.
My background more recently has been working on APIs and platforms. So I was the sixth employee at Twilio. I was the first full time developer evangelist. Those of you who know who Danielle Morrill, who is the first employee and head of marketing there, I was her first hire. Eventually, I grew into lead of evangelism role and taking over management of that team. Going into about six, seven, eight evangelists, sort of where it is now. I then moved on to be a product manager on the developer experience team. So everything that the developers interacted with day to day when building their apps - so dashboard, docs, helper libraries, all that sort of stuff.
Last April, about a year ago, I moved on from there and went to a company called If This Then That as their platform and partners lead. I worked with a lot of different companies that we were integrating their APIs and dealt with technical problems and business problems, worked on a big partnership with ESPN for the Olympics; that sort of thing. So, I did the deals and then built the channels on IFTTT. I spent six months there and then I got the itch to leave and go found a company joined by one of another former Twilio employee who was lead API developer there for the past two and a half years. And we are working on solving some of the problems developers run into when they integrate APIs into their applications.
These are some of my side projects.
So, Hero Making is the title of my talk. What is hero making?
So I know when I first heard this term it actually kind of rubbed me the wrong way. I know it's cheesy, this idea of making heroes. You don't really make a hero, right? Someone's either naturally a hero or sort of steps up in the moment and becomes a hero. But, what I saw at Twilio was this sort of philosophy when I came on of helping developers be successful at whatever it is that they wanted to be successful at. What we did was, we saw that this paid off in short term and in long term and we framed everything that we did from the developer evangelism team and developer evangelist program around this concept of hero making.
What can we do to make any given web developer the hero for what they're trying to accomplish?
So, if it's a developer who's in a big company who's trying to get promoted and he goes into a board room and gives a great presentation and makes a CEO's phone ring and the CEO is impressed and he gets promoted to team lead, we helped that developer be more successful at whatever they're trying to do.
If somebody's trying to raise funding and we know an investor and we can introduce that founder to an investor connection that we have, then that's what we're going to do to make that developer successful.
You'll note that none of this says make the customers, or convince the customers, or sell to the customers or developers out there to use our product. That was not our charter. Our charter was definitely not sales. Sales is the last thing your developer relations team should be doing. There will be times when they need to do sales-like things and they need to recognize those opportunities. But, in general they're not sales people. They're not marketing people. They're not biz dev. They're not support. They do all of those things but the ultimate goal that makes the successful developer evangelism team is this concept of making heroes and making developers successful.
Once you have sort of this big common goal for everybody, it's actually really easy to make decisions about what you should do in any given situation. If you're wondering whether or not you should spend, you know, five thousand dollars to sponsor an event, you can think,
"Well, if I go there, is there a developer there that I can help be successful at something?"
If the answer is no, then you don't spend the five thousand dollars to sponsor that event. Decisions get really easy when you have this sort of guiding light that drives all of your decisions.
Once you make a hero, you're friends for life essentially. If you really help somebody be successful, they will never forget you. And this pays off sometimes even three years later. So, it's been three years since I started at Twilio and people still email me that I met at my first event there and ask me Twilio questions and are still Twilio customers and have decided to build start ups on it. It really pays off, even sometimes six, twelve, eighteen, thirty-six months later. So, this concept of enabling somebody really endears them to you forever. They will never forget the person that helped them launch their company or get promoted or launch a new app or get on the front page of Hacker News or something like that. That really means something to people. And it's also really satisfying from the developer relation side to know that that's your goal, for helping people and enabling people. There's really no better feeling than that.
You'll ultimately make your money from your big customers. We all know how the graph looks for cloud services. The vast majority of your income comes from the huge whales at the top of your graph. But a lot of your exposure and a lot of those customers can actually be generated through the community. So, by building the community, making heroes, you can actually funnel more people into the top end of that graph.
Alright, so here I've got a series of slides of, I guess, platitudes maybe, for lack of a better word. These are in no particular order and I would actually encourage you at anytime to grab one of the microphones and ask questions. I actually prefer Q&A to giving just a straight up talk. So, please feel free to raise your hand and interrupt at any time and if I don't see you just whistle at me or something.
If your product stinks don't even bother evangelizing it. You cannot pour gas on water and expect something. Right? You can only really pour gas on fire. I should have worked on that one a little bit longer. But, you can only really pour gas on a fire, right?
So, if your product stinks you should just focus all your effort on making sure you have a good product first.
So, I was really lucky. When I joined Twilio I was in love with that product. And I could see the vast power it had and I was in the .NET community at the time and I could see the .NET community wasn't really getting on board with the API thing at the time. They were still very kind of soapish, soapy, I guess. And I just saw that there was an opportunity for me to take this to a wide audience of people and help enable them. But, I had behind me a really great product. It was the easiest telephony product out there to get started with. You could get started in five minutes, you could view all your logs, you could buy your phone number in seconds. You couldn't do this anywhere else. So, I was basically really getting on a rocket ship of a product. The first thing I would say is, if your product stinks then don't bother with any of this other stuff.
The best products enable you to do something you couldn't do the five minutes before you discovered that product.
In my case, I had this app that I really wanted to build for Twilio that would, in the case of inclement weather for a sports league I was running, instead of everyone calling the hotline over and over again, I wanted the hotline to call you. I wanted to build this for, I don't know, three, four years, and I'd looked into building it myself with Asterisk and setting up my own instances and all of that stuff. And it just wasn't possible. And the moment that Twilio launched on TechCrunch I actually ran around the house - this has become sort of an infamous story. I was so excited that I finally could build this thing I wanted. It was literally like leveling up in a video game; like someone just awarded me whatever pendant I needed to have the skill to now complete the boss level that I wanted to complete.
Make sure that your product sort of has that moment to it where you're giving somebody a skill that they didn't have before.
Alright, my next bit of advice :
This is the biggest thing people come to me and they say,
"Hey you were at Twilio. We want what Twilio has. So, we want to copy everything that you want to do."
I have sort of a... if you thought 'hero making' was a cheesy term, this is going to blow your mind... I had this phrase once where one of our competitors was copying our every move. Their documentation was updated to look like ours. They would go to events and it was weird. It was like sometimes looking into a mirror. And I said to my boss at the time, I said,
"They can copy our hands, but they can't copy our hearts."
Which, again, is so cheesy. But it really was true. They could only see what we were doing externally but they didn't know what was going on in our heads. And they did not realize the meaning behind what we were doing. They don't do it anymore because it wasn't successful for them. I think that's because they weren't genuine about it. They saw tactics and they tried to copy tactics and then, you can't copy tactics and get the same results, if you don't understand the meaning behind those tactics.
Also, your customers are probably a lot different than Twilio. So just by the nature of the problems that they need to solve are probably not the same as, you know, Telecom. That's kind of a unique area. So, you would be doing a disservice to your customers if you try to apply Twilio's direct ideas to somebody else.
Also, if you're copying somebody a lot, you can't keep up with them. They will always be out innovating you. You will always be lagging behind on innovation behind the person you're copying. So, if you want to get out ahead of it you've got to sort of set your own pace. I would encourage you to find your own voice and your own personality and your own style to do it and try things and see what works and figure out what doesn't work.
When you copy somebody you actually don't know what they regret. We would actually go to events or do certain types of blog post or whatever and then they would just fall flat and we wouldn't do it. Well, our competitors would copy it to the point where sometimes we actually felt like doing honey pots where we would do things to see if they would copy them because we knew they weren't going to be successful. But, that's just how much they copied, how we were being copied and how it didn't work out for them. So,
Define your own voice and personality and I also would say don't copy anyone, really. Blaze your own trail.
Consistent voice is an extremely important thing for evangelists. So, every touch point that a customer has with your company you want to make sure that they get the same message. So, when they come to the website and they read the documentation and then they email support and then they run into evangelists in the field and then they talk to sales later when they're ready to convert, that they should get the same message every time from each one of those touch points. And so,
Building a consistent voice and having really one message but many messengers really goes a long way to building trust with your potential customers and developers especially.
Also, developers have a really, really high BS bit. The moment they sense any sort of BS they'll shut you off. Like you know this. We deal in true and false all day long. So, the moment anything deviates from that we get very uncomfortable and anytime somebody, we feel like somebody's trying to pull something over on us, we start to get a little unnerving. And this happens a lot when you will run into and evangelist who is also somebody who codes in the field and then you call up sales and then you start getting pitched to and somebody's giving you a solution to a problem you don't have. That sort of raises a lot of red flags.
So, making sure that you have a consistent voice. Making sure that when you describe the problem to somebody in person vocally that you're using the exact same terms that are in the documentation and that your support team is using, and that when you talk about bulk pricing you make sure that you're talking about it the same way your sales team is talking about it. Really making sure that you have a consistent message across all of the different mediums. That builds a lot of trust.
Alright, next round of topic. No questions yet on any of those? I go fast so, you're going to have to stop me if you want to ask questions.
So, the human aspect of evangelism has really become most of where it happens now. So, when I first started Twilio we were nine people. Danielle and I were doing all of the events, all of the blogging, all of the tech support and all of the other marketing things that we were trying to come up with. But, now most evangelist mostly spend their time in the field. So, this is really important that you get people in the right place at the right time.
For instance, one, you want to make sure there are developers there, or there are customers there that you can help be successful. If there are not going to be customers that you can help there, it's probably not worth your time.
You want to do events that you align with your values. So, if you are really pushing open source things and you send your developer evangelist to some big enterprise software conference they're probably not going to thrive there because there is a mismatch between what is important to them and the kind of customers that are there.
You can't put developer evangelists in situations where they're not comfortable and expect them to be authentic. Authenticity is so important in that you really want to make sure that they're in the right environment for being effective.
When you are at the events, you want to make sure that you are present and available.
So, we did this thing at the time, before we came up with the track jackets, which was not my idea, but before we started doing the red track jackets, which you will see at pretty much every Twilio, or any event Twilio is at now. I think we only had seen that at Facebook up to that point. It made it really easy to be found.
So, we would go to an event and we would say "Hey, I'm at the front lobby. Look for John in the red track jacket and go get a t-shirt," or whatever.
Just being present, just finding the right place to sit and hang out was always valuable and always lead to really, sort of like, facilitating serendipitous conversation. So, you just know where to be to run into people and to get the kind of conversations that build relationships. We would spend our time not in session rooms, but in the hangout room or in the hall, or if there's a hackathon, you are always at the hackathon. Or if there's a room where there's just a group of people working, that you go work with them and then you go be amongst them. Make sure that you are out and about and very visible to people and available.
That's a great question, metrics and measurement.
I'm going to give you right now the "It Depends" speech - that depends on what your most important goal is at any given moment. If you're really in a drive sign ups mode and the only channel that you're using to generate those sign ups, then, you can sort of say, "Hey we went to this event, the next day we saw a bump."
This isn't really tenable over a long term because it's really hard to source people. So, we tried tracking sources for a while and what we do is we would meet somebody and 6 months later they talk to sales and then they would sign up and become a customer. You can't really attribute that to any given thing. It could have been the documentation that they read even before they met the evangelist.
We tried handing out tracking codes at events. Those wouldn't convert. People just wouldn't even use them. To me there's really nothing that concrete that you can do to drive value.
However, there is one thing that's sort of more femoral that you can use. W hen you leave an event, you know if it was successful or not. Like, your gut will tell you, "That was a waste of my time" or "That was really successful. I met a lot of high-value people."
If you start taking notes and start tracking your events over time, it's really easy to start seeing patterns of events that really paid off based on sort of your gut reaction to them.
So, I would pay really close attention to that. And I know that's not a very good measure and iterate type answer. But, I think it's a lot more accurate in finding what was actually working and what wasn't working.
Meet as many people as possible. Your goal is to build relationships and you can't build relationships if you don't start a relationship. So, I would always go to parties where all the people are. You want to be there. Right? Again, be ready and available for people.
I had a standard set of six questions that I could run down quickly with somebody to sort of get their background and get contacts and what they're doing, and again not to sell them anything but, just to understand where they're coming from.
I mean, you can get a lot of information from people very quickly and sort of build the base of a relationship. I'm a talker. If you ever get to me one on one, I just don't shut up. This was actually a really difficult lesson for me to learn was that when I went to events, I actually had to shut up and listen for a while. So, eventually, that's why I developed this standard set of questions because I knew I could go through it and I wouldn't jump ahead because I was thinking about the next question and not the next thing that I wanted to say. Listening is a really important way to build relationships. You can ask my wife how bad I am at that.
Drink-ups are really popular, kind of trendy right now. So, some of the more established companies have sort of established these as sort of a regular thing that you can't necessarily compete with. However, if there are no drink-ups at the event that you're going to, there is no better way to spend $500 at an event than a drink-up. You get all the people who already like you coming to hang out, talking to all of the people who are curious about you or are just plain social. It's like the ultimate mash up of the right kind of people. And they're in a relaxed, fun environment where they just socializing. It's very informal.
One of the companies I advised, Xamarin, has started doing drink-ups, and they're like "We love it! Our best customers come and then they do our job for us talking to all these other people that are just curious about us." So, drink-ups are really valuable.
If you can come up with a way to do them in a unique way, I would encourage you not necessarily to follow the mold. Again, blaze your own path. Also, make sure that you're not competing with official events because you don't want event organizers to feel like you're working against them in anyway. So if there is an official party, either plan your party after or before or the night before. And also try to make them as open and inclusive as possible. Nobody likes being shut out from, like, a private event or anything like that. So make sure that you can get as much room as you can afford and yeah get everyone in the same room. They payoff over and over again.
I'm going to talk more about this to tease, more about this after the cameras are off because there are some specific tips and tricks I have here. This slide is incomplete. It's a bad copy and paste.
Ultimately though, you want to sponsor things that, again, reinforce the message that you want to do.
So, if you are a framework, you want to be at conferences where there are going to be developers who are using your framework. If you're an infrastructure tool, you want to be at the place where the developers are hanging out relative to that. The one thing that I will say now is that everything is negotiable, so, keep that in mind when you're talking to event organizers. But, be respectful, right? So, they have things that they're trying to accomplish at their event and they have a lot of balls, they're juggling a lot of balls, trying to get a lot of sponsors on board. But make sure that you feel like, after a sponsorship deal is done, if one side feels like they were ripped off then something went wrong there. So, both sides should feel like they got what they wanted out of it.
Most event sponsors will also be really willing to work with you to come up with something new and unique. So we started coming up with all sorts of different kinds of sponsorship ideas like they would want to throw in tickets and so we would say, "Alright you can throw in tickets, but we're going to turn around and give those to our community." So, now we're giving something valuable to our community and they're really willing to work with you if you approach them in a reasonable way and are not trying to undercut them in anyway.
Sponsorships, you can spend a lot of time and money on, but I would actually be really careful how you're spending that time and money and make sure that it reinforces the message that you're trying to send and associations that you want to have. We'll talk more about that.
Evangelists are your company personified. This is the human representation of the abstract concept of your company out in the world. When I quit Twilio, somebody sent me an email and they said, "I can't believe you quit Twilio. You were Mr. Twilio." And from that person, that was actually a really high compliment to me that I had done such a good job representing the company that they had so closely aligned my personality and what I had done with the company in that they felt like the company was losing something from that. That was a really strong validation that this idea was valuable. So, I kind of went recursively there. Anyway...
You want to make sure, though, that when somebody's representing your company that, one, they're really good at consistent voice and, two, that they have extremely good judgment.
So you're sending this person out into a lot of unknown environments, around a lot of unknown people and factors and you're asking them to basically speak for the company authoritatively when they may not know all the answers all the time. We would get the craziest questions coming in and you have to stand your ground and be able to speak as if you are the company in a way that instills trust in people, even if you don't know the answer.
When you're evaluating for hiring evangelists, one of the top things you should be hiring for is judgment. Any of the issues that we had with evangelists out in the field were always them exercising bad judgment â€“ misinforming a potential customer or just making basically a bad decision about how to handle a given situation. Judgment should be way, way, way up on your priority chain for hiring evangelists.
Q: How should evangelists respond to questions above their technical level or that they don't know the answer to?
I think you partly answered your own question, but I will reinforce what you said. So, one, you would never want to bullshit somebody, right? The goal is you want to build trust in a relationship. Just the way you wouldn't bullshit a coworker, necessarily, because you want to have a trusting working relationship with them. But you can, in that situation:
So we only hired working engineers; you had to be previously working as an engineer to come onto Twilio. That was really, sort of a... what's the word I'm looking for? A clout thing, right? Like when you met me you had to respect my technical ability or there was no way that you would take technical guidance from me.
I was a .NET guy. This is really a difficult background to have when you're in Silicon Valley in the Bay Area. So, I immediately started trying to brush up on everything else that I could â€“ Rails and Python and Node when it came out. I felt like, if somebody came to me with a Rails problem, I needed, at a basic level, to understand the most general kind of problems that somebody that was working with Rails and Twilio would encounter, so that I can help them solve it as much as possible. It's actually really hard to find somebody that, like polyglot programmers just in general are hard to find, is a polyglot and can communicate in all those different languages and all those framework is very difficult.
So a good way to evaluate for judgment is to take a prospective evangelist to an event, pay them for the weekend to go to a code camp or something small or regional, and actually observe them 'in situ' as I would say. See them in their natural environment.
This is kind of how I got hired at Twilio. I went to Vegas with the company that I was working for at the time for a different trade show and Twilio was at another one. And I snuck into their trade show to meet Jeff and Danielle because I was such a huge fan of the company, and I walked up and Danielle was like "Jeff! This is John. He's my first choice for developer evangelist job!" which is news to me at the moment. And he's like, "Oh great, let's chat.Why don't you come to dinner with us?" So, we went to dinner that night and there was twenty of us at dinner and there was a whole bunch of our Twilio investors at the time and Twilio customers and other coworkers.
And, for an hour and a half they got to see me do the job because they knew I had all the Twilio knowledge, but they got to see me interact with all these people, and see how I exercise judgment when asked a question I didn't know, and how I would deal with prospective customers in helping them with technical problems. And sort of, on a lesser level at that night, but evaluate my technical abilities.
So, if you can see that, that was a really powerful thing. And when I left there they said, "We want you to come in for interview. We're going to fly you out to San Francisco." I was living in Minnesota at the time. "We want to fly you up for an interview." They sent me an offer letter two days later. Just having that experience was better than any interview that we could have had.
Also, going back to the facilitating serendipity, there's certain people that have a knack for putting themselves in the right place at the right time. One of the evangelist that started shortly after I did, John Briton, had this amazing ability to always be in the right place at the right time. If there was a reporter around that was writing about an event, somehow he got an introduction to that reporter every single time. He got more press than any evangelist I've ever seen everywhere. He's at GitHub now. He's one of the best evangelists that's ever been out there from a public facing standpoint. He did a big presentation at New York Tech Meetup, right as Twilio was taking off, that Fred Wilson ended up calling the best software demo he had ever seen. The way he finagled into that, we weren't really in New York company but he got in there. He facilitated serendipity. That got us more pages views. For the longest time that was a record-setting day was after that New York Tech Meetup thing.
Find people who just have a knack for being in the right place at the right time.
When you watch them at an event, you will see if they are picking valuable places to go and you can sort of pick this out of the wild.
Q: How do you keep an evangelist's ego in check?
One, is don't hire people who are already famous. That may work in some situations, but in general if you're hiring somebody who is already famous, you're doing that to sort of use their community to boot strap yours which may or may not work.
We also very much look for people who had a track record of doing successful things, but then how they talked about it once they were successful. So when they launched a project â€“ was it all about them or was it all about the project? Or did they show humility when press wrote about them? The way they spoke, was it like "Oh yeah, our team did a really great thing," and not necessarily even if they did the great thing?
I think you can look for humility in people. This is again something that I very much struggle with because I'm not very humble. But I try very hard to be when out in the community and amongst my peers and coworkers. People will also tell you if you ask for a reference â€“ humility is something that will almost always come up if they are truly humble. So I would look for that in the references as well.
How do you convince an evangelist to join your company? This is sort of like the purple cow, right? Somebody who can code and who can also talk about code. The thing is this is a great job for the right kind of person if you can find them. One of the ways, again, going back to Xamarin, that we attracted a whole ton of candidates when they opened up an evangelist position is that there was a whole set up perks that I think a lot of people don't think of when they think of evangelism job.
One is that you will build an immensely huge network. I was an evangelist for 18 months, met, I think... I use this as an illustration point for some people: my LinkedIn went from like 50 connections to like 600+ and over 800 in months. If I want access to any investors now, I have access to numerous investors. My connection to Heavy Bit came through my Twilio network from doing events around Heroku and meeting James through that. My network is invaluable to me now and it was solely because of that. That was worth doing that job just for that.
But there's other things you do. You travel enough that it doesn't suck anymore. You know there's a threshold that if you fly enough the airlines will start being nice to you and treat you nice, and give you better seats, and you can go through security faster, and you start not to dread it as much. So, somebody who really likes seeing the world, you can see a lot of places. We were a US only service at the time. I saw the US about 50 times in one year. I don't ever want to go back to... I won't name the city. Again, but they all get to go to exotic international locations now. You get to travel a lot. You get to see the world.
You get to wear a lot of hats. So if you are thinking about being the founder of a company in the future, you will get really good at support, you will get really good at biz dev, you will get really good at marketing, you will get really good at sales, even though you're not directly doing those things.
You can build skills that you don't have really quickly. So if you want to be a better writer, you can say 'Well, I'm going to be an evangelist and I'm going to focus on writing blog post, the technical content." You'll become a better writer very quickly.
You can become a better presenter and speaker. If you saw me speak my first talk when I first started at Twilio, it was a mess. Now I think I do OK, but I came a long way over giving a lot of pitches. In other situations where I've needed to give pitches that skill paid off immensely.
Like I said, it's really a great base for being a founder later. So, if you can find somebody who's looking to do that later but is maybe lacking in some of the skills; I knew nothing about marketing when I came to Twilio and I learned a lot about it very quickly. You can entice somebody by giving them that carrot.
It also gives people who are social and maybe feel locked up in a big enterprise company the ability to spread their wings and fly little bit. I know I felt that way. I was working in the basement of a medical building in the Twin Cities with five people and it was dead silent all day long, and it drove me crazy because I wanted to be social. My outlet at the time was Twitter, but when the opportunity arose to actually go talk to for a living about code, what I love doing, I think that was really when my career started to thrive.
The soft benefits are a great way to attract evangelists.
This is really tricky and this goes back to the judgment thing because what will happen is when you're out in the field you will get asked about competitors a lot. So, the most common question we got is, "How do you compare to so- and-so?" (that people thought was a competitor). That actually was mostly copy cat, but that was the first question that came up.
The first thing that we had was a great product. So, that helped a lot. If we had had a terrible product I would have dreaded that question every time. But, because I knew our product could stand up to them, I could confidently say to somebody when they asked me that question,
"Go try us. Go try them. We'll give you $30. If you need more than $30 to check it out, I'll give you as much promo code as you need. If you have any questions you can email me directly. You can call my cell and ask me questions. I'm here for you to help you get through this. But just try them both for a week and then come back, and then tell me which one you think is better and which one will work best for you."
I'd say 90 percent of the time that person came back. That other company may have had feature parity, or even had more features than us, but they were not ready and willing to help at the level we were. They didn't have as good API documentation. They maybe didn't speak as consistently about stuff.
So we considered competitors and any conversation about competitors to be a huge distraction to our underlying goal. Every minute that we spend thinking or dealing with competitors was a minute that we were not spending helping to enable one of our customers be successful at what they were trying to do.
It does get very hard when you're out there a lot and you get asked a lot about it.
If somebody wants to write a blog post that says "Oh I ran into so-and-so from Twilio and they told me that you know, such and such has lots of bad time, or down time..." Now it's attributed to me and now it looks like I'm dragging them through the mud as much as they're dragging us through the mud. So, never, ever, ever speak anything negative about a customer, not a customer, I wrote customer on my notes, but about a competitor publicly.
Any other questions about that? That's usually something that people like to talk about.
Content, kind of, is not fun to generate. Not many engineers like writing for fun. Again, it's kind of unique thing. So you kind of just have to grit your teeth and do it. If you get big enough, you can separate this out so you can have dedicated content people that write blog post. But I guarantee you nobody's going to write better technical blog post about your product than one of your developer evangelists that's out in the field talking to people and seeing their problems first hand.
We kind of did some things that make it easier to write a lot of content.
One, we wrote in batches. So, I would sit down on Monday and I would write as many blog post as I could kick out on Monday and then I would schedule them for three, four, five weeks out and then I wouldn't have to deal with that again. It actually makes it really easy once you get in the rhythm that you even crank out more and more stuff. So, batching it up instead of waiting until the last minute for a deadline to crank it out always helps a lot.
You can also get a lot of mileage out of a piece of content. I would write a sample app and then I would release it on GitHub. Then we would announce that such and such sample app was available on GitHub, and then a week later I would make a screen cast. I would talk about how I built it and show going through it. That gives people who learn that way that content. And then I would take that screen cast and I would put it in a blog post and I would rewrite the blog post, basically the text that I spoke in the screen cast. Then I would take that blog post after seeing feedback on it and then I would re-purpose it into documentation that was formalized based on how people reacted to it. So, if people weren't getting really hung up on a certain part, tweak it, put it in the doc.
You can take one piece of content and basically use it four or more ways very easily.
You can also very easily get content onto other sites. Everyone is starved for content. Who here would not take a guest post about your product on your blog right now? You're all raising your hands. Because the cameras can't see you, you won't believe me. But if somebody came to me and said, "'Hey, I want to write a post about how we work with your products and we want to run it on your blog." I'm going to say "Yes" in a second. That is like a complete no-brainer. So we would take Twilio and basically apply it to some other technology and framework and run it on their blog. It's good for SEO, and it's good for reaching an audience that maybe you didn't reach before.
You can also do sort of the inverse of that. You can outsource a lot of your content. So, one thing we would do is when somebody would build an app that we would come across on Twitter or something, we would say, "Hey, we love what you built. You want to talk about it?" We had a standard set of questions you could basically just email back these questions. Or, if you're really feeling ambitious, sort of write a guest post for our blog. And for a developer that might be out in the middle of nowhere who's maybe looking to increase their profile, getting on the Twilio blog would be a big deal to them. So we're helping them succeed at their goal and incidentally we get a great blog post that sort of lays out a real world use case, that gives another customer that comes by later validation that what they want to do is going to work. So you can outsource a lot of it too, if not to your customers then to other potential partners.
I'm not going to poll the audience. They're not for what you think they are. So, this is probably the number one misconception about how people would copy Twilio and not get the value that they were expecting. A lot of times people would say,
"Hey, we've got a new product. Let's hold a developer contest. We're going to draw in all these developers who are really interested, or maybe curious, or maybe heard of us before and want to do what we're doing and they're going to come. And they're going to build all these great apps, and then we're going to have this amazing winner, and we're going to promote that and I guess that'll be all they want to do. So we'll give away a big prize. We'll give away $10,000 or whatever. And we'll do this one time and it'll last three weeks."
And then it's over and then you don't get any more value out of that. So contests, you should never really look at them.
I've never found [contests] to be effective for generating new sign ups or new customers to your platform. What they're really good at is generating content opportunities.
So, you put a contest out there, we ran one; this is actually how Twilio came to know me. I entered their developer, it was a netbook at the time, netbook contest and won. And they sort of knew me and wrote about it. But every week for years we ran this netbook contest. Crappy $200 Dell netbook. Did you guys win too? Yeah, yeah. So PagerDuty previously won as well.
And all that did was give us great blog posts to write about every week. We'd get 5, 20 depending on the topic. We would pick a new area of something that we didn't have contact for. We would get a whole bunch of sample apps. We would have like 10 things to write about. If I could tell you right now that you would get 20 leads on good blog content for $200, would you do it? It's insane. You would be insane if you didn't do it. So, we spent maybe $25,000 the first year on, we didn't even call them prizes, but more like rewards for doing something cool, or recognition for doing something cool, a token of our appreciation for that.
And everyone's like, "Man, that's a lot of money to spend on prizes just give to away for nothing." We had a blog back log of like 400 posts that we never even wrote. I mean anytime we wanted content, you could go to that list. You'd have a sample app. You had a screen cast. You had everything that you wanted to write a blog post in ten minutes. We couldn't even get through it all, we had so much. Like, that was invaluable. We had content for literally forever from that.
If you can use contests to support some other goal other than lead gen, I would try to use it and appropriate them for that.
Any other questions about contests? Alright.
So, relating to contests, hackathons, which were not really a thing when I started so, Startup Weekend was pretty much the only ongoing one. Now we can't walk down any street in this town without tripping over a hackathon. It seems like they're everywhere.
They're a really great opportunity to help people solve problems and get really amazing product feedback at the most important time. So that first five minutes that somebody's trying to use your product is by far the most important time.
If you get them over that hump, you've pretty much won if can solve their problem in five minutes. So, you're sitting next to them at that moment. You can't replace that anywhere else. And it's actually way better than casual user testing because the casual user testing environments are sort of fabricated. But at a Hackathon, they are coming up to you and they're like, "I have this problem. I think you can solve it. Can you help me do it?" That first 15 minutes is immense.
I mean Groupme's success, we like to take credit for it as evangelists because our team was there when they started at a TechCrunch disrupt. Bu t Danielle and another employee, they were at the point where GroupMe was like, "Hey, we have an idea." They sat down with him, they made that first prototype and 18 months later they were bought by Skype for tens of millions of dollars. I mean that's a huge, huge success for Twilio as a company because of the big customer, but also for Skype for getting GroupMe the outcome they wanted. So, being there at that moment is immensely valuable.
The problem with hackathons is that they're starting to get a little tired. So if you've been to them, you know that they don't really have that same feel anymore. Part of that is because of prizes, which I will talk about in a second, but I would never do a hackathon just because everyone else is. If there's a long list of sponsors, first of all, you're going to be drowned out anyway; and even if you get that 3 minutes at the beginning to pitch your product it's probably not going to be that valuable. You could probably get the same value by tweeting about that hackathon hashtag on Twitter and saying, "Hey we're available via Twitter to help you if you run into any problems."
What I would suggest is that you actually try to find a way to re-invent the traditional model. I don't know what this is yet, so I'm working on a developer tool. In a couple months when we launch, I'm going to be very motivated try to find a new model for how a hackathon could be run. I have some ideas, but I don't really know what they are yet. But find ways to stand out from the noise without sounding desperate, looking desperate, looking like it's a stunt, or looking like you're pandering to developers. That's the worst. Developers just don't want it. It feels more like a sales pitch than like a genuine outreach for help.
So, this leads us to prizes. Prizes are great. Like we said, we used them as a token of our appreciation. Like, it was the crappiest Dell netbook that I got when I won that Twilio developer contest. You've no idea how proud I was of that netbook. Like, if I could have ran down the street being like, "I won the contest! Yes! I'm the best developer in the world!" It literally took me 15 minutes to write that out, mostly because Twilio is awesome, but I was so proud in that moment that I was on the Twilio blog and that I won that. And that netbook to me, represented that success in that moment.
So, what I would try to give people or what I would do is that cash never gives people that feeling. Cash will only make somebody feel like they were a hit man in a way or a hit woman.
Cash prizes don't build relationships, don't attract the kind of customers they want. They attract mercenaries who only care about getting paid and moving on.
So, all these hackathons now that have all these big cash prizes, show up, the company will drop ten, twenty-five thousand sometimes fifty-thousand dollars on the lead prize at TechCrunch disrupt, I would be very surprised if they still had a relationship with those developers that won that hackathon since the last disrupt. It's just not the kind of developer that you want in your community that's going to pay off for you in the long run, that's going to go to events and speak for you, that's going to be on Twitter advocating for you and all that stuff, answering questions for you on StackOverflow or that sort of thing.
Cash prizes, just not worth it. You should just walk into a hackathon and just randomly hand it to somebody and walk away because that's as much value as you're going to get from it.
Alright, so, that's all my random topics.
Sure. When I left evangelism, which is about a year and a half ago, there were six to eight evangelists and maybe six to eight other marketing people. Most of those marketing people were in support of evangelism â€“ community managers, event coordinators, that sort of thing, logistics for getting to events. So, like when we would go to an event I would show up, and there would be a box of t-shirts waiting at the hotel shipping desk for me; I didn't have to worry about that anymore. I also got to graduate from checking in a huge box of t-shirts at every airport when I was running around the country which was really unfortunate.
Sales, I don't remember how big it was when I left. When I started we were 9 employees, when I left we were about 120. I'd say it broke down to about 50 percent engineering, 50 percent everything else. So, if you'd distribute marketing and sales and biz dev amongst the rest of that 50 percent. What was the rest of the question? Oh, that was it.
For the longest time evangelism was Twilio's only marketing channel. That was the only way that we brought people into the funnel. As we started expanding, we started to do online marketing stuff. That was good because it worked for us, but it was actually kind of frustrating for the evangelism side because we couldn't measure anything that we did anymore. So, when we were the only marketing channel we just attributed everything to what we did. It made it really easy to know when we were moving the needle or not.
We set a goal to generate a piece of content for everyday in the year. Danielle and my goal was that by the time we had the team we wanted, which was about seven people when I left evangelism, that we were at one event for every day of the year and we had one piece of content for everyday of the year.
I don't think there's too much content out there. I would just try to make sure that you keep the variety up much as possible.
We kind of got in a rut for awhile where the customer interviews, which were so easy to make, that if you went to the blog you would just see ten of these, if you read them in a row they got boring because they were mostly the same â€“ these customer interviews over and over again. We kind of got addicted to that one because it was easy and it was relatively successful. We could have probably been coming up with a better content that was more successful, but I don't think you can have enough.
And it's also important to have it be everywhere the developers are so I was constantly on Stackful asking and answering Twilio questions as a form of content. A lot of times I would take those questions and turn those into blog posts as well because you've already taken the time to write out a big technical answer. You got a post ready to go there. I was always on Hacker News making sure that make we were responding to any news that broke appropriately. You know Quora questions. You know, wherever else developers hang out now, be on App.net. Okay nobody laughed.
So, they didn't. Not in any measurable way. So people would lose them. They didn't really seem to value the $10, especially when we're already giving you $30 free. It might work if your trial structure is set up a little differently or if you're really giving them something like of worth. But most people are not really going to get too attached to a $10 promo code, and the ones that are probably not going to be very good customers.
Sure, since you don't have a mic, I'll repeat the question just to make sure. So the question is, how can you make sure the hand off between evangelism and sales is smoothly probably in both directions, right? Make sure that you're getting the right person on the line.
I would always tell people, a developer who came in and they're like, "Yeah, our bosses are kind of pushing on us to do this."
I'd be like, "Why don't you just prototype it and then once you have the prototype just stop there and let me know. And then we'll go back and we'll hook up your project manager or your technical lead to our sales team and I'll be there all the way through. So, when you go off to sales and you still need me, I'm still here for you. If your sales people start asking technical questions, I'll still be there for them."
On some of our bigger customers we would actually go to the extent, I did a lot of sort of incidental sales engineering, where they'd say, "Hey, we want to use it."
And then they'd throw a developer at me and the developer would come to me and be like, "Alright, I guess we're doing this because the business said we are. Can you help me?"
I'm like, "Alright, I'm flying out. I'm sitting down with you for a day and we're going to hack it out. Right, we're just going to do it."
And then he accomplishes the goal. He goes back to the person on his side, he's like, "Yeah, they were really helpful. We can build this. We can do it. It's technically feasible."
They go back to the sales person and continue doing what they're doing.
I try to make sure if I was the original point of contact that I stayed involved all the way through the sales life cycle essentially.
If you can get your evangelists to do sales engineering, which is kind of not. It's a different thing, right? Most sales engineers are compensated like a sales person, based on commissions, or a lot of them are, maybe not most of them. So the incentive structure is a little different because you're trying to close a deal to get the commission. You're not necessarily trying to build a long time lasting friendship or relationship with them. But if you can get your evangelists to do that, then they're going to understand when somebody comes to them what the sales side of that life cycle is going to look like. You can tell them ahead of time what to expect and who they're going to hear from and when and what's it going to be. You can probably anticipate what the objections are going to be from their bosses ahead of time.
There was this common set of things that we would run into and I would be like, "Oh, when they ask you about that, you can tell him this, this and this." Or we would lay it out in an email for them. So you're kind of like enabling them to continue to be successful regardless of what happens in the sales cycle, and that the people that are in the company making decisions won't ever feel like they're in the bottleneck.
I'll admit we did not do a great job of that which is why I eventually became the product manager for developer experience. I felt like I was ready to explode with product feedback. I had literally been accumulating it. Do you remember Kirby the video game where he just gets bigger and bigger, like he was going to explode. I had been doing it for 18 months and everyday I was sitting down next to developers in that first 5 minutes and watching how some of the things that we had changed weren't working for them anymore. I would just go, we didn't have doors, but theoretical door knocking on the product managers and would be like, "We need to fix this. This is a huge barrier." Then when I became the product manager I got to fix some of those problems before I took off.
When I did become the product manager, I knew all those evangelists had so much data. So I actually set up a weekly meeting with them and I was like, "Alright, you can just lay it into me right now. You can't hurt my feelings. Like, I know there are broken things. I want you to tell me like the three things this week that you saw that are the most pressing things that cause somebody to give up on us at a hackathon or something like that." And they would just lay it into me and tell me what was going on.
So we started doing those regularly. And I kind of wish I had been there longer to solve a lot of those problems, but it was an amazing feedback channel and we made sure that the remote employees, because generally evangelists will be remote, had the ability to get into those meetings and weigh in just as much.
I also did that with the support team. So your support team â€“ under-appreciated in a lot of companies for just how much day to day â€“ they could probably list off five things off the top of their head to make your customers lives better. If you have a dedicated support team, a lot of time they don't get asked that question. I just tried to make it a structured process to make sure there was time set aside for that every week.
I think we hit Twitter in its sweet spot for developers. Like I use it now and maybe I'm just not interested anymore, but I don't seem to get nearly the level of interaction that I used to get from real working developers on Twitter. For us at the time, that's where everyone was hanging out. That was before Twitter was a media company and people actually had code conversations on it. So, we found it really valuable and it was really a good way to see how something was playing out very quickly too.
If you start your first press story and it hits TechCrunch and everyone starts talking about it on Twitter, and maybe like they're not quite getting it, we would go back to the other press outlets and try to adjust the follow-on story. So somebody else was going to pick up that story later, then we would adjust it. Or we would go back to our blog post and edit to be more clearer on points.
It's the tightest feedback loop you can get other than sitting next to somebody. So, if you can watch it for that.
I would also, on the complete opposite side of that, advise against watching it too closely because when I switched to become product manager and I stopped watching the Twitter feed as intently as I had a good for the previous 18 months, I felt like I had escaped this bubble. We always talk, "Don't get myopic, don't get myopic." I was like, "We're not. We have a good pulse on everything." T hen I stopped watching Twitter and I completely felt like we were in a bubble. So, in a way, you kind of want to unfollow yourself a little bit.
Again, I'm not good at this. I am obsessed with my Twitter status to this day, but if you can, it can be a distraction sometimes to watch the minutia of every tweet that comes across.
We had Get Satisfaction, which didn't really do a good job of that, but I would highly recommend giving your customers the ability to help each other as much as possible. I always wanted a better forum than the one we had. We kind of would get away with that on Hacker News. We were kind of Hacker News darlings for a while, so like any sort of, not necessarily the technical code questions, but like, "Oh, I was thinking about doing this with Twilio, but then I ran into this blocking problem and I couldn't do it." So, you might see that in a thread and we're like right on top of that to make sure that we answer that question.
Stackable was a really, really, really high quality traffic driver. It's been a while since I watched the number so I don't know if its still driving quality traffic like that. But that person was in the moment of solving that problem, and if you could get on that question within a few minutes â€“ I set up every possible alert to notify me about a Stackable question about Twilio fast as possible. I think my phone would buzz like 6 times if something happened. I just did not want to miss it. Those questions live forever, t he best answers get voted to the top, and you become the definitive source of information on that.
We would answer questions that weren't necessarily generally applicable to Twilio. There's a question I still have on there that talks about email to SMS gateways. I'm like, why wouldn't I want to use this. I answered the question pros and cons and never mentioned Twilio until the last sentence. And that question would drive us tons of traffic and I didn't even have to work very hard to promote what we were doing. Which would have gotten the question voted down anyway, but it just shows how valuable that channel is.
So how do you encourage consistent voice? What is the process for that? One, every Twilio employee when they start has to write an app. You're guaranteed to have interacted with the documentation at one point. And our documentation, we viewed it as marketing material. We put as much effort into it as any other word that was on the website, or tried to, at least, put that much effort into it. It was basically like the Twilio Bible. If it wasn't in the docs well then that feature didn't exist. Two, however the docs describe it that's how you should be talking about it. We had basically a canonical source of all things technical in our documentation and if there was any gray areas, then we should probably make it clear in the docs to sort of solve those gray areas.
The other way we would do it, before our product launched we would make sure that we would get support in evangelism, in marketing and everyone that's not an engineer and even the engineers that didn't work on it, in a room â€“ brief it, have people hack on it, make apps.
We set aside, sometimes an afternoon, for people to work on new apps so that everyone had had a chance to go through basically the same process that our customers were going to be going through after something came out.
A really good way to hone consistent voice, too, is that we would have these VIP weekends, sort of like six months before a big product launch where w e'd fly out 30 of our best customers, and in that scenario if you mess up a little bit, it doesn't really matter as much. You get a chance to practice on people that are going to be doing it and seeing it in the real world before it actually happens out in the wild. So those were like golden opportunities to hone the instructions and look for where they got hung up on the docs, to make sure that we were all talking about the same. If you would hear somebody explaining it in a new way over and over again, it would make the hair on my neck stand up because I knew something was inconsistent somewhere. That our docs weren't clear or that we didn't brief people properly or we didn't give them complete information.
There is sort of a balance between "is this going to pay off?" and "is this not?"
I think between the judgment and people putting themselves in the right place at the right time, you will luck out if you hire people that know how to do that. I know that's a very nebulous thing: "Oh hire people with good luck and you'll be successful." But you get kind of a spidey sense for what's going to be valuable or not. Sometimes I would go to an event and there'd be only be like six people there.
Oh, here is a great example. I wish I put this in the slide. Alright. W e had been going to like every major city in the US over and over and over again. We were in New York. We had multiple evangelists in New York, we were in Austin constantly, w e were in Seattle a lot, San Francisco. We were in all these really high profile places over and over again. I was based in Denver. I said, "I can fly Frontier Airlines, flies all the regional fights here, they're super cheap. I can probably get in and out of somewhere really quickly and maybe get them to line up events for me there because they don't get a lot of events and we'll see what happens."
So, I planned this trip, I pinged some friends in Montana and they're like, "Yeah, great. We'll set up a meet up in Missoula and then the next night you can go to Bozeman. It'll be awesome." And so I booked my ticket to Montana where they promised me 30 people at each of these events. It was small. I got there, and I think there was probably right about 30, 35 people at each event, and gave the talks. Those people were so excited. They never got talks. Companies never came to Montana to talk to them. They brought me to customers, they would walk me into the office of these big companies and be like, "You should use Twilio and here's the guy that's going to help you."
I didn't have to do anything for that trip and those people became huge fans. They still talk about it to this day and I still talk with many of them. I reached maybe seventy-five people and two customers on that three day trip to Montana. I nearly died at the Continental Divide at midnight driving back between cities, but seventy-five people, maybe a hundred total, most of them are customers. A lot of them became really valuable customers. It just paid off immensely. With Twitter the world is small to them, they can be in Montana, but they can still talk to all their dev community friends. That was like the most surprised trip. It really paid off and I just had a hunch that going somewhere that we hadn't been going would payoff.
At the time the CEO gave out these little plush owls for notable achievements and I got an owl for that. So, that was my 1st owl for the Montana trip. The guy from the company he's like "John said he was going to go to Montana." And we were all like, "What is he doing?" He's off his rocker." And it ended up being one of the most valuable trips that I did while I was there.
One, we kind of solved that by hiring a lot. We went from two to eight just to do more of what we were doing because we did see that it would payoff over enough time. Eventually, they kind of capped out around eight, nine or ten, I think is where they're at now. Where they just couldn't be in more places no matter how many more people they added. There just wasn't enough high quality events to sustain it.
One way to do it is just brute force. Just hire a lot of people and be everywhere all the time. Two is making sure you really understand what a customer is trying to accomplish, what a developer's trying to accomplish.
The more context you have, the more you can understand where this is going, how important it is to the business they work for, how interested they are. Did they come to a hackathon and work on it and get really excited then, but then the next weeks they kind of fall off. They don't talk about it anymore. They decide not to pursue it. You know it's really easy to then spot that that wasn't a very valuable thing. But, in the context of the event, they still had a good time and you still have a good relationship and they may pay off later. I'm not really answering the question.
Good judgment, I mean you do it enough and you develop a really good sense for what is going to work and what isn't going to work and you just try to do more of it when you see something that's working.
The remainder of the talk is off the record.