August 16, 2016
SF Metrics Meetup: OpenTracing, and Druid
Heavybit member company Librato recently hosted the SF Metrics meetup in our San Francisco Clubhouse, where speakers Ben Sigelman from Light...
Thank you so much, everyone, for being here. It's really different to be on the other side of the stage. Just before I get started, I wanted to give a quick, brief intro on who I am.
I actually started off as a front-end developer at Google, and a couple years ago switched over to the "dark side," and now work on marketing. I worked on marketing for the Android platform both to make more consumers use it as well as developers.
I also worked at Dropbox on everything from consumer launches to business launches, and also all of their developer platform work, and have been at Stripe for the past couple of years.
At Stripe I do everything from our brand positioning, our competitive marketing, and thinking about international launches and our sales collateral. But most importantly, I work on all of our product launches.
Over the past couple of years I've been a part of every single product launch that Stripe has brought to market. This is probably a flow that you guys are all familiar with, right? When you're building a feature or a product, you create the code, you iterate upon it. You might do a beta or a dark launch. Then, once it's in shape, you are ready to ship that product.
So what are you expecting from that launch, from that product or feature being shipped? Well, ideally, there's a lot of excitement. You're building a product or feature that is really useful for developers. You're also hoping that people adopt it and start using it and getting out there. The other is, ideally, people start paying for it, right?
But where does marketing fit into that picture? Sometimes it comes mostly after the ship. Sometimes it might be non-existent, or sometimes you might just say, "I'll just throw it in the next newsletter we're sending to our users."
So what actually happens when you launch? There's a couple of different scenarios that might happen. You get it to launch and no one really notices. You know that this feature is super useful. Your beta users have loved it, but no one seems to care.
Or you send out a newsletter to your core audience,these are people you know will benefit from using the product or feature, but no one actually opens the email. And now what? No one's heard about it. Or everyone gets super excited.
There's a lot of excitement around it, a lot of commentary and discussion, but then no one actually uses the product. All of this adds up to making me really, really sad because all of your companies out here exist to make developers' lives easier.
Whether it's in infrastructure, or how people code, or how they test their code, or even in how they accept payments, like Stripe does. Your marketing should do the same.
It should really help your developers understand the product or feature and the value that it adds to their workflow in order for it to be effective. I know that every launch is different and every launch takes a lot of unique preparation, but I'm going to talk a little bit about the three different things that Stripe considers, or I consider, at Stripe, when we're launching a product or feature.
The first is this concept I think of is the business value of delight. The other is around personalization. Last is creating connections. All three of these things can help you have the launch, have the impact that you hope for it to do.
Let's get started. The first is the business value of delight. What I mean by that is that every launch is a way for you to connect with your wider audience and your user base, whether it's potential users or existing users. Each of them you might reach in very, very different ways.
Delight can make a niche launch, some feature or product that's only meant for a small subset of users, actually reach and resonate with a wider audience.
So what do I mean by that? Well, usually you segment your audience into two different sections. There's the core audience, the people you've identified, either part of your existing user base who you know this feature is going to help immediately. Then there's this potential space where, sure, if you reach them, that's great, but you don't really expect them to be a part of your launch.
But I actually add this third layer in, which is you want to not target your launch so much that you don't let people surprise you. If you say a particular feature is only for a QA developer, then you're maybe excluding people like a PM or a designer who might actually think about this and implement it into their workflow.
How do we actually put that into effect at Stripe? Here's a launch, which we did a few months ago, around Stripe Connect. Stripe Connect is a feature built mostly for marketplaces. These are people like Lyft or Postmates, where they're accepting payments, but they also need to get other people paid.
If you take a Lyft ride, Lyft accepts that payment from the end-user customer, but also needs to pay out the driver. The audience for this product is actually pretty niche. There's maybe a few hundred, or the low thousands, of marketplaces in the U.S. that might actually use this product.
But on the other hand when we launched this product there was a huge response at launch. I want to dig into a little bit of why that happened. This might seem a little silly, but it actually just comes down to good design throughout.
This idea of surprising or delighting your users just means that you're thinking about ways to engage a wider audience.
When we actually launched the landing page for this product, it had this cool animation built in. It's subconsciously telling you that the marketplace is this gravitational field pulling payments in the right direction.
Philip, one of our amazing designers at Stripe, partnered up with me on this landing page. We spent a lot of time agonizing over the details here. We made some tradeoffs, as you can see, but the idea was people who might never have considered Stripe, especially Stripe Connect, were starting to talk about the company. This asteroid field is obviously not the only thing that you launch.
There's other parts of the launch as well, but if you spend a little bit of time on the details, what it actually does is it shows that you consider the marketing and the entirety of the product with as much respect as you do the product itself.
Because you're putting in a lot of work to make developers' lives easier, that marketing is doing the same thing and it's delighting people.
I'm already assuming here that your product has value. Developers want to use your product. You want them to actually use that feature. This just reaffirms your company's commitment to solid product through solvent design. Delight isn't just about a launch. It's about building a brand.
This is a quote comes to the real heart of the issue, is that people who might not ever consider your product come back and engage with it. One thing I like to say is that a launch does not exist as an island. It kind of builds upon each other.
They might not be impacted by the particular feature that you're launching, but come in the future, you want them to be engaging with your brand, looking at stuff that you're doing, looking at the stuff you're launching and figure out that might be useful.
It's kind of like marketing for a university, because if Harvard runs a campaign today, they're actually trying to reach middle-schoolers who might apply to college years from now. What you're trying to do with your product is that a developer might not need your product today, or even in the next couple of months, but they are going to need it, hopefully, at some point in their career. Whether at their current company or the next one they go off to create.
The next thing I consider with every launch at Stripe is the idea of personalization. Personalization gets to the heart of how you connect with your existing users with new features and products.
There's two things that I think about. One is how to target the right audience, and the second is to think about and anticipate what that user's going to do next in order to actually use that product.
So, targeting. As I said, there's many different parts to the launch, so this isn't he only thing that you're doing. Your blog posts or your landing page might cast a really wide net, but you can use other channels such as email to really hone in on that particular audience.
One question you can ask yourself is, "Who are the set of people for whom this feature is going to add value today or right now?" and get those people first.
Your launch is not just that one set of targeting that you do. You could build on top of it. You can see how it resonates with a small audience, iterate and build upon it, and then expand it, as well.
So one example at Stripe was our launch of multi-currency. This was the ability for developers to accept payments in over 100 currencies. All they have to do is pass a single line of code in, and suddenly they are accepting pesos, or Euros, or Australian dollars. This is particularly handy for people who are doing a lot of foreign volume.
Obviously, anyone can benefit from this launch. We could have emailed out our entire audience, but we actually ended up choosing a very small subset. We chose the people who we see saw, in data, were doing more than 10% of their volume outside of their home country.
So what does that practically mean? It means that maybe you're an Australian business, but you're primarily transacting with Chinese customers. Or if you're in the European Union, maybe you're seeing all of your payments from the Unites States.
What I'd like to consider sometimes is that every launch has a different success metric, and you have to define that before you go in.
The industry benchmark for emailing your existing users is if it's a stellar performance, about 20% of your users open the email. About one to 2% might clickthrough or respond to that email.
We actually saw a 68% open rate and a 10% clickthrough rate on that email. Let me put that into context for you guys.
Targeting can be a little bit counterintuitive because you're emailing less people, but actually more people are responding the way you want them to.
Let's say we'd actually emailed 10,000 people, and about 20% of them opened it, and about 1% or 2% clicked through. We're actually only getting 20 users to engage with our product or feature.
On the other hand, for this multi-currency launch, we emailed roughly about 2,500 people. Sixty-eight percent opened it and 10% responded, so we actually saw more people engaging with the product than we expected. That's just something to consider is, actually, sometimes when you make the pool of people you engage with smaller, you can actually get a bigger response from them.
The second aspect to personalization that I consider is anticipate. To anticipate means to go one step further than you usually would.
So Instead of just announcing a feature, you should explain its value and tell them what they need to know to make a decision.
That can take multiple different forms, and it actually does not only have to apply to product launches. I was talking to someone before, and they were asking about how to think about pricing and how to approach a pricing update.
Actually, it was a great segue into what I already had in here, which is around our pricing launch in Australia. When Stripe launched in Australia a couple of years ago, we brought the same set of features that we had everywhere else to Australia. That included our simple pricing.
Our pricing is a flat transaction rate that we apply to all transactions. But over the course of the beta we actually realized that people were, in Australia specifically, businesses were either doing primarily domestic volume or primarily international volume, and it was doing us and them a disservice to blend our pricing rate across the two.
Domestic transactions were more expensive than they needed to be, and international transactions were actually less expensive than it was costing us. So we made a business decision to actually split the pricing in Australia.
As you can imagine, if you're a developer receiving an update that the pricing has changed, what's the first thing that goes through your mind? It's are you better off or am I worse off, right? What impact does this pricing change actually have on me? So when we updated people about the new pricing, we actually built into the email, right in line, what the price would have been under the old plan and what it would have been under the new plan.
It might seem really daunting to do this level of personalization, but all it takes is a simple data pool. We use Campaign Monitor at Stripe, and it makes it very easy to get these personal values and variables into your emails. It also has, very importantly, a fall-back mechanism, so you never end up with something like this.
You can use tools to make sure that you're not doing something crazy, and you're actually helping your users immediately put into context what a change or a feature or an update means to them.
So what impact did this have on Stripe? We saw an 80% open rate, which, as I mentioned before, every launch will have different success metrics. I don't actually see this as particularly good because if your pricing is changing, you're going to open that email.
I don't read too much into it. We actually only received 22 replies from the email, and that seems fairly low, but when you actually have a pricing update, you want people to respond less. You want people to not be confused. At a previous company where I was we did a pricing change and the support load went up 300% that week. People were confused and angry and wanted to cancel their plan.
That's not what we saw at Stripe. We saw pretty peaceful responses. This might be conflating a little bit, that this particular pricing change affected Australia, so I'll report back when we change the price in France and Germany. We even got two job applications out of the email campaign, so all in all it was pretty peaceful as far as pricing changes went.
The final thing that I want you to consider with every launch that you do is a way to create connections with your product and developers. This is all something that you guys, I think, think about.
You're building products for developers to really make lives easier for them. What I mean by creating connections, I feel like I keep coming back to the idea of marketing for colleges, but bear with me.
I think about this as a professor in a college expecting the students only being taking their class, right? That professor thinks you're only taking Physics 103, and you're reading all of the materials, doing all the code labs, reading extra materials and coming to class completely prepared.
Well, your developers aren't following along on your blog, especially since Google Reader is dead. They are being hit over the head with tens and hundreds of launches every week. If you're on Product Hunt, you know that people are being inundated with launches.
All I'm trying to say is get people thinking about adoption and integration and put into context exactly what this does for them.
You all know that developers are resilient creatures. If they think a feature is good for them, if they think a feature is going to be helpful, they're going to go through crazy amounts of documentation.
They're going to do all the code labs and try out the examples and integrate it themselves, but they shouldn't have to. Your lives are about making developers' processes easier, so your marketing should be doing the same thing.
We do A/B testing, we do optimizations, we add additional payment methods. So when we launched Bitcoin, it automatically was something that was available to our Stripe checkout users.
When we first created the landing page for this launch and we sent it out to a few of our trusted testers, it just wasn't having the impact that we wanted it to. People weren't really seeing the value of Stripe Checkout beyond the beautiful design. We weren't getting the understanding that we wanted out of the launch.
So we actually went back to the drawing board to get this done. We took some time and we built out a beautiful animation that showed exactly what Stripe Checkout does. One of the things that we did pretty close to the launch time was we actually embedded a live version of Checkout right into the marketing page.
This particular button says "Donate to Watsi." Watsi, as you guys know, is a YC company. They help bring healthcare to a bunch of different countries around the world. What happened on the site is you can just click right there, see what Stripe Checkout is, and play with it and see exactly what happened.
This launch got a lot of coverage. It got a ton of traffic. It had some great comments across all of our channels. The product itself got a lot of adoption, but what really surprised me and what really pleased me was that we actually ended up donating almost $1,400 to Watsi just from that example on the page.
We also extended this example, piece of it, to our documentation. So when you came to our documentation, instead of hitting our landing page, you still got the benefit of seeing that example right there. You could click the button and see exactly what the product did.
Creating connections can take a lot of different forms. In traditional marketing, it might take the form of a case study, but for you guys, it could be just as simple as putting in a code example or showing exactly what that feature or product does for your users.
You usually have this consideration period, which might last anywhere from a couple of hours to a couple of days to a couple of months. All adding examples does is it shortens that time span from which a person first hears about a product to when they actually start using it and adopting it.
Here's a couple of things of how you actually put it into action. As the only marketing person at Stripe, I've had to be a little scrappy on making sure that these principles are applied every time we launch. I never want to seem like a blocker or ever be a blocker to launches.
One of the things I consider is first, you get into a room and just ask the person why we're launching the product. If an engineer is working on a piece of code, you want to make sure you're aligned on what value you think that this product or feature is having on the user.
Sometimes at Stripe we actually write a spec blog post. We have no intention of shipping this blog post, but we take about 15 to 20 minutes to do the process of writing it because it aligns with engineering and marketing on what is the value that we want to present to the user at the end when we're actually launching it.
The other thing that it does, it helps size your launches, because it's very hard to communicate to an engineer that the thing that they've been working on for the past year is not going to get the whole PR, landing page, crazy example personalization bit. But if you're working together on what value does this actually add, you can help a little bit align on what size of the launch are you creating for this.
Sizing your launches also helps maintain consistency because you don't have to compromise on quality in order to get something out the door quicker. You just change what goes into the launch and what you're expecting from the launch.
I would highly recommend that you beta test your marketing. That means just send it out to one or two trusted users. The size and scope of your beta testing follows how you do the same thing on code.
For your code, it might just need one or two people trying it, or it might need hundreds of people trying it out, or you might need to test it with an entire country before you launch it widely to your audience.
In the same way with marketing, you can send it out to one or two of your friends or you can send it to people who might potentially be users of the product or you could try it with an entire country, like we did with Australia, before you send it out to everyone else.
The last piece that I would really urge you to do is to build marketing into your launch process. All that means is, when you're looking at a process, something like this, instead of tacking on marketing to the end, just add it into the beginning of the process.
You're iterating upon it, you're beta testing it. It should be a part of your launch. What I like to say is that you're not feature complete until you're both code complete and coms complete.
Just keep that in mind as you do your next launch. Make the assumption that when you're shipping great product, you're also shipping decent marketing because it's going to help your developers, at the end of the day, to understand what your product does and how that feature can help benefit them.
It'll help your launch have the impact that you intend it to do. That's all I have to say. I'm happy to take any of your questions.
We actually, as you can imagine, use a bunch of different things. We pull our data manually from our Redshift databases, and then we plug it into Campaign Monitor. Then we look at responses there.
For transactional emails, we actually use Mailgun, so it's a bit of a combination. We are looking to put that all in one platform.
Two things we did. One is personalize. We actually found the percentage of people outside of their home country that were coming from a particular country and put that in the email.
The second, what we did was we tailored it to talk about adding value immediately. It's just, you add this one line of code, and we put that piece of code in the email so you could just go put it into your code now and get the benefit the same day.
My rule of thumb is that a personalized email should be the perfect email for about 90% of the people you're sending it to. You let that other 10% kind of surprise you to respond to that email.
Yeah, we definitely have some Hacker News experts in our office, which means that they know just the right amount of people to upvote before it hits the voting ring. Maybe I'll do a blog post on how to get to the top of Hacker. No, I'm just kidding. We don't expect much from our employees themselves other than figuring out who the audience is.
The other part that we do sometimes, and we're starting to do more of, is we do follow-up events. We've done office hours with the engineer who actually built the product so people can ask questions and can come in for different types of help. I think we're starting to understand that people learn very differently. There are people who will just copy a piece of code, and there's other people who need more hands-on help.
Yeah, we have a QA server. We whitelist a couple of IPs so we can send it out. We actually have a crazy non-IP whitelisted staging server that we just removed from Google. It works until it doesn't work.
One thing that I would hate for this to do is you come to your home page and now you have something like, " Are you an accountant? Click here. Are you a developer? Click here." The thing that you have to kind of go through is some user research. We're in the process of doing that at Stripe.
What we've discovered is that there's commonalities of someone who is part of the entire decision-making process, whether that's a technical PM, or VP of engineering, or even the developer themselves. What we're trying to do is kind of partner with them on helping them understand the lay of the land.
If you imagine you're going out to buy a laptop, you kind of know the specs that you need to consider. You know hard drive size, screen size, battery life, and you're kind of ready to make a decision. But oftentimes with developer-facing products, it's hard to even know what to compare on. Most of the time, you're creating functionality that doesn't even exist yet, or exist in the market right now, to compare against.
Taking a bit of time to educate people is just more of how to think about your product. I would say it's more layers of content. The first is more of the, "Why?" Then it's the, "What is the feature?" and then it's the "How do you implement it?"
That why does not need to be a falafel-y, like, built for modern business kind of stuff. It's just more take a sentence or two to explain the business value before you dive into the code.
I think within that same post you can have a couple of layers of content. You don't want to alienate your developers, so just take a sentence of two to set the context before you go into the details of the feature.
No, there's just one design team. When I first started at Stripe, it was just three people on design doing all of product design and helping me with marketing. Now we're a team of six, but they definitely are pooled resources across the company.
Cool. Well, thank you, everyone, for coming out. Really appreciate it.