June 14, 2016
Beta Testing for Early Developer Teams
UX designer and former Treasure Data product manager Luca Candela discusses defining your feature tests, cohorts and research experiments. H...
Thank you. I'm really glad that everybody's here tonight and excited about developer marketing, the best topic in the world. I've been the founding marketer at a couple of companies now. And I spent about 15 years engineering before I got into marketing and sales roles. I wanted to share a lot of stories and tidbits of things I think were really important to the success of both New Relic and GitHub.
I think those are pretty broadly applicable to a lot of different companies and are actually quite different as well. So we'll walk through a bunch of different examples of how those companies sold up from single developer deals all the way to large enterprise deals. As Dana just mentioned, I'm a principal of Reify, which is a software consulting company. We work with a lot of companies, helping with sales and marketing, so this is kind of applicable to that work as well.
So to set the stage, I want to just talk first about the difference between selling top down and bottom up.
In the SaaS space, we don't really talk about the top down stuff. This is sort of an old model that has thankfully gone away.
But what we mean specifically is top down selling is reaching out completely cold to an organization at the highest levels and trying to gain credibility from the start.
It's a really cold call, right? They have no idea who you are, what you do, and there's not a lot you can do in the marketing process to help those sales. In the SaaS space, we do the opposite, right? We go bottom up, we sell it to individual developers, we hope that they might recommend us to their coworkers, maybe their whole team or department starts using our products, and then we go up into large enterprise deals.
So let's walk through the differences of how those might work. Land and expand, as we all know, is sort of crucial to this bottom up selling notion. There are three ways to expand your account, to get more value out of the relationship you have with that customer. If you have usage based pricing, you can get more usage. You can onboard a customer to use more of your product and you get a bigger value out of that relationship.
If you have user based pricing, you can get more users. If they recommend your products to other members of the team you have a larger engagement with that client. And the last one, which I don't think we talk about enough, is more SKUs, more things to sell them. If you sell them a product that's $50 a month today, but you've got four or five other products that you could also sell them, that's another opportunity for expansion into that account.
So we gotta think about which of these models work for our businesses and how we can actually build that into the model of how we interact with customers to encourage this adoption to go up.
There's a huge number of differences between these individual sales, the sales at the mid-tier of management all the way up to executive sales.
At the persona level, they're different buyers. They have different concerns. They work for different people. Everything they think about is going to be different. So your messaging has to hit at these different layers. You can't use the same message for all three audiences. So I'm actually going to give you some examples of that in a minute. The value prop to these audiences are vastly different.
They're buying your product for totally different reasons. A lot of times products will have features for different personas. A report for example, may be the ideal feature for a management person, or an executive but not really a daily use for the end user. But those features are incredibly important as you move up the sale. The cycle and deal size are obviously very different. Small deals close quickly, big deals usually take a long time, but they're worth it, and we can do a lot in marketing to help shorten that cycle.
And I'll talk a lot about the different marketing assets that I think go into this sale of moving up the stack. So we think about developers as buyers. This is what I think most of us are very familiar with. If we are a developer ourselves, we're selling developer tools, we think, this is what I would like. This is the messaging I think is going to resonate with this audience so that's what we do. This is an easier sale for a lot of us.
There are ways though that you can encourage adoption and build this into how you market your product. So in the earliest days of GitHub, one of the pain points of open source software development was that SourceForge had a single taxonomy. If I had a project name and you had a project name that were the same, I couldn't create that same project. So this initial idea was let's create a username space in the URL that actually means all of my projects are underneath that umbrella.
Well it has a really interesting side effect. Our very first GitHub T-shirt featured a "GitHub.com/blank_space". And we would write in our usernames. We're claiming our space in the open source community by putting our name there. And it has a really interesting secondary effect because now, every time I talk about an open source project that I worked on, if you look at that project or if I put that at the footnote of a talk, my name is in the URL.
So we get that virality built on vanity.
It's hard to pull these features off. At New Relic, we had a feature where you could actually share a snippet of a graph. You'd go through and select a time series, you'd look at the particular graph you wanted to share and create a sharable URL. I think it's probably one too many steps, and so screenshotting a graph turned out to be the most popular way of sharing that stuff. So it's hard to do, but I would encourage you to think about how you can do that in your products.
This is something hopefully we've all heard a lot, "do things and tell people". This is sort of the essence of developer marketing. I would encourage you to only do them in this order. We don't want to tell people before we can give them a product to play with. Make communication part of every ship. It's incredibly important to show that your product is alive, that your company is doing something, you're writing about all the improvements you're making, no matter big or small.
This is interesting, these numbers I looked up recently, and I couldn't believe them. The first commit to GitHub was in October of 2007, and the public launch, the beta launch was in April of 2008. So first of all, it's an incredibly short window. Six months to go from first commit to something that a lot of people were using. It's a really short scale. In that short time frame, there were 40 blog posts on the GitHub blog.
Confusingly, the second blog post says welcome to the blog, so I don't know how that happened in reverse, but by the end of 2008, so effectively just a year in, there's 280 blog posts on the GitHub blog. So I have never seen a company start this intensely communicating what they're building. And you can actually see in the founders' heads exactly what this was like, it was in the morning, I grabbed a feature off the list, something I've been thinking about since last night.
I'm going to work on it, and when I ship it, I write a blog post, and I go do the next thing.
It was part of that cycle, and this is true for big and small things, so check this out. This is the blog post for pull requests. This is one of the most important blog posts and more important features to the platform and it's really short, this is the entire thing. Right now, in fairness there's a link here to a guide that would walk you through. But it really starts to position how important this content is and how easily you can do it if this is all that's required.
If we all had to just write a paragraph today about what important feature we got out the door this week, I think we could do it. We want to figure out ways to build content into our development schedule. I think there's two important ways that marketing teams can help make this better. The first is you want to encourage everyone on the team to give them the ability to just contribute an idea for a post.
Somebody might have a great idea, but they don't really want to flesh out an entire blog post. They don't really have the energy, they may not have the skill, they may not want to be the person who writes that thing, but they have a really good idea. They worked on the product, they worked on the feature, they're working on a cool tool that's behind the scenes. Something that's interesting to your audience.
So if you can enable somebody to give them a channel to share a half-written blog post or even a paragraph, take that over the line for them, and everybody wins. The second most important thing though is that
The person who had this idea for the post should get the byline.
This is important because you want to encourage everyone in the company to contribute as often as possible, and so you reward that behavior, and hopefully that generates more and more. So when we think about sharing content and just writing things down, there aretwo other important things here. First I think is it can be really daunting to look at a content map and say, we've gotta get all this content.
Here's a road map of everything we think people care about. Here's all the things that we're working in, all the tools we're using, all the cool stuff we could say, and then figure out how we could create that content. It's daunting and effectively no one does it.
A different model though is to document, don't create. Whatever you're doing, your actual daily life, your actual workflow, the products you're building, the tools you're using, just document along the way.
Get content out as quickly as you can, and move on to the next thing. Another way to think about this is that replies to your customers should include a URL. So in customer support, we have this. We have canned replies which are sorta half that feature. We'll say I'm sick of answering the same exact question all the time, I have this canned reply answer. Instead, think about if my customer has a question, I have failed in some way of communicating to them the answer.
So how can I use this as an opportunity to constantly create content so that now, in this first response and hopefully all the ones that follow I have a URL to respond with? And this is how you build up guides and documentation and blog posts.
Answer with a URL that is discoverable by all of your customers, not just that one.
So we talked a little bit about the messaging that's different depending on who your buyer is and where they are in the organization. We can walk through some examples with GitHub actually. So if we're talking to an individual developer. In the early days, we're looking at enabling a developer with social coding. What does this mean? We want to talk about empowering a developer to work with other people to make better code faster.
Those are things that resonate with all of us as individual developers. We can think about this and we would market it as the best tool for the job, that it's easy to use, we focus a lot on design, we have a great workflow, it's very feature-driven. An individual developer wants something that checks the box, I have this problem, here's my solution. I like what this product does.
As you move up in the organization, your messaging has to start changing because instead of focusing on maybe individual developer productivity, they're thinking about different things. They might look at GitHub and say this is a team communication platform, I'm enabling conversations across teams. I'm thinking about time to market a little more than I'm thinking about individual productivity.
I'm looking at efficiency in my team, and I like the fact that there's not a lot of training required. That wasn't so true in the early days. Git was obviously very new, which is why training was so important to the growth of the product as well. So this is a different set of messaging that we might need to satisfy this buyer. When we move up again, their purview is looking out years ahead of time and thinking about the holistic organization and how software plays a role.
They're looking for answers to how they can innovate in a large company at scale. They're looking at a platform that could help them bring open source principals in house to all of the developers at the company, which usually is referred to as insourcing. We want to actually source engineering talent from across the company to help on projects. I'm looking at this as a tool that's an industry best practice.
I'm looking to develop talent, to have my best developers working on code with all the new developers, and everybody share this same culture and grow the code base together. These are very different aspects of the product. It's the same product, maybe the same company, but we're talking about it in different ways. So now we have these customers, these individual developer customers, and we want to start thinking about how we can grow up in the organization and start selling to those teams.
And what's critical here is to
Work with those early adopter customers as champions in your product.
You will know them. A lot of times, you know their names because they write into support more than anybody else, not necessarily because there's a problem, but because they want to help make the product better. And so these are champions of your product. They want to spend their social capital internally in the organization to champion your product because it helps them so much.
You're putting value in their pocket by elevating what they can do in the organization, and they want to share that, and this is a really, really special relationship and crucial in this bottom up motion. So there's a lot of content we can create to enable that person. Imagine that somebody says, "God, this product is so great. I really think we should buy it". And somebody's like, "I don't know what that is. I've never heard of this product, I don't know what it does".
So what kind of content can we create to enable that conversation and make it easy for that person? New Relic has done a really good job at this. They have a ton of e-books. This is obviously very focused on the sector that they're in, very topical for them, but imagine that if somebody says, don't we have a product like that we bought 10 years ago? Isn't that good enough? This is something that person can use internally to send to their boss to convince them that this is a good product buy.
My partner at Reify, Michael Bernstein, worked at Code Climate. He created a physical printed book. It is an amazing piece of collateral that he developed because Code Climate was really popular in the Ruby community, and what they found out is as they went into other language communities, there wasn't similar awareness. And they had to figure out, how do we go and tell people about what we do that's not so self serving and just kind of product marketing?
It's not just about what Code Climate does, but it's about this concept of effective code review, and it happens to be effective code review of poll request because this is a great integration into the GitHub ecosystem. And so this book becomes a great piece of collateral to share with audiences that have no idea who you are, to talk about the broader benefit of the product, not just specifically the features that you have.
Early on at New Relic, we were looking for our perfect target buyer. And internally, we would talk about performance engineers, like as if that was a job title, that was a dream title. Like if this title existed, it would be the ultimate consumer of this product. And around that same time, this phrase gets popularized. We start talking about dev ops, and it turns out that a lot of the characteristics of that community resonates really well with what New Relic was doing.
And so this becomes kind of an industry trend that you want to latch onto and start educating people about because there is a lot similarities between those audiences. So again,
We create collateral that speaks to the broader movement and ties our product into that as well.
At GitHub, every year we'd release the state of the Octoverse. This is a really interesting piece. I loved putting it together. I worked on it for a couple years where we looked at all the things happening in the community and how we could talk about what was new and different that year. It became kind of the piece that showed off the power of the community, all the great things that were happening as well as what features the product had that enabled some of this stuff to happen.
These kinds of content pieces are really helpful in building awareness to whatever sector you're working in. So I would encourage you to think about what unique data you have that nobody else has? What do you see in your customers that other people don't see, and how can you package that up to present insights on some regular basis? Now this is a really well-produced big deal thing as a marketing asset that didn't always look like this.
The first version of the Octoverse that I wrote just as a blog post, was a really interesting first exercise to actually dig up data. It was actually the first time that we created these kinds of graphs. It's still online, you can go check that out, but it goes through all the different ways that the product was enabling these different behaviors and what the workflow of GitHub was.
Case studies are absolutely crucial in selling as you move up the stack in organizations.
We all rely on our friends to tell us what the new thing is. We're all encouraged, we want to see what our friends are buying. We're interested in their vetting of products too, and so this sales model of looking at people that we look up to, whether they're friends or folks in the industry, their perspective in that case study and understanding why it is they chose that product.
What use case they have, how it's helping them accomplish some goal, allows them to personalize that and think about how it can help them. These are early videos, this is I think 2011, but this is something that's crucial to continue to do as you go along. So take those champions, those early adopters that are really fanatical about the product, and just ask, ask if you can do a video interview, audio interview, even if it's just a pdf you put together.
These are really great assets to be able to send along when somebody feels a little unsure about the product, that you've got another asset, somebody else kind of vouching for you. Similar to the Octoverse, The 'State of the stack for Ruby' was something we developed. Thinking about what do we have access to that people don't really see broadly in the community?
In the early days, New Relic was Ruby only in the very beginning, and so we had a unique view on what Ruby versions were running in production, what Ruby gems were in use across production code, not just who downloaded them, but how are they being used? This is a fun thing, we're tapping into that vanity again. I want to know how popular my gem is in production. I want to know, actually the Ruby quarantine wants to know how popular different versions of Ruby are.
So we put this together and this collateral, again, communicates that we know what we're doing, we're a big player in this space, we're communicating something of value back to the community and again, these are great assets that get shared amongst the people as they bring those into their organization and share them with other people. This email, probably a lot of you who have gotten at some point over the years.
This is a Monday metrics email from New Relic. The goal here was the onboarding of the platform. In the beginning, if you didn't deploy the New Relic agent, we had to have some way of communicating to you that we're still here, we're waiting for you to deploy the agent, every week you'd get this email. And of course, the data was just dummy data because you hadn't deployed anything, so it was a good onboarding experience.
If you did deploy, this essentially became a report. And interestingly, there wasn't really a similar report in the product initially. And so we'd hired a designer pretty early on who wanted to tackle this email because it was an email that was generated by the system. It wasn't the best looking thing. We wanted to redesign this, and of course, this is early in the day, so there's no testing. It's just let's redesign and ship.
We got a lot of negative feedback actually about the email initially. And so we talked to some customers. We emailed them back and said, why is this so bad? It looks so great, right? And it turns out that the problem wasn't the design of the email. The problem was that the people that were receiving this email were regular daily users of New Relic, and they had installed this agent in a bunch of different applications because they're moving software to the cloud.
And it was a chaotic environment, and they want to show to somebody else that they've got it. It's under control, the applications are healthy, things are fine. And so they were taking content from this email and generating a report for their boss. So this is an insight that there's a new use case. There's a new user, it's not just about performance management of a single application.
There's a use case of someone in the management tier being very concerned about are we on top of all the apps we're deploying in the cloud, yes or no? And this email became a big feature for that particular audience, again that feature differentiation by tier. So if we're looking at the economic buyer, we've moved up the funnel and now we have arrived with somebody who has a lot of budget and at the same time, has a lot of hoops to jump through.
It's all fun and games until we get to the big deal conversations and we talk about how we can closer enterprise deals. What's helpful though is that we have built up this huge user base of people that love the product. They've recommended it maybe to other teams, and now you have a whole bunch of use cases from that company to talk about.
You're very well positioned to have this conversation. Unfortunately the sales process is still complicated. Right, so we can think about marketing not just as driving customers through this funnel and making that number bigger than it was before but
A major use case of marketing and sales funnel is to reduce friction. To take all the bumps in the road and smooth them out with some kind of collateral.
You need a demo, you need some kind of sales deck you can walk through, but what else do we need in this process? Let your customers tell you what you need. A lot of us have probably received this email. Here's this giant security checklist. Can you please fill this out?
This probably isn't even the same person who you've been talking to this whole time. You think you're selling, you're all excited about talking to somebody who's going to buy the product, and then you get this random email from somebody totally you've never heard of who gives you an Excel spreadsheet with 75 tabs and a bunch of questions that seem completely irrelevant to you.
They don't actually want you to answer all these questions. They just want to check the box that you have some kind of security story that passes muster. So we can think about an 80 20 rule here of spending 20% of the effort delivering content that's going to satisfy 80% of your buyers. Sometimes, you're going to have to fill out the checklist, but hopefully a lot of times, you can respond by sharing a security document.
But the security policies and information document walks through all of the various concerns that people have. It talks about HIPAA compliance, it talks about what they do in the data center. A lot of this content, this is like terms of service for startups, you just gargle all this stuff. So you're going to find other security documents. Look at where that information comes from most importantly because if you host on AWS, you're going to get a whole bunch of information from the AWS security page that you can just lift into your own.
By proxy, that's your security information or whatever provider you use. Just have something that's got your logo on it that says this is how we deal with security and hopefully this gets better and better over time if you get objections, but you have a piece of collateral now, so you don't have to spend all that time responding to those things. Again, all of that reducing that friction.
We're moving to the cloud, there's a million apps there. I don't know what to do. That's not even a question about New Relic specifically, but it's like I'm dealing with all this chaos, please help me out. We created another document, managing applications on the cloud. Anytime you get friction in the buying process, you think about how can we respond with a piece of marketing collateral that'll smooth that all out?
This seems expensive, right? Awesome, that means you're doing something right. I hope people are telling you you're too expensive. Even in the earliest days of New Relic, there was great feedback from both users and the executive team about pricing. We talked about pricing probably every month. And it was important that we were positioned in a way that we could continue to grow the way that we're intending to grow.
If you're not getting pushback on pricing, you're not charging enough.
We want to develop a framework for evaluating that ROI though, and it was difficult because cloud pricing was crazy in the early days, and no one knew where it was going to go, so we had to figure out a model that would scale with different types of deployment options. But have something. All these things can get better over time. But again, start to respond to these questions with documents.
Create a one pager that everybody needs. You have a sales meeting. The one person in the room already knows who you are, the three people you need to convince don't show up, what are you going to give him? Take me from zero to 60 in one document. Give me all the compelling reasons I should care about this product. These are really painful to keep up to date. This one has way too many words, sorry. I'm sure it's gotten better. But these kinds of collateral assets are really important and will make you feel so much better leaving a meeting when you've got something you can leave behind like that.
It's important to put together a 'Starter Pack' of all the things you need to close big deals. I think this is sort of a fun way to walk through all the difference assets.
1) Enterprise/POC: Include a little bit about what I always refer to as honeypot words. If you put enterprise or you talk about proof of concept somewhere on your website, the only people that would click on that have a lot of money. Developers don't click on those links, so don't worry about what you put in there.
What's important is you have something to say to this audience. If I don't see this stuff, I'm getting the feeling as a large organization that you don't know how to deal with me, that you don't know how to sell to me, that you may not be able to walk through a longer sales cycle with me. There's a level of maturity that you need to show in order to do the dance of a long enterprise deal, and these are some of these steps.
2) Corporate authentication: People want to log into your product, but if I'm going to add 500 or 1000 users to your service, and I'm going to do that one at a time, you're crazy. I'm not going to add your sad startup into my onboarding and offboarding process. So this is a key feature for that audience.
3) SLAs: This is going to be something that may even be baked into your contracts to figure out where you're comfortable.
4) Custom contracts: Having this stuff on your paper is always preferable, but it'd be nice to have a line in the sand that says if you're this tall, you can use your own paperwork. So come up with a big number for that.
5) Account management: I think this is a great trend of seeing customer success and account management becoming this hybrid role. I think it's really important that you're not just looking for people that aren't having problems but who are actively happy with what you're selling.
6) Product roadmap: Everyone wants to know what you're going to do next. You don't know what you're going to do next! So don't commit to dates. Don't tell them what feature you're building next, but a roadmap just sets vision and direction. What are you working on, what are you going to help me do in the future, how can I engage with you as a big customer? And how can we work together to make the product better for everybody?
7) Discounts: Everyone loves to ask for discounts. So think about that in your pricing.
8) On premise: So with New Relic, Lew, the founder of New Relic, had always said in the very beginning that we would never do on premise software, and I think there's a very good reason for that. Obviously at GitHub, there is an on premise product. There's a decision that everybody needs to make for their product and their audience, but the funny thing about on prem is it basically doesn't exist anymore because literally everyone hosts at Amazon anyway. On prem is just "deliver at AWS". So this is something really interesting because it really reduces the barrier to entry and there are a lot of companies that can help you deliver a SaaS product in that environment pretty easily.
I would be remiss not to mention the RC helicopters that maybe some people in the room received at some point in 2008 - 2009. If you don't know what I'm talking about, we had really aggressive campaigns targeting folks that had signed up for a New Relic account and didn't deploy the New Relic agent anywhere. And so we had to incentivize them. We would say "here's a really expensive RC helicopter. If you do a deployment into any software product at all and you send us performance data, we'll give you this helicopter".
Honestly, I was a little embarrassed at how ridiculously well this worked. But there's an important lesson here about aligned goals. So if a marketing team was tasked just with signups, we wouldn't care about deployments. But if you signed up for the product, the sales team couldn't do anything with that. They don't know what language you're working in, they don't know the breadth of the product you have, they don't know how it's deployed.
There's no information, and so if you think about a lead handed off to sales being somebody who's actually deployed the agent, now we have shared alignment. It's going to be a smaller number, but now we're incented to do what everybody here needs. And so you reduce that friction between those teams. And so I think this is really important to think about how you can incent all of your teams to be focused on the same goal. And I guess the RC helicopters was the way we could do that.
So I want to tie all of this together and just talk about a recent client that we worked with actually and give some examples of recommendations we made to them. So this is a Reify client Rank Science. They have a really interesting product that will drive organic traffic to your site doing automatic SEO changes behind the scene. So this is fascinating to me because it's such a targeted buyer.
Not everybody in the world wants this product, but if you're a head of marketing or a founder to a site that has 1000 or more webpages online, this is something that's going to be really interesting to you. It's a very, very targeted buyer, but we see a classic mistake, first of all, in the messaging. This is very what we do focused and not why you want to buy it focused.
So we worked with these guys for a couple days to figure out what the buying process was like, what their customers were like, and what recommendations we could make to steer this in the right direction. So the first thing is to update this message and talk about your target buyer. A little bit less on how you do it, and a little bit more on why it's interesting and talk about that organic traffic.
We want to create an anchor resource that will drive interest from a broad number of people with as little work as possible.
They happen to have an intro to SEO talk they've given at a bunch of difference places that always gets standing ovations. And we think this is a really good anchor asset that you can offer to people on the web and drive a lot of traffic in and hopefully, those folks will be qualified in to see what else you have to offer. They have a paid trial, not a signup. We're not looking at developers signing up for a product.
For them I think shifting the language into what a paid trial looks like is a much better response because a head of marketing isn't necessarily signing up for this product. There's a lot of work involved, and so this trial experience again positions this longer engagement in that you know how to sell to that buyer. Beginning targeted sales outreach. So there's a very, very, very specific kind of buyer for this, which reduces the cost of going out and looking for those folks online and doing some outbound sales.
And the last piece again is as you bring these customers in think about how we can create case studies and assets and all the A, B tests that you've seen in a while that were very successful for your clients. Write about that stuff because it talks about the value that your product provides in a very genericized way. You don't have to target what customer that was, but it shows how your product functions.
And it's again something that they see that nobody else does. Nobody's seeing the actual A, B tests at scale and how that impacts on the Google algorithms over time. So this is a unique thing that they can talk about. So I hope this helps frames some of these examples of like small marketing things that we can do to effectively increase sales through that funnel. So thank you very much.
So this is interesting use case because this is where you see those roles being compressed. So the person who's actually going to deploy this is probably going to be somebody who's maybe a technical marketer or even somebody who's on the web team. So the way that this works is through a proxy and so there's some technical changes they have to make. But they don't do a lot of day to day usage in this product.
And so we wanted to see "How do you qualify these leads and build up that hit list, how do we go after this target client, and how do we get collateral in front of them that will make sense?".
Obviously, when you think about distribution, it's an entirely difference set of questions. Like what channels are going to be effective? What can we automate? There are a ton of places where we spend time with our eyeballs. We're looking at Product Talk, we're looking at Hacker News, we're looking at Stack Overflow for questions. There's a really wide surface. Personally, I like to separate those concerns.
I would think about distribution and those kinds of thoughts to be focused on what the marketing team can do with the content that you generate, but to not overload the generation of that content. So if you're a writer, you shouldn't think about that. If you're somebody on the engineering team and you just want to get these blog posts out, just get the blog posts out. You can resurface content later. You can discover what channels are working for you retroactively and go promote things through those channels again.
But I like to separate those concerns because I think that's one of the reasons actually why content is so hard to create. Because
We're taking on the burden of getting an idea, packaging it up for this audience and thinking about distribution and social sharing.
There's a whole bunch of activities involved. In both of these examples actually, I don't think there was any distribution at all for New Relic or GitHub blog posts. It was a blog post and a Twitter account was literally the end of the channels.
But you can go back and resurface that stuff. YouTube was also big for both companies actually. GitHub spent a lot of time building up educational materials because the tool set wasn't widely known. New Relic similarly spent a lot of time educating people about application performance management because it wasn't a suite of tools that developers of Ruby and Python and PHP were used to. And so there's a lot of education materials there.
So I think that separating those two concerns out is probably the way to manage that.
Yeah, so you really need to figure out, and we didn't go into pricing at all in here. But we've spent actually a lot of time working with several of our clients working on pricing. And discovering the willingness to pay for your clients is a whole process. You want to figure out not just what price point they're willing to pay but also what the price model is.
So a lot of times you might have a product that may be charged per user. And that scales linearly with every hire that I make. So there's a certain kind of model, and there's a certain product that might work well with that. Somebody says I make a new hire, I pay you guys a little bit more, I get that, fine. Usage based pricing makes more sense for a product that not a hundred people need to log into. If it's a very passive kind of software application, you're not going to be, user pricing's not going to work.
So part of it is figuring out what the model is and then if you're price points aren't being pushed back against, you're leaving something on the table. So you shouldn't get pushback in every conversation because obviously, you're priced a little bit too rich, but if probably 10, 15% of your customers aren't telling you it's too expensive or more, then obviously there's a gap between what they're willing to pay and what you're charging.
And you want to discover that through a lot of customer conversations.
Yes, so thinking about kind of quality and quantity with content is a really good question because if you look at those blog posts, I would argue, there's a lot of them. There's 280 in a year, which is insane. That pull request blog post is like five sentences. So what I like about it though is I love every one of those sentences.
There's a sense kind of love of brevity in that case in that we all have enough attention to look at a five sentence blog post of an interesting feature. And the secondary effect is even if you're not reading those blog posts, you're seeing a bunch of them, which is communicating another thing to you entirely, which is that these guys ship. Something is happening over here. So there's a momentum there.
So I think if you're sensitive to kind of the time investment, which I think we all probably are, you want to focus on documenting. As I mentioned, kind of how do you document your process and make that your pipeline for creating content. So it's not sitting down at a blank piece of paper and coming up with an e-book, but actually writing about what you did yesterday, which is a much easier thing for probably all of us to do.
And after you have enough content, you can start to run analytics and look back in time and say, wow, people really like this kind of thing. And these kinds of things that we love to talk about, no one seems to care about, or the competitive analysis and content, lots of people talk about this other thing. What can we talk about that's unique to us that no one else has?
I think you can use a large quantity of content and the analytics thereof to get a plan for where you might want to invest more significant time in creating those other pieces. I would also honestly just put kind of a cutoff on that stuff. I think these are the kinds of projects that could just be forever, like you could have an e-book in the works for seven years. So like any good project, kind of scope it and say this is what we're going to do, this is the time we're going to invest in it, and we're going to ship it when it's done.
I think that's really appropriate because you can definitely polish those things forever.
You almost want to think about places you put content, and there's obviously very low threshold stuff, which could be as simple as an FAQ on a long list of questions. Some of it might be your product documentation. Some things might be up a level again where there's a whole guide. Like, here's a whole kind of use case, and we're going to walk all the way through it.
And so, actually another answer to the e-book question really is how can you leverage the content you have and repackage that in a condensed way that answers a particular use case? Because at some point these stories tell themselves. You've generated a bunch of FAQs. You've generated a bunch of guides. These questions keep getting answered. You know what to do, right?
People are asking you the same information so just think about how you can package up that content in the same way that you're packaging up features into use cases. So the question is mostly about developer tools being primarily used on the desktop, but obviously there's sort of a growing trend that we all have phones in our pockets. And what's interesting is that's where most of our information is consumed.
So what's a really good insight there though is that sometimes subconsciously we think product equals desktop. Maybe the blog equals mobile as far as view ports. But
It's important that your product documentation works on mobile
And it's important that the interstitials on your signup flows. This is actually an important thing to think about with signups because a lot of companies are using OAuth with other platforms because it's so easy.
One click and I'm signed up. Well one click on a desktop and you're signed up because you're already logged in. If you click a link on Twitter into Twitter's own browser, you're not signed into that platform, and so it's an awful experience for people to go though that OAuth dance. So one thing that I've seen used in that example actually is if you're on mobile and if you have only an OAuth integration to just offer, here's an email address that you can send an email to me, and then on desktop, I'll go do the signup flow.
Just follow up with a response, but yeah I think you have to assume that all of your content is consumed on mobile and think about what all the calls to action are and how relevant that's going to be on a phone. Thank you.