May 16, 2019
Ep. #30, Improving Security Culture with Justin Somaini
In episode 30 of The Secure Developer, Guy speaks with Justin Somaini, a security industry leader and Founder of Somaini LLC. They discuss h...
I'm Tugce Erten, I work at PagerDuty. I lead the pricing over there. Just a show of hands, how many of you know what PagerDuty does? Everyone.
OK. We'll cut this short. You guys know that we're a platform for real time operations. We recently surpassed our $100 million in ARR, and then we raised $90 million in round D in September. I wanted to talk about how those are two indicators that we know we're in a good product market fit. The indicators of that are coming from the fact that a lot of people use us.
They could be small, they could be big, they could be Fortune 100 or they could be a five person team. People use us across industries because all of the industries are digitizing. If you're digitizing you need PagerDuty. I wanted to talk a little bit, give you a little bit of background on how we do pricing at PagerDuty.
It might not be relevant to all of you, you might be a lot smaller organizations. We are over 400 employees currently. This is going to be a little bit more complex for those of you who are in smaller orgs, but pricing is not a lonely job at PagerDuty. We make sure that we touch all of the parts of the organization and we talk to growth, we talk to sales people, we talk to product all the time. I also talk to finance, purchasing, billing, etc. It's a full effort across the organization, even though officially I'm the only pricing person.
I report into VP of product. She's over here. This is not generally unusual in technology companies, because in technology the product leads the sale. If we were talking about a consumer packaged goods, you have marketing leading the sales and pricing would sit in marketing. But sitting in product is not just because it helps us with the sales, it's also because I'm sitting with a group of product managers and they're the ones that really are solving the pain of the customers.
They're the ones that are defining the value of the product. Also, we strive to do value based pricing. If you want to do value based pricing you have to understand what the value is. I talk to them, collaborate with them very closely. I actually use them for my research as well. Besides that, I sit with growth. I was a recent hire, and the product manager at growth came and grabbed me, and she said "You're going to sit with us because we're going to write experimentation." I was like, "Great, because you guys have sunlight."
I was like "This is great, because vitamin D." We have been running a lot of experimentations on our pricing page, we've been thinking about how to make ourselves flow better. We tried to give tender loving care for our smaller customers that don't get to talk to a sales person. It's been very effective in that way. I also have a very symbiotic relationship with the sales team. They give me feedback.
I go to them and I'm like, "Look at this proposal that I have. What do you think?" They are good at knowing how the customer is going to react, because they talk to them. They have conversations that generally revolve around pricing, and talking about the value, but there is a lot of negotiations that happen as well. Besides that I help position our pricing, enable them on our new structures or strategies that are coming out. It's a very deep organization, in terms of how it cares very deeply about pricing.
That makes my job a lot easier. If you don't have that kind of a deep organization or don't have all of those functions, it is very easy to go and copy your competitor. That is something that no one should be doing. It's lazy pricing. I think that is the biggest mistake of anyone that is trying to focus on their product by just saying, "We have a great product. It's going to sell itself. I'm just going to copy what the competitors are doing and then I'm going to make it a little cheaper so it sells more."
That is definitely the wrong attitude when you're thinking about your product. You need to be thinking about how you're going to sell it, how you're going to deliver value and how you're going to get your customers to pay so you can continue investing in that product. There are three pillars of pricing and these all have to be thought of when you're thinking things through. These all have to be done in parallel. When you're researching them you can't think about pricing metric and not think about pricing levels.
What are those three pillars? It's pricing metric, how your customers are seeing the value of your product. Then there's pricing level, which is how they're going to be willing to pay for your product. Then there's pricing structure, if you're going to offer good, better, best or you're going to offer a la carte. All of these are asking a question of, "How are you going to measure? How are you going to balance adoption and monetization, and how are you going to segment your customers?" Again, I'm saying that there is three things you need to find, and don't copy anyone, and it's difficult. It's a hard problem.
That's why I have a job, and there are ways at which you can address it. These are the four things that we generally use at PagerDuty and every other company that I worked at. I also recently read an article that listed off all four as the top ways to go about your pricing, to find out what you could be doing. The name of the article was Good, Better, Best Approach to Pricing.
It's in Harvard Business Review. You should definitely read that article, it's informative. And they were saying that the research recommends the same top four methods as well. These are the methods that you could use if you don't want to be lazy about things. The thing is, because you spend a lot of effort trying to figure out how to go about these and they do take some resources, they do require some investment from your side. I wouldn't count on anyone else doing it right, because they might be copying someone else and everyone might be just blindly leading each other. Because we don't want to do that I want to go a little bit deeper into each of these techniques that we use.
The top one that they recommended was expert judgment, and I highly value this. What we mean by expert judgment is talking to your executives, talking to your sales people, talking to your product managers. The reason why these are key individuals when you're thinking about pricing is they talk to customers frequently, they interact with them, they have the pulse on the value of the product or how it's being perceived. In our organization we are able to do that. You might be at a smaller organization and you might only be two people currently, so who are you going to talk to? I would recommend talking to investors, if you have any board members, anyone that is respected in the industry.
Make sure that you get someone with some expertise, try to get someone to go out to lunch with you and then you can pick their brains. The thing is there is no right number of people that you should talk to, the more you can do it the better it is. I generally try to get at least 5 to 10 sales people's opinions, but you might not have that luxury. I would recommend trying to get as many as possible. One way that this is helpful is if you do not have much data, or if you do not have the opportunity to collect some data. It could be that you went to customers and asked them already about something else so you can't go back. Or if you have done all that research and this is their final method of validation. I do think that while this is predictive it's less scientific than some other methods that you might want to consider.
This is a no brainer. Everyone should be doing general market research. What I mean by that is asking your customers, "How much are you OK with in terms of how much you are going to be able to pay every month, or every year? How would you like to be charged?" You want to get all of this information out of them, as well as how much they're currently paying. "What did tyou like about it, what did you not like about it?" The ways that you can go about that is several. You could have a customer interview where you straight up ask the questions, you could have surveys. We'll talk a little bit more about those. You could have a focus group, and you should be doing a competitive benchmarking analysis anyway. Those are generally things that we're trying to get at with market research.
Customer interviews, it might seem like this is pricing, and "I don't want to talk to them about pricing. It might create friction in our relationship." You might be right. Some customers do feel like this is not something that they want to share with a product owner or the company that they're purchasing from, but there are a lot of customers out there that are willing to give you a ballpark. Which seems fair, especially if you have good relationships with them. You're talking about the product, the value that you are bringing to the table, how you are going to alleviate their pain. It is okay to say, "Look. We want to build this and we think this is how much we would charge. What do you think? Why, why not? I generally don't ask these to customers, because I don't have that kind of relationship with them, but our product managers do.
I go to our product managers and I do ask them, "Would you be willing to do this?" They found that there were a lot of customers that were OK with sharing that information. The other thing is, if you can get a focus group together then you can get a lot more design information out of them as well as some pricing information. However, focus groups are harder to get together, you have to coordinate. You have to have customers that are willing to come together for a period of time. That's a challenge. You have to at least get 5 to 10, and also you can't ask them too deeply about pricing in those settings. It feels a little bit more uncomfortable when you're surrounded by other customers. Then there's the survey. This is the method that I personally use the most.
This is when you can ask, "What do you think is the appropriate pricing metric? How much do you think is a fair price?" The method that I generally use is Van Westendorp. This is a series of three questions. You're asking, "How much is too much? How much is acceptable, and then how much is too cheap that you would doubt the value of the product?" I'm sure some of you have either received such a survey or conducted one. There are some answers that are going to be in there that's like 0, 0, 0. Some customers do find that annoying. Again, it's pricing. Nobody is saying that it's an easy thing to do but there are customers out there who are willing to say how much they are willing to pay, how much is OK and how much is acceptable.
If you are scientifically driven like, "We have to be statistically significant," then you have to survey a lot more people because you will have to clean the data. For someone that has sent out a lot of surveys and knowing that your response rate is never going to be above 3%. When you get to 30 it's like, "Great." You send 1,000 people an e-mail and then 30 people give you clean answers, that's fantastic. In that one you also want to make sure that you are capturing people from all segments. Then the other is competitive benchmarking. I've been saying, "Please do not copy your competitors' pricing." But you're not going to be working in a vacuum. You're not going to be in the market in a vacuum. In fact, you want to get out there and see what they're doing and then figure out if they're doing it right, if they're doing it wrong, can you use it to your advantage? You have to ask your customers what they like and what they don't like about the current structure.
The third method that the article mentions and we do at PagerDuty, or that we are trying to do at PagerDuty is conjoint analysis. How many of you know what conjoint analysis is? Great. I see that many of you are not familiar. It's a great tool for pricing, it's one of the most highly regarded quantitative tools. It's great for your roadmap prioritization. You are asking your customers a series of questions, you're showing them a series of screens saying, "Look. This is your options. Which one would you pick?" What you're doing here is you're showing them different packages, different levels of features, and across that you're showing them different prices. When you set this up you are getting a series of answers that lead you to figure out what are people's willingness to pay for each feature.
It's not a direct question, "How much are you willing to pay?" People are making choices. When it becomes a trade off exercise they tend to be more honest, and they're not thinking , "They're trying to figure out how much I'm willing to pay. They're trying to figure out my ceiling." It becomes a different type of exercise. That's why it is regarded more highly. The downfall of this method is that you need to get a lot of respondents for it to work, otherwise it's garbage in garbage out. When you're looking at it in the simplest conjoint analysis you at least need 100 people to respond. Depending on how many features you're testing, how many versions, how many packages you're offering. That number goes up quickly.
The last resort is A/B test. This is, I'm sure all of you have heard about A/B testing, you just show someone the control which is your status quo. Then you change one thing and you measure the impact of changing that one thing, and you only want to change one thing because otherwise you're never going to know what really caused the change. I have run experiments where we did accidentally change more than one thing, and we couldn't figure out-- So don't do that, but the right size cohort is going to be dependent on your traffic. For example, if this is on a pricing page, you need to get the number of traffic to show which layout is more effective in converting people to trials. Getting them to trials. Generally you want a 95% confidence interval, so a sample that's going to get you there would be what is useful.
This is good for convincing your management team, because it is effectively creating an environment in which you are testing what the consequences are without having that full commitment to it. We recently went through a pretty big change at PagerDuty, and having A/B testing helped us show that people want to purchase our platform business product. It was one of those one of those things that got us over the hurdle of convincing upper management. Now that we've talked about how we can approach the research, I want to go back to the three pillars of pricing. The pricing metric, pricing level and pricing structure. As much as you want to do your own research, there are common things that we've learned in the market that you should be aware of. Pricing metric is supposedly the easiest thing in pricing to figure out.
You're asking your customers, "How do you want to pay? How should we charge you?" It needs to be simple. The reason why I say that is we recently released a product, and before that we were having a customer advisory board meeting, and one of our customers said "We would definitely buy this, but don't you dare make it complex like your competitor's is." You could be making the world's most beautiful product, the most useful product, but if you're not making sure that your pricing is clear and understandable then you're not going to be able to make the sales that you need. The other thing is you want it to be fair. Nobody wants to pay for anything that feels like an extortion.
I'm one of those people. I work in pricing so I'm very stingy. Every time it's too much, or they're charging me in a way that I don't feel comfortable with, I'm very upset about that. The other thing is that you want it to be predictable, especially if you're rear ending your payment. If people are using your product, and then you're charging them, you really want that to be predictable. Customers don't like it when they can't see what their bill is going to look like at the end of the month. Also, it really needs to be-- This is a no brainer. You need to be able to implement it and you need to be able to make sure that people are not taking advantage of the system. You could come up with the best pricing metric, but if you can't use it to charge people then you won't be making that money, and you want to launch earlier than later.
So if it's going to take you months to build it, then maybe you should reconsider. Other than that the final note that I want to talk about is scalability. You want to make sure that your pricing metric appeals to small customers as well as large customers. If large customers think that it's not going to scale for their size, then it's not going to work for them. You either have to separate them and segment them in a way that they can scale, or you come up with a good metric that can serve both the small customers as well as large. You also want to make sure that customers are not thinking about using your product. They shouldn't have any friction in the scalability or in the adoption.
You want to encourage them to use more of your product, you want them to get more embedded so they have a higher switching cost. You will want to take that into consideration. The other thing is that when you're entering a market or creating a new one, you're going to realize that customers are already accustomed to paying a certain way. However, I would recommend not assuming that what others are doing is the right thing to do. In fact, you could be finding out that they don't want to pay that way and that would be one way that you could disrupt the market.
That's exactly what we did. We recently released a product called Event Intelligence, and it helps us reduce noise so when you're on call you're not getting alerted for things that don't matter. Before we entered the market we already knew that there were competitors that were charging by data volume or event volume. We thought, "OK. We could build that, but we currently charge per user and it makes sense to charge per user because at the end of it the value we deliver is per user. The end value is with the engineers that are on call that are not waking up in the middle of the night for nothing. We have this hypothesis, and it was going to help us get to market earlier if we didn't have to wait to build and to charge in a different way.
What we did was we asked our customers, we surveyed them and we said, "What do you think about us charging per user?" They did the math. They looked at how the system scales versus how users scale, and when you look at that events are going to exponentially increase as you grow, whereas you're going to have a pretty linear growth in your user growth. We went out there with per user pricing and so far it's been working really well for us. It's too early to tell, but we haven't had any customers pushing back and we've heard a competitor is trying to sell now per user as well. The other example is the reverse way of coming at this.
I'm sure you guys all know about JIRA service desk. This was created as an add on product on top of JIRA software. When we created this product we matched it to the core product. The pricing metric was going to be the same as JIRA software which was per user. So we would charge if a customer raised an issue, saying "Your website is down," etc. We would count that as a user. You can't predict how many customers are going to raise an issue, and this definitely didn't resonate with the market. So we had to go back, roll that back out and then start charging by agent. We would only be charging you for your customer support team.
The second pillar that I wanted to talk about was pricing levels. This is how much your customers are willing to pay for your product. It's the million dollar question, I struggle with it everyday. It's the hardest part of pricing. You figure out that it's easier to figure out pricing metric, and it's easier to get at how people want to be paid. You can ask that question. However, this is where it's a difficult territory. Trust me, it's not just hard for you, it's hard for everyone. It's hard for Microsoft and it's hard for Oracle. But one thing that I would highly recommend here is to try to strike that balance of monetization and adoption.
If you're early in your journey you are going to be looking for adoption, but don't wait too long before you monetize because you need to survive. You need to invest in your product, and customers are willing to pay for something that solves for real pain. If you look at Salesforce, they were at the beginning the ones that disrupted Microsoft and Oracle and they were the cheap ones. People couldn't afford Microsoft and Oracle so they purchased Salesforce, and now look at them. They are amazing at monetizing their tools, they created new use cases and they are touching all the small customers as well as large customers. They're one of those few companies that are still growing at 20-30% every year.
This talk wouldn't be complete if we didn't talk about Good, Better, Best. This is one of the norms of pricing in SaaS business today. You have to have something that resonates with customers that gives them choice. People love choice, and people love picking the middle one because that's what we like. The one thing that really helps with this, one thing about this structure that helps companies is the fact that you can serve your small customers without cannibalizing your large customers with the bigger pockets, and you are moving your customer away from "Should I buy?" To "Which one should I buy?" The thing is this is known to work. It has worked for a majority of the companies.
I would recommend not to do two things, one of them is don't give customers too many choices. The recommended range is three to five, and anything above that is going to cause some products of choice. They're just not going to buy. They're going to be paralyzed by the amount of choice that they have. The other thing is don't copy your competitor's packaging structure. You have to do your research. You have to make sure that you identify the good triggers for monetization, good fences, you need to make sure that the good package appeals to smaller customers as well as best appeals to larger customers or customers that are less sensitive. The thing is, even though this feels like a silver bullet, it in fact is not. The reason why I say that it is we recently side stepped from it.
We outgrew Good, Better, Best. Here we are showing some of our problems here, and what we have realized is that we had too many features that were stacking up in our Standard. The most popular package. We also had too many people on Standard that were not realizing the value of Enterprise. We had a product that was coming out and we didn't have a place to put that, and we want to show our customers that we're a platform company and we wanted to show them that we have new products that are coming out. So, we looked at our structure issues and we were like, "We need to make sure that the higher end of our bell curve is fatter." We went from something that looked like this, to this. Making ourselves a platform company.
We wanted to show that all of these new products could get their own space on our website. We did release three products at that time, but one of those products that are sitting in there was part of our Standard package, which really deserved to be its own product and which really should have been a choice for customers. Rather than making them make that big job and forcing them to adopt features that they weren't ready to adopt and creating that friction for us. This is still really early to tell. We went out with this pricing in September, but our early signs are showing us that this has been successful. Again, there weren't that many companies that did that when we went out there, but we did a lot of research.
We worked with an outside consultant. We organized our internal structure accordingly. It was a big change, but you can't do a big change without doing a lot of homework. In short, I want to leave you guys with three takeaways. The first one is make sure to use a pricing metric that is right for you and right for your customers. Second, check with your customers how much they're willing to pay. It could be direct, it could be indirect, and be flexible when your business changes and modify the structure so it makes sense as you grow. It could be that there isn't anyone that has done what you want to do before, and it could be because we are at the beginning of the SaaS evolution and it might be that no one has thought of that yet. So, so your homework and take risks, and make sure that you align your business and your pricing.
Obviously that's not the only thing that we would rely on. It's one of the data points. You want to go through the whole cycle of research, and that would be one data point that you would use. I wouldn't just take 30 people's voices and call it a day. You want to make sure that you that you think things through and that you do your other research, such as talking to your customers with interviews, you talk to your sales people and then you also talk to experts if you can find them. But at the end of the day you might only be getting 30 responses, and you might not have all the other resources, and what you have to do at that point is you have to think about a backup. If I were to go out with this, "How can I pivot?" You have to have backup plans. Things don't always go smoothly.
It's difficult. You would ideally go out with a conjoint analysis, and if you don't have that many customers you could buy prospects and hire a company that could help you do that. You might not have that money, but it is a small investment for a large return. The other thing is there's a couple of good practices, one of them is that you put stuff that is a little more costly and wide appeal in Better package, and then the Best stuff should be stuff that are achievable with less cost but again have some appeal to a good set of customers or a decent size of customers.
You don't. Ideally, again when you do a conjoint analysis you figure out where your relative value lies and then using those you figure out the packages that makes sense. But again, you also have to think of it from a product perspective. You're not going to put stuff together that doesn't make any sense. For example, at PagerDuty, in the recent round where we went from Good, Better, Best to platform plus product, we did think about it from a product perspective a lot. What made sense, which features should be grouped together. We also had a lot of feedback from our customers at that point. So we had the luxury of having a lot of customers to give us that feedback, and we knew where things would be falling. If we didn't then we'd have to do that conjoint analysis and try to figure out where we could tier them up and try to reserve the high value ones for Better and Best.
We did that because we are a company that innovates a lot, and when you are innovating a lot you are coming up with new products. We had Event Intelligence and it's one of our new products that we're excited about, and we're entering a new market with that. When you have a big event like that, how are you going to have a product launch if you have just four bundles? That's one of the drivers of that, and the other thing was we were seeing the signs already. We wanted to drive adoption of some of the features in the Standard package that we were not seeing. The other thing was that people were asking different feature sets, people were asking for discounts or deeper discounts saying "We're not going to use all of these features." So we already were getting all of this feedback, and we wanted to test everything else as well. We wanted to make sure that we were using the right pricing metric, that we were using the right pricing levels. So we did work with a consulting firm that helps us also do deeper research as well as validate our hypothesis.
I definitely am a big fan of trials. My first job was at Atlassian and they don't have any sales people. If you don't have trials you're not going to have those sales, because people will not be purchasing your product. It is absolutely imperative that everyone gives their customers some free trial. In the market I've read that two weeks is about right for trials. Any longer and you're not getting any better, and shorter you're not giving them enough time. I would definitely say that trial is an important part of your sales motion, and I do think that at the end of it if you are good at providing value then that customer will convert.
We do have products that have different pricing metrics. I don't think that you need to have one pricing metric for all of your product portfolio. Obviously if that's how you're delivering the value then it's fine. I do think that for us we wanted to make sure that we weren't missing anything. It seemed like the complexity would scale and we would run into the issue of retaining customers, because at some point it becomes prohibitively expensive. And we've known others in the market that this is pointed as one of their weaknesses in terms of their expansion strategy. If your bill is going up exponentially then you're thinking about that every year. The other question was about dual pricing metrics. I do think that is a possibility, and if you can build it would be good but it depends on your billing and purchasing team's ability to deliver that.
Scott worked in retail , and the margins there are a lot tighter. They have actual cost of goods. It's not that there isn't in software, but we do value based pricing in software, in the sense that we deliver way more value than people pay us for. We solve big problems, we reduce their churn. For us it's more about "How are we going to be able to extract that?" Obviously there are some products out there in the SaaS world such as customer support or express services, etc. that have real costs, and until you get into those types of goods you're not thinking about that cost. Because it's more about your investment in R&D. If you look at SaaS companies the majority of the cost is people.
Freemium is great for companies who are entering a new market and they want to disrupt it and they're fully focused on adoption. You can get to that in my opinion, this is my opinion, with a trial or through a free trial if you're giving them enough time to take full advantage of your product. Otherwise a lot of companies that have freemium products have not been successful in converting those customers into paid customers. The reason for that is it's very hard to find those triggers where people are going from not paying anything to paying something. But if you price it low, people with real pain will have that almost freemium experience. For example Atlassian has $10 for 10 free starter licenses, and those created almost the same cycle for Atlassian.
Again, you want your pricing metric to be scalable. You want to make sure that it makes sense for them. If you look at Atlassian's pricing they do offer volume discounts as you go higher, and that drives reality in the higher positions since they don't think as much about that expansion. If you have sales people you could do some true up models where you could set some boundaries, and then when they surpass that you could give them further discounts. But I do think that there is an element of satisfying your enterprise customers, because generally they are the ones with the bigger pockets and they're the ones that show that you should be a trusted brand or not. You want those Fortune 100 customers on your website.
Resellers and partners. If you don't have any other channel to go through they're definitely powerful. If you don't have your own sales team they would definitely be useful, because they are invested in making it work for their customers and if you're passing through some of the discount to them, then they could be beneficial for your company. It depends on where your company is in its journey.