February 11, 2016
Frameworks for Success: OKRs and KPIs
Heavybit had the pleasure of hosting Sean Byrnes, founder of Flurry and Outlier.ai, as well as Betterworks’ Kris Duggan at our weekly Spea...
In episode 16 of Practical Product, Craig and Rimas are joined by Connie Kwan to walk through the process of buying and selling products on marketplaces.
About the Guests
Connie Kwan is a 15-year Product Management veteran with experience spanning enterprise, SaaS, consumer apps, cryptocurrency, health tech, energy and semiconductor industries. She was Head of Product at Atlassian’s App Marketplace and Chief Product Officer at Carrot and has shipped consumer apps at Microsoft, founded 2 software startups, and is now a product strategist.
Rimas Silkaitis: Welcome to another episode of Practical Product. Today on our show we have Connie Kwan.
Connie Kwan: Hi.
Rimas: How are you today?
Connie: Great, thank you.
Rimas: She wants to talk to us about product and marketplaces. Thank you for being on the show today. Can you give us a little bit of background on your experience in this area?
Connie: Of course. I worked at Atlassian for three and a half years as the head of product for the Atlassian Marketplace. This is the App Store for Atlassian that transacts over 2,000 products within the marketplace to the tens of thousands of customers that Atlassian has.
Rimas: 2,000 products? You did not source all those products yourselves, these were people that came into your marketplace?
Connie: That's right.
Rimas: OK great. Because I can imagine sourcing 2,000 products being very challenging.
Connie: It wasn't all at once, it came in over time. The marketplace has been in existence for over 10 years now.
Rimas: 10 years. That's a long time. Were you overseeing that over the course of that whole ten years?
Connie: No. I joined when the marketplace was commercially transacting for about a year, and prior to that there was a time when it wasn't Marketplace.Atlassian.com yet, it was more like a [distribution] list, and people were able to find out solutions that they can buy directly from the provider. Not through Atlassian, it was more like a Craigslist style type of interaction. Over time there was such a demand for a more formal way to transact that the Atlassian Marketplace was born.
Rimas: Awesome. You've got 2,000 products in there and you transition from this listing service to a marketplace, how did you create that demand?
Craig Kerstiens: You had to start somewhere. Whereas now they come to you and show up, once you've got 2,000 you've got a large marketplace. People show up and expect to reach out to all those developers. Where do you go and get customer number one for the marketplace? How is that transition from a distribution list, where I'm sure people weren't showing up in the same way, people are recommending other things to each other. How do you go to customer number one when you're launching a marketplace?
Connie: You're talking more about the side that buys the apps, versus the providers of the apps?
Rimas: Actually, the provider side. Because that's the side that I'm most curious about as a product manager. I have a product that I have, or I want to go build something new, why would I go into your marketplace at all? What incentives do you create for that?
Connie: Yes. Often if I was trying to judge whether I should list in the specific marketplace, then to me that is a distribution channel. I would look at, "What is the reach of this channel for me? Do they have 50,000 customers or 30,000 customers or 10,000 customers?" However many are in there, I should understand that. I should also understand whether those customers are all available markets to me. Some of them might not be the right target. Understanding the mix of customers that are within their reach is important. And then I determine, "OK. I make dev tools. 100% of Atlassian's Marketplace customers or Atlassian's tooling customers are actually customers of mine as well. So I want to reach all of them and that's a great fit. But if you go to a different marketplace like GitHub, and you say 'Only about 50% of those customers because they're way more specialized. Just 50% of those customers are going to be important to me. But these customers have a strong need for my product.'"
There's almost two dimensions you're judging on, the reach and sheer quantity, and then there's the pain. How much will your product provide alleviation of that pain? Because both are important in terms of the number of conversions you'll get later, and the value of those conversions.
One is more sheer number of customers you get, and the other is more customer lifetime value.
Rimas: As a product manager looking to enter a particular marketplace, do these marketplaces or the companies associated with them publish a lot of that customer information? About who their customer base looks like, so that I can go make those decisions that you're talking about? Or is that research that I typically have to go do on my own?
Connie: The fastest way to do it is just to Google it. You can probably find out if it's a public company. You can most likely download a 10-K and actually see how many customers they have. They often talk about that, but if they don't then maybe you watch their LinkedIn or Twitter feed and maybe they have celebration milestones that the company announces. Those would be good ways to find out. If not, then you can just email the marketplace and ask them. I'm interested in listing. You are going to get some money out of this, because you're going to take your distribution cut from me. So as a customer from the supplier end, I want to understand exactly how much reach I will get from your marketplace, and also what types of customers might be in there? Can you share with me that info? I think that if you're legitimately interested in listing and they can hit your website and see that you're legitimate, there's no reason why they wouldn't share that with you.
Craig: Generally you'll find each of these marketplaces, if they're large enough, will have a full time PM dedicated to it as well as an account rep that is like a salesperson for the marketplace itself. Usually there's a point of contact that's pretty easy. It's not like you're just going through standard support. This is what they do, is onboard customer after customer as a provider in the marketplace really commonly.
Rimas: Didn't you run marketplaces at Heroku, Craig?
Craig: I did for a little while. We went from that early stage of early add ons ecosystem, more self-service, to growing into a full time partner manager into dedicated PMs and hired several other PMs onto that team, the dedicated engineering team. It's not just tacked on the side of these products. Oftentimes it's very key to their growth. I'm curious a little bit on the other side of the tradeoffs. It's a valuable channel, but for a PM out there that's building a product and growing your own customer base, investing in this channel is also a tradeoff when I'm not doing something direct. How do you weigh the pros and cons, or what are the downsides that you should be aware of before you go and say, "Let's go get on 10 different marketplaces, because then we have 10 more channels. That means 10x revenue. That's how that math works, right?" What are the tradeoffs you should be aware of and careful of and think through before you jump ahead on in?
Connie: The channels aren't free to you. For every channel that you choose to invest in, it doesn't make any sense to invest on the surface level. And by that I mean to go in there and dump your assets in, type in the description and then log out of there and hope that money starts flowing in. Most likely that won't happen, at least nowadays. Because there is so much competition within the marketplaces already that it's hard to get to above the fold. So you need to work at it, and a good amount you need to work at it. It's most fruitful if you can concentrate your efforts in a couple chosen marketplaces that you want to be a part of.
Keeping in mind that while you are giving up likely 25% or so, 30% maybe depending on the marketplace in terms of the revenue that you bring into that marketplace, you really want to spend time leveraging any resources that they might provide to you so that you can rise to the top of the pile. Believe me, that will be a big pile in there, but hopefully you've done your research and you've looked at the marketplaces and you've understood their reach. You've understood the percentage that's going to matter to you, and the pain of those customers, and also the competition that's in there. So that you can have a very clear picture, and most likely the same competition is living in there alongside your direct channel. It's not new research necessarily, but it's a slightly different angle in terms of "OK. How do you position yourself specifically around the subset of competitors within this marketplace?"
Versus that other marketplace. Because the reality is customer X is going to come in and they already buy Jira. It's likely they'll just buy things that connect to Jira, so they'll only look in the subset of things that connect to Jira. But you're in there, and instead of competing with 10 things now you're competing with five. Which is great for you, but how do you stand out amongst those five? How do you get in front of those customers differently? You might want to start investing in showing up at the events around those marketplaces. Atlassian for example, has an annual conference, and if you're at the Salesforce marketplace they have an annual conference. GitHub, I don't know. Do they have an annual conference yet?
Rimas: Yes, trust me. We've got an annual conference. It's in a couple of weeks.
Connie: There you go. GitHub has an annual conference. Do you have a conference specifically around marketplace developers?
Rimas: We do that as part of the conference. We have that full week, and then it's in there and we do some work with our partners. Yes.
Connie: Cool. Some of them split them up into two different conferences, so keep in mind that that could be true. The customer conference is more useful to you as a channel for winning customers to yourself. The other one, the developer one, where you go in as a vendor partner that one is useful for meeting other vendors in the marketplace. So that you can understand different ways of transacting and being effective in that marketplace, and you'll find people that are not necessarily direct competitors to you. They might be partnerships to you, they might make complementary products to yours, and those are great people to meet and just learn from each other on because you can be complementary to each other.
Throughout all of that the investment that you put in is directly related to what you get out.
It was the number one question that I got at Heroku when running the add ons marketplace there, which was from the add on partners, "How do I make more money? How do I get more out of this? When I have someone who goes over their API limits, how do I get them to upgrade to the next plan?" And my first question was, "Have you emailed them? Have you turned them off? Have you cut them-- Do you have a hard limit?" And they're like, "No. We don't do any of that." "Do you do that on your core products?" You need to, with all of these integrations with a marketplace, if you build a better experience you get better returns from it. What you put into it very much determines what you get out.
Connie: For sure. You're talking about a deeper integration now, because obviously you can either iframe something into their page, that's a light integration that may take you a week or two. Or you can literally have a whole team dedicated, which happens. Some vendors dedicate entire teams just to integrate with the Atlassian stack, and you get this really deep experience where it feels like part of the core product and those are products that really stand out when customers choose. You have that tradeoff to make. Do you feel like this is a strong enough channel for you to actually invest features into your product that serve only the subset of customers? Or do you feel like you need to keep it generic, but realize that you're going to be less competitive in some of these marketplaces? The other aspect I didn't yet mention is pricing.
When you go into a marketplace you sometimes lose the ability to price the way you want to. That's unfortunate, but it simplifies the purchase path for your customer, which is the reason it's done. The benefit you have as a vendor when you enter a marketplace is your customers no longer need to get you on their vendor list, they can just buy through the marketplace. That's huge. You know all of the hurdles you have to get through in order to pass through all those signatures to get yourself paid. With listing in a marketplace you skip all that. The downside of course is you price how the marketplace decides you should price, and if you cannot price the way they do, for example you charge by seat of active user but the marketplace requires you to have two different types of users. One that is a write user and one that is a read user. Then you must create some way of conforming with their need, and sometimes that's easily hackable, sometimes it's not. The pricing aspect might also be another factor to help you determine which marketplace to choose.
Rimas: You talk about price. I'm curious in the general context of, you've entered into this marketplace on the operational side, when you are interacting with a lot of these customers, whose brand suffers more when there are problems and there is a support issue? Or there is something that happens, maybe it's the connection that exists between the two products, or something else like that. Is it your problem as the integrator, or the supplier? Or does that problem land on the company or the core marketplace provider?
Connie: The platform provider will always seek to shield themselves from liability as much as they can, or bad PR. That's true for any platform. Look at Google and Apple, they take big steps in ensuring that it looks very different what's theirs versus what's yours. That's understandable because ultimately if it breaks, 9 times out of 10 it is the integration's problem versus the platform's problem. Now, 1 time out of 10 it could be something on the platform went awry that broke your code. But unfortunately you will get blamed for that as well, so it behooves you to stay up to date with any API changes that might happen on the platform. There are quite often mailing lists you can subscribe to or RSS feeds you can subscribe to, ways to get notified so that when things change you'll have some heads up about the changes, so that you can get ahead of it and test it before it goes out. Then you need to get your update out around the same time so that customers have time to update and not have broken code.
Rimas: The lesson here is that if you are going to go into a marketplace, you have to nurture it and stay on top of it. There's some brand risk for yourself because the platform or the marketplace is going to say, "Here are our boundaries. This is where the problem is, it's over there. If you're going to interact here be a participant, be there, do those things."
Connie: Yes, exactly. It goes back to the, "It's not free," you have to adapt your product and messaging to that platform, but also stay on top of it. It's an investment on your part.
Craig: I've often found that the support channel is one that is often neglected and not thought about. Like, a ticket is open, and it comes through the marketplace central support system and then it gets routed. You should be in there looking for those, paying attention, responding in the right SLA timeline fashion and that sort of thing. Maybe you went through that onboarding when you first launched the marketplace, but is the rest of the support staff fully trained on it? Are they paying attention to it? That's one of those things that I often see that's overlooked and missed, and it can create that negative experience for anyone using the integration.
Connie: That's a really good point, and I think the training load is sometimes overlooked in that when you add more channels it complicates life for your support folks. Having clear documentation certainly helps, but it will make it more complicated. So, choose wisely. Maybe start with one and then expand.
Craig: I'm curious on one thing. We mentioned pricing briefly. Most of these marketplaces take a cut, some do not take any cuts, it's pretty clear that if you can fit into the right axis that's maybe slightly different than what you price by direct but it's close enough. Or if it's seats and you can price by seats in the marketplace, great. You're identical. But what happens when the marketplace does take a 20-30% cut. Do you charge the same as you do direct and take that margin hit, or what do most companies doing? What works and what doesn't?
Connie: Here's the reality. A customer will find you on the marketplace listing, but they'll find your normal website too. So if you have $10 listed on your normal website and twelve dollars listed in the marketplace, it's going to raise some eyebrows, and they're going to want to know "OK. Is the $10 version different than the $12 version?" If you can say yes to that you can probably get away with it. If you say, "The $12 version is because we spent some extra effort to integrate better with the platform, and you'll find these particular benefits because of what we've done. It represents an actual investment on our part and therefore it costs more." "OK. I can understand that." But the optics will get funny and you might get some complaints about it. You'll have to dance that line, and the example I gave is that it's a small enough gap that they might not notice, or they might just say "That's fine. I am willing to pay the price difference." If you make it much larger you might lose customers, they might lose faith in the way you do business.
Rimas: Generally though, I've been looking in the news, platforms at large or marketplaces, we're using those two words interchangeably. There's a lot of competitive pressures on them right now. The last time I checked back in 2017, Salesforce reduced their take rate from 25% to 15%.
Craig: There's nowhere out there that is 0%. AWS is a great example. AWS is nice and small and quaint.
Connie: Exactly. Google and Apple platforms also reduced their rates. I believe they went down to same 30% for first year and then 15% for subsequent years, which is a huge shift in the culture of how marketplaces charge because they're big leaders in this space. It's worth asking, again you are a partner but you're also a customer to this marketplace. Keeping in mind you are also paying the bills for some of the PMs or dev teams that are in there building the marketplaces, so act like one and ask questions of them, and make sure you're getting the value you need to be a part of their ecosystem because you're helping each other out.
Rimas: At some point if you get large enough then maybe you can throw some of that weight around. If it is a big enough distribution channel for you and it is for them, as you being the supplier, then to your point you can be the customer. Ask for things, do stuff. I saw Netflix is even asking to get out of the in-app subscriptions at Apple. So, there you go.
Connie: You can certainly ask for things, and within reason. Asking it artfully and respectfully is important, this is a partner and they're not dependent on your business. In fact it's quite the other way around for the most part, but don't be shy to engage in that conversation is what I'm saying.
Craig: We've talked a lot about going into a marketplace. I've got my product and I'm building to get into a marketplace. I'm also curious on the other side, "I'm building my product. I want to build a marketplace and platform that others can build on." When do you start to go from "OK. I'm just a product," to "I'm a platform now that has my own marketplace component." When's the time for that? How do you approach that decision?
Connie: You ever see in the ocean the big whale shark that goes by, and all these little fish that swim next to it? I use that analogy because that's salient here, where if you're the whale shark then you're the platform, and you can support a number of partnerships on your side. At some point one of those fish can grow to be quite large, and they can become another whale shark and split off. You can be your own and you can have your own ecosystem, but it's going to take some time and the transition is gradual.
It doesn't make any sense to invest in your own until you have a strong base of customers, because it takes effort to build an ecosystem.
Companies are hosting conferences, companies have separate teams to manage the ecosystem, you're managing a lot more relationships. You have transactions happening in there. There is off the shelf marketplaces you can rent. It's gotten to that point where a lot of people want marketplaces that you can just rent that service, but that's another thing to maintain. So it's really that you've got to think about it as another product. The question is, "Is your company large enough that you want to support this product that allows you to become a platform?"
Craig: Is there a common size in terms of employees or revenue or users, that you often see that you're across this threshold, it makes sense?
Connie: Interestingly, it has more to do with the type of product you have than the size. Obviously if you hit a certain size you have a much higher success also. But that size is much larger than the audience that we're talking about, at least definitely larger than startups. But if your product is uniquely a platform, then it makes sense a lot sooner. I want to focus on those cases . Twilio for example, they're a platform product and they have an ecosystem of things. So if you are a platform product and you're likely to need to connect A to B in order to be successful, then in fact it's part of your product strategy to provide this marketplace.
It may be a marketplace, or it may just be a Craigslist style listing. It may be you create the interconnections on your own. You build those integrations yourself, like some key ones because you absolutely need those. There's a huge spectrum. But it's useful to understand, "Is integrations a huge piece of my success? Can I be unsuccessful entirely if I don't have it?" And then, "OK. If I need to have it, how much do I need to be successful as a product? If I just need these five key integrations to be successful I'll just go build it. But if I'm going to need 2,000 because I sit in the middle of a workflow that is going to have all these offshoots, then it might make sense to invest earlier in creating that ecosystem."
It will take 10 years to get to the point where you have 2,000 suppliers in there and you convert one at a time.
Starting early and keeping it simple in the beginning is going to be key for you.
Rimas: Can we go back to one thing, can you define platform for me? Because sometimes I have trouble discerning between what is a product and what is a platform. I know for a lot of folks that I talk to, they split the difference sometimes. How do you define platform?
Connie: Let me define the product first. I define a product, a pure product play as your customer or the person who uses your product is the one paying you. Same person. A platform is one that is two different people.
Rimas: I would assume it's that you've got a seller, and you've got a buyer, and there's a transaction happening without-- You might be facilitating the transaction but you're not necessarily the one taking it from both sides. Maybe that's not the right explanation.
Connie: Airbnb for example, they are a platform because they list all these rooms, or somebody else lists all these rooms actually, and that person gets paid through you. So you're getting paid from A who is renting a room from B. It's not a direct, it's not like a hotel where I'm providing the rooms and so A comes and stays in my room and pays me. That's simple. If I'm Airbnb now I have to juggle two different relationships. I have this chicken and egg problem where I need to grow enough rooms in my system that aren't rooms that I build, and find enough people to stay in those rooms, and then somehow take a cut out of it.
Rimas: In software platforms it's still part of the same concept. That applies in those contexts as well.
Connie: There's the technical platform, sometimes we will use that in the technical context, where they're talking about something you can plug in with APIs. It gets a little blended because when you are a platform in the business sense, you need to have a platform in the technical sense quite often, especially with dev tools. In order to enable this interaction to happen, not so much with Airbnb because it's a pure consumer platform.
Quite often if you have a dev tool that is a platform, you need other dev tools to plug into it, and in order for other dev tools to plug into it you need to have APIs.
Now they plug into your product, so now you need a transaction platform to enable that A to B selling to happen through you. That's why you have two layers of platforms that are happening at once.
Rimas: For a lot of the developers who listen to our show, if you're thinking in API contexts you're already starting in that direction in which you can start thinking of yourself as a platform. So long as whatever it is that you're building, maybe it be API related or not, so long as that can facilitate some transaction between two parties that can happen. Then sure, fine. You're a platform.
Craig: I'm curious on the common stakeholders that you have as you've made the decision to go ahead and integrate with this marketplace, who usually is involved in that decision? How do you come to it? Is it just one product and engineering team, is it sales and marketing? How many people do you have to get on board? Who's optional, who's required, what's the typical process of getting that buy in there?
Connie: Most likely you'll want to get sales and marketing involved, and talk through what your shared responsibilities are and what you each own. Because marketing might come in seeing this as a channel strategy. Sales might also see it that way. It's important for the company to fully understand how everybody is seeing it, and you should see it in the same way. For example, if you are looking at it straight up as a marketing channel strategy, you're dipping your toe in it and you're just going to go ahead and register for free and list it. You're not going to do any integration work. In fact, the listing is just going to have a button and it's going to come back to your own website. That's a pretty simple listing, and that might just run in the marketing camp for a little while. It's an experiment that they do.
They're going to track the clicks and they're going to see if anything converts from there, and that gives you some indication of the demand from that marketplace.
It evolves into, "We have this overwhelming influx of people. We're going to want to build some features around it." All of a sudden your product team is more involved and you start having to mutually understand who's going to start taking those reins. If someone comes in from support channel asking about it, who's going to answer that? Is support trained to answer that, is product going to train support to answer those questions? Or is marketing going to train support to answer those questions? It depends on the depth of your integration and who's involved as a result of that. But certainly anybody involved usually in sales marketing and product should be in the room. Then when you get to pricing it's the same folks again, and you might even involve the CEO in that case depending on the size of the company, to talk through that.
Rimas: You mention marketplaces being a distribution channel, or being the top of the marketing funnel for you and your product, etc. What other strategies exist for me as a supplier of going into a marketplace, beyond customer acquisition? Are there any?
Connie: There are some marketplaces that offer or are contemplating to offer support from a monetary perspective, to products that list in their marketplace. Salesforce is worth looking up to see if they have a fund around it. I know that Invision has a fund, Invision is the design tool. They have a fund around that.
Rimas: Doesn't Slack do this as well? Where they will fund integrators to build on their platform?
Connie: Slack. Yeah, that's right. Slack has a fund. You may be able to apply and qualify for some of this money that comes through, and even more importantly than the money is your relationship with them because now you are much more prominent in the eyes of the people within the platform company. You have those relationships that help you understand, "OK. How do they rank and stack the products in there? How might you be able to get access to ad spots if they provide those? What do customers gravitate towards?" There's a lot of information that will be helpful in positioning your product in the marketplace that you can only gain from going to the events, meeting the customers, knowing the team. It takes time to build these, and that's the investment that you do.
Whatever way you can do that and if it's by being a part of a funded company, that's a really great way. You should also ask if they run any hackathons or do they do any, not monetary sponsorships necessarily, but development weeks where they might bring you in and do some development together. Sitting next to the team that builds the APIs, you should be able to understand the value of that, but they are interested. They are interested in understanding you as a vendor, "What do you want to build?" So that they can provide you with API endpoints, and you're interested in getting those API endpoints.
If you can sit next to them for a week or two and if you can find an opportunity to do that, just ask the platform provider if they can do that with you, or if they have a program for that. It will help you build the bridge for later on if you need to do something special with your product. You can ask for those API endpoints and make your products shine.
Craig: Beyond that, we mentioned the channel in distribution, but there is also the knowledge. If they're the big fish, they have a better view of the landscape, they have more direct access to more customers and they can help inform you about where the market's headed and how things are changing. Where you should be headed as you build the product. There's a lot of opportunity, not just from a "Let's have an extra channel and get some extra revenue in users," but from a strategic perspective as you build up that relationship with them.
Rimas: At what point do you say, this is to take the conversation in a little bit of a different direction, at what point do you say "I'm pulling out of this marketplace." What are those reasons why you would want to leave a marketplace?
Connie: I can see a scenario where you invested in say, four different marketplace channels and one of them is performing very poorly. You might decide, "Going to pull that one." It's not without risk because all those customers need to get transitioned somehow, and if you have a deep integration then it could be a hard conversation. You're essentially deprecating them, and you're not supporting them anymore. You might be able to transition them out if there is not a deep integration, and you might just maintain them for a little while, but you're cutting off a set of customers. So that's a hard one.
Rimas: Is it better to get rid of the integration itself, or just let it sit there and just leave it as is?
Connie: It's better to tell your customers as soon as you're able to, and give them a deadline for when you stop supporting so that they have ample opportunity to look for alternative solutions.
Rimas: We had a separate episode here on Practical Product where we talked about deprecations. It's important to have that relationship with the customer and tell them as soon as possible, or as soon as you can. Because they're counting on you to have that service, but it's also a very challenging thing to do.
Craig: Just because it's a marketplace doesn't mean you're immune from all the other standard product practices.
Connie: It's possible you decided to have your own marketplace, and you're going to stop integrating with other ones. That's an interesting transition. I personally haven't managed through that before, and that might be a trigger point for you to stop listing in other ones, but you may still list as a way to keep the funnel going without deep integrations. I don't know what that looks like yet but it seems if you've done the research and invested in a channel, and presumably you've done a bit of either experimentation to see what the flow might be or you've directly participated and talked to some customers in that space, that it would be a pretty serious decision to remove yourself from that ecosystem.
Rimas: There's always a whole host of variables and decisions companies go through, or changes in strategic direction that may require changes in distribution. Who knows. Any number of reasons could exist for someone to want to pull out of a marketplace. We've spent a good amount of time talking about marketplaces today, both from listing your service in one or creating a marketplace itself. What are ways that I can learn more about it?
Connie: There's a couple of great talks within the Heavybit library that you might want to reference on this topic. I as a product strategist help companies chase down product hunches, so if you have a hunch that marketplace might be a good fit for your company, I certainly welcome a conversation and you can reach out to me at ProductMaestro.com.
Rimas: Awesome. That's great. Thanks for being on the show today, Connie.
Connie: Thank you.