November 20, 2018
Ep. #23, Automation with One Medical’s Zach Powers
In episode 23 of The Secure Developer, Guy speaks with Zach Powers, CISO of One Medical, to discuss the evolution of security at One Medical...
In episode 14 of Practical Product, Craig and Rimas look at the differences between following a product roadmap and framing a strategy around product direction and user engagement.
Rimas Silkaitis: Welcome to another episode. Today we're going to talk about roadmaps. That's one of my favorite topics because I absolutely despise the word "roadmap."
Craig Kerstiens: That's fair. Who doesn't hate the word roadmap? You hear it from engineers and it's a pain. Now we're working on things for the next two years with no creativity, with no input to the process and it's not going to go how we planned.
Rimas: Is it fair to say that's your definition of roadmap?
Craig: That's a good question to start with. Before we jump into it, everyone hates roadmaps, but what is a roadmap? How do you define it, and what terms make sense there?
Rimas: For me, there's varying definitions of roadmap. I'm glad we started off with that question first and foremost because what roadmap means to one person can mean something different to another person.
There's this spectrum by which roadmap can mean something feature based, where it is, "I'm going to implement X," and that's one type of roadmap. To all the way on the other side, which is, "We're going to build some grand vision that may materialize five years from now."
Craig: Those extremes make a lot of sense. You've got the Steve Jobs vision iPhone, which I don't know if there was a roadmap for that. There was definitely some vision there, but much more abstract. Then you've got a project plan. You've got your old Microsoft project timeline Gantt chart, "This feature depends on that one." Very waterfall, which is a very different structure.
For sake of this conversation, how do we want to frame roadmap? It makes the most sense first to drill into, what's the goal and reason behind a roadmap? Why do we even have them in the first place? Because those are two different things, a long term vision as well as an exact project plan. Which ones are for which purposes?
Rimas: Great question, Craig. In typical PM fashion, answering a question with another question. I like that. In my mind the intent of roadmaps are a communication tool for what is happening inside the company, outward to the customer, in which case features can be part of that. Where you're talking about what things you're going to implement at a granular level.
But I like to bring it up a little bit higher because for me, the intent of roadmaps is to communicate a story. "This is where we're going to go over some certain timeframe," but really give the development teams or technology teams the optionality to change up what needs to happen relative to that story.
Craig: You hit on one key there that I agree on, which is it's for external.
It's for customers. A roadmap very explicitly is outward facing, it doesn't necessarily have to be public so that anyone can see it, but it's very much outward facing. You don't create a roadmap only for internal people, you create a roadmap to share with customers.
Rimas: That's interesting. I've seen the roadmap done by various companies where they do publish everything and it is very feature based, and it's super transparent. But for me what worries me about taking that approach, is that it puts you down a path where if you want to do something that may not be in line with what your customers want at a granular level, you now have a very difficult conversation to have with said customers.
Craig: That jumps to the question, "OK. You have a roadmap. What's the best way to communicate it to customers?" Maybe first though, what is the right level to communicate to customers? Because we talked about that for a brief minute there, on how there's a specific feature project timeline and there's more vision thematic. What's the right level that makes sense?
Rimas: Going back to the intent, for me it's, "What's the story that you want to tell with your customer?" That story can happen at many different levels depending on where you are and what industry you're in. Classically, on this show it's always about, "It depends." If you're talking about an industry where, let's say there's massive amounts of competition. Maybe it's very price competitive, and each granular feature that you implement puts you on a path that is maybe ahead of your customers.
Then maybe the appropriate approach is to say, "We are going to implement X because it enables some functionality that you might not have otherwise had in said industry." But for me personally, the intent that I have with my roadmaps is to communicate a story, a broader story in which this is about themes.
It's about, "We want to accomplish something like leveraging our data to enable developers to do more on their behalf." With that theme, that gives the engineering teams that I'm working with the optionality to say, "OK. We need to leverage our data to do something on behalf of our developers. Let's go test a bunch of different things out and see what works, what sticks. Then we can tell that story down the road when we launch or release said feature."
Craig: That sense of themes gives you a lot of flexibility. You have a thematic vision, but also some idea of where you're going. We led off with the fact that we both hate roadmaps, they cause a lot of pain to engineers and they cause pain to salespeople when you give them a roadmap. They give it to a customer and you don't deliver on it, so now they just sold a thing you can't fulfill and you're giving money back, or a really unhappy customer.
A roadmap is to tell customers a path of where you're going. I also view it as the opportunity to use that to capture input.
Here at Thematically is we're going for the next 12 months, maybe in six-month integrations is a big theme for us. I use that as an invite to talk to our customers and get input of, "What integrations do you want? You told me you wanted our product to work with X, Y and Z, but we were thinking about A, B and C. Which of these makes sense for you?"
Rimas: I like to use the term "product direction" instead because it automatically implies that this is a way in which we're going, not to say that this is the roadmap. Because roadmaps imply that there are signposts and things that tell you that you are definitively going in a particular direction, whereas product direction is a little bit more vague in that regard.
There's a little bit of a side note. I'm curious, Craig, how do you create themes for those product direction or roadmaps, as we're calling them today?
Craig: There's a few different ways. I look at a bunch of pieces of input. One is the engineer team. For me, there's a lot of gathering input from support tickets from our engineers directly, from market trends, things like industry analysts, but especially customers. That's number one front and foremost, and it depends on the size of your company, for a larger company may already have this defined process.
For a smaller startup, if you're 5 people 10 people 20 people, you probably don't have something like a customer advisory board. Any large corporation or enterprise is going to have one of these, and you may not have experienced it, but they've got a set of top customers or a sample set of customers.
They're Fortune 1,000, they are SMBs that give them input on, "Here's our candid feedback." They put them in a room together, take them out to dinner, and "Tell us where you want us to go," in a very open fashion, "What do you like? What do you dislike?" One of my favorite things about getting customers together, is they'll share both the good and the bad. You can see the exciting ways people are using your product, but it's also a place to sit and be quiet and listen to all the feedback. That gives you the places to improve.
Rimas: Do you think that the mechanism can be used regardless the size of the company? the Customer Advisory Board, specifically.
Craig: Not quite. You need to have a certain base of customers. If you're starting out and you've only got 10 customers, you're a little early. If you're 50, 100, 1,000 customers-- It doesn't need to be all your customers. At larger scale you can find multiple ways to capture this feedback. I find a customer advisory board is a selection of certain personas of customers and types of customers, and ones that speak to that broader set.
But it's not every customer, so you're a limited bias there and a limited selection set. Which works out well, but once you have 50 or 100 customers you could have the opportunity you have a customer advisory board. This is assuming you're a higher price product.
If you're a lower-price product, $10 a month, you're not taking them out to a nice steak and wine dinner.
I might, just because I want a good steak and wine dinner. But you're probably doing something more like a beta program for feedback, that sort of thing.
Rimas: OK. Got it. Interesting. You're talking to customers, granted the whole show is about talking to customers. We're product people, that's our job and it's in our nature.
You've gone and created a few themes, you talked to your engineering teams, gone and looked at support tickets, market trends, etc. You've come up with those themes. Now what happens? How do you communicate that out to those customers that you've been working with?
Craig: There's two different ways I approach it. There's one customer advisory board where I'm being much more transparent. "Here's our themes, they're early. They're rough."
Rimas: So it goes both ways, you're saying--?
Craig: Yep. I'm communicating back out. Oftentimes I'll have a day once a year, we'll invite them out, we'll have them partially for updates on the company. The product, the direction. Then we'll go open sessions where they talk and share with each other. Wrap it up with a nice dinner, that sort of thing.
It's a two way street. We're giving them a lot of insight into where we're headed, and they're giving us the feedback to help shape the roadmap. That's one piece. The other piece is I'm giving a version of this directly to sales.
As we mentioned in previous shows, I love to rotate through my interactions with different teams. I'll have a meeting that's the same time on my calendar every week, Tuesday at 10 am. Once every six weeks it's with sales, another week it's with marketing, another week it's with sales engineering, another week it's with marketing, and so I get coverage with all these different teams. In the sales meeting, I'm very much giving a, "Here is what you can communicate about the roadmap." I'm explicit on two things.
One, "Here's where I'm thinking on the roadmap, and what I'm thinking of theme's direction focus for the next 12 months. This is for your knowledge, so that if you see someone say, 'I want support for Rackspace.' That perks up the salesperson ears a to come to me and say, 'OK. Let's drill in. This person is interested in this.'"
They're a great person for customer development, but then I'm giving them another version of this that's a slide that is not as transparent and not as detailed, saying, "Here's what you can share with any one. Direction, thematic, higher level." Because the worst thing I can do is give them something of, "Maybe we're going to work on this in nine months," them go sell it to a customer and us not deliver on it.
Rimas: That's the balance that you have to strike with these product direction decks or roadmaps, is that you want to communicate the future but you can't communicate it too far in advance, because the risk to your point is that if you don't deliver on those things or don't deliver anything associated to that theme. The risk is you say things and you can't put your money where your mouth is, your customers start trusting you less and less as a result down the road.
I want to come back to the sales piece of this here in a second, but I want to talk about these two points. They sound very enterprise focused, is that a fair assessment?
Craig: Overall, a good bet is yes. One thing that you'll see, and my head of sales came to me ironically earlier this week saying, "I need a roadmap." I'm like, "You have these things. I need it packaged this way. People are asking for it. We're talking to these bigger companies, we're talking to this Fortune 100, that Fortune 100." Very much as you go to the enterprise you're going to expect a roadmap. It's one of those key terms they're going to expect, want to see.
Enterprise buys, if you're a startup, they buy your product not just because of the features, but because of your direction. They're investing in you, not for today, but for three years or five years from a directional perspective.
They want to see if that aligns. So you can miss a whole bunch of features that you think need to be there, and if you can paint a picture of where you're going, you can punch above your weight class much more than you expect.
Rimas: OK, sales and roadmaps. I have some comments on this one as well. Because in my own career what I've seen is sales, to your point, wants a roadmap done in a particular way. For all of you out there, that usually means that they want to see a punch list of features.
Craig: It's this quarter, next quarter, the quarter after, the quarter after. It's at least four quarters. And here's the features.
Rimas: To the extent to which the fiscal year ends for you. If your fiscal year ends on January 31st, they're going to want to know all the way up until that point what features are releasing, so that they can hit their numbers throughout the course of the rest of the year.
Craig: There's a small tip there that maybe not everyone knows, certain companies have different fiscal and financial years. This is a very interesting thing that you'll learn in the sales process. I'm sure it's been covered on another Heavybit episode, but maybe we can reference some notes if there is one.
Basically, some end of the financial year is January 31st, not December 31st. Which often means they have budget and it's use it or lose it, so you want to know when you're selling to someone what their fiscal year is or when you're buying from someone. You can have more negotiating power.
Rimas: OK. Sales, features, roadmaps. They're going to want to know the exact feature and the exact time, when it's going to be delivered and what it's going to look like, and how much it's going to be sold for. Because I can't tell you how many times I've had a conversation in which it goes, "OK great. You've launched this feature, but are you going to price it? Can we sell this thing or is this just included for everybody?"
That's where the rubber meets the road as a PM because you have to pull that conversation back and say, "Possibly you have to have done that work ahead of time to make sure that as part of your thematic roadmap, that you've said, 'OK. This feature is a part of this and we're doing this and releasing this in a particular way to meet a certain customer need.' Sales may or may not be happy with that in your roadmap."
Being able to pull that back is super important as a product manager. It's going to take practice. Trust me when I say that, because you're going to try it once, you're going to try and land on themes and sales is going to say, "No I need to know what I can sell. Because I've got a bunch of customers and I can go back to them and sell more stuff to them."
Craig: Your ability to do that will impact your ability to be a successful product manager. If you go ahead and roll over to sales and take what their direction is, you're going to develop a mishmash product of a bunch of features without a theme, that marketing says "OK. How do we market this?" You need to hold the reins of that very tightly.
It is a balance. You're working with sales, they sell your product. You're working with marketing, who markets it out. But you do need to hold the reins of that and find that balance.
Rimas: Definitely. To double down on this conversation, when you do work with sales when it comes to roadmaps, it's not just about the feature that you're releasing. It's about that story, and what we'll save for maybe another episode, is solution selling as a product manager. Because when you go in and talk to customers they're there to listen to you as the PM, because you're tied into technology, and they want to say "OK. Tell me what's going on with my own business and how you can make me successful with your product direction."
That's why I keep going back to roadmaps being about product direction and stories. Because if you've done the commiserate work around customer advisory boards, all your support tickets and talking to customers. Everything else like that.
You should be able to pull all that together and say, "Here is the arc that we want to go commiserate with your vision." If you're the CPO or CEO, or if you're working with an exec, you're able to pull what they're doing together or your own stuff, and say "OK. This is where I want to go, this is what I'm hearing from the field. This is the story I want to tell and this is how this solution is going to make you a successful company via this roadmap that we've created."
Craig: Early on if you're a small startup there's a good chance you've built something just to win one customer, or hacked on something that's held together with duct tape and glue. It was that solution selling, or solution product. It was, 'Here. This isn't our core product. We built this."
A lot of times your roadmap can be, "We've got to take this thing that worked for this one customer, and now we're going to productize it." So you know a lot of that, "We built this one thing," make sure you do the actual work to push that over the finish line and make it readily available to everyone.
Rimas: 100%. I cannot stress enough to all of you listening that this is about how you make the customer on the other side successful. Because they have jobs, they have things that they need to do in the course of their own daily work life that if you are going to be creating a product or creating a product direction or roadmap, whatever we want to call it today. That thing has to be able to go in and say, "I understand what's going on at your company. This is how this solution makes you successful in it."
And again, don't focus on the features. Focus on the story that you want to tell, and by extension I guarantee you-- Well, I don't know if we can put that in writing. But it's more probable than not that you'll be able to pull those customers in, convince them that your solution is the one, and you'll start seeing sales and your product direction decks will work for your sales teams.
Craig: This makes sense when you've got a lot of input and clear product, and you can create a clear thematic vision. I'm curious how you approach this when strategy or major initiatives change, how do you create a year timeframe roadmap when you're completely up and shifting your market focus every few months?
Rimas: I hate to use the agile analogy here, but when you look at agile and what it's prescribing, it's called Scrum. You're taking the entirety of the software development lifecycle and compressing it into a smaller time frame, which for a lot of scrum teams usually means one to two weeks. The same can apply to roadmaps as well. Typically we build roadmaps for a fiscal year, or a calendar year, or something to that regard.
If you know that there's too much uncertainty with your direction that you're creating, you need to compress the time frame.
I like to start with seeing if I can pull off a full year, if that's not possible because of the uncertainty that exists out within your industry, within the market, within your segment or whatever it may be. Compress that time frame down to six months. If that's still not possible, bring it down to a quarter and that would be my advice.
That the quarter is the smallest that you can go before it gets into feature land, because you cannot talk about a story within a quarter, meaning three months. Build something that is meaningful and deliver that to a customer in that timeframe. You might be able to do one thing against that story, but that's about it.
Craig: That or you've got some really proficient engineers that can build faster than you can ever imagine, and I want to hire them.
Rimas: You mean the 10X engineer.
Craig: Exactly. You've got ten of those. No, that makes a lot of sense. Bring it down smaller. But if you can't create a roadmap for at least three months of where you're headed, then you need to slow down in some way and step back and say, "What are we building? What's our direction, where are we headed?" Or talk to more people. Maybe you're not talking to enough people so you don't have a clear enough vision.
Rimas: To that point I would argue that if you are unclear, it's OK to take the time to figure that out, and say that you don't have a vision right now. You're probably not going to go to a customer and say, "I have no vision. Still buy my product."
Craig: I wouldn't do that. I would not go to a customer and say, "I have no idea where we're headed."
Rimas: You can still be open and say, "We're defining where we're taking the next stage of our product. Please, customer, give me input." You turn the discussion around from, "I don't know what the heck I'm doing because we don't know what that direction may be," to, "OK. Please provide us input." So you can misdirect the conversation into something valuable.
Craig: A perfectly valid answer to that is we're refactoring things, we're doing some cleanup to allow ourselves to move faster, which is also a very valid thing to have on a roadmap. It doesn't have to be, "We're adding this new fancy feature." It can be, "We're giving ourselves more ability to have more momentum in coming releases." You'll see us slow for this quarter, that quarter. Because we're paying down tech debt to be able to move faster.
Rimas: Or alternatively, you could talk about reacting to market forces here and saying, "Look. There's a whole bunch of stuff happening in this industry. We're taking a pause for a moment to reassess. We would like your input on this, because how is this affecting your daily work and what you're doing in your company? And what is the concern that's happening maybe at your executive level to say, 'We should assess this thing happening. What do we do?'"
Sometimes in some industries that happens, where everybody stops and says, "Holy crap there's this new thing. How do we react to this?" It's natural to say, "We're all figuring it out."
Craig: So, you create your roadmap, and you deliver on it perfectly every time. That's how it works?
Rimas: No way. Never deliver on it perfectly.
Craig: What happens when you fail? What happens when you miss? First off, what are the fun stories around where you've missed or you've seen people miss?
Rimas: Failure is a spectrum. It's always, "It depends."
Craig: You're at the peak of it, yes?
Rimas: Maybe, maybe not. When I think about failure in terms of product roadmaps it's really about delivering against what we had talked about at the onset of that roadmap and whether or not we met that.
So sometimes it's natural as a PM to create a theme for a product direction deck, or a roadmap and say, "OK. We're going to leverage our data to do something," and you get to the end of six months and you're scrambling to build a feature that barely meets that goal. To some extent you might feel like, "This was a failure, because we didn't put enough time and effort into this thing. But we delivered something and we put our money where our mouth is and said, 'We did this thing.'"
Craig: You checked that box.
Rimas: Yeah, you check the box, and sometimes that feels like a cop out but I would argue that's not a failure. You turned that around and said, "OK. We were still able to meet this from the market standpoint." So long as you learn something from it, and that's the big takeaway from the show, from anything that we talk about.
So long as you learn from that scenario and apply that learning to whatever the next scenario is that you run into, that's the important part.
When it comes to failures in roadmap, for me it's really the big misses along the thematic. Meaning that either something we didn't see or we didn't have the appropriate conversations, or this one is not really your own fault, but if there is a change in market dynamics and you don't pull your roadmaps back and reassess those based on whatever's happened in the market. That's a huge failure and you're doomed to fail as a company overall.
Craig: It makes a ton of sense if you don't stop and say, "OK. We're going to reassess this." It makes a ton of sense and you need be in front of it. For me the more painful ones are when I didn't have control of the roadmap, or someone didn't, and it went off road. I've had a CEO commit to something by a certain quarter, it wasn't delivered a year later. Not only did he miss by a quarter or two, that was a big miss and refocused the entire engineering team because the CEO promised it.
There's a lot of pieces that happen there. "Who owned the roadmap in the first place? how did you get in this situation?" Part of that is you need to explicitly be communicating your roadmap. Saying, "This is the roadmap." Not, "We're hoping to accomplish this feature, or build this new product and launch it." Very explicitly, "Here's what's happening. Here's what you can communicate."
That explicit form of roadmap internally needs to happen, otherwise it starts to leak out externally and create problems.
For me some of those misses are when someone communicates something they shouldn't and you miss it, it's a big break down in process which is the most unfortunate kind.
Rimas: But, it's the CEO. They're not beholden to anybody. They're not beholden to you as PM. Wait, are you a CEO of your company?
Craig: We may have covered that one before, and I do not want a CEO job. But they are to the customer, and it's your job to enable them with the right information. That's not necessarily a failing on the CEO, I'm not pointing the finger at the CEO saying, "I can't believe he went and said this." It's a failing in the entire process.
"Where did he get the idea that we would have this ready by then? Where did he communicate that? Did he come up with that on his own? Was this written down on a napkin, or was this from a slide? Where was the breakdown in overall process?" For me the roadmap needs to be something that is carefully controlled in those processes and who communicates it and how. Just as we're vague in times in it, we're also explicit in times in it. That needs to be maintained pretty well.
Rimas: I'm still not sure if a CEO will still abide by any process that you create. I'm still a little hesitant on that.
Craig: If he feels enough pain from missing things from customer, or pushback from an engineer, he'll react. It's a partnership, you need to have a frank and open conversation with, "Here's what we can commit to with these resources. You give us more resources, here's what we can do." That sort of thing.
Rimas: You're talking about it more from a portfolio management perspective, where as CEO if I'm an individual that has many different investments, and the investment by extension is somebody working on something to support a theme. If I deem it appropriate I can have somebody else work on, or kill something, in that regard to say, "Let's work on this thing because I deem it more appropriate."
Craig: But your roadmap is never one thing, it's a bunch of things.
Rimas: I hope so.
Craig: To say that the CEO can deem one thing more important, he needs to understand those tradeoffs. That's coming from product as the linchpin of all those spokes to coordinate.
Rimas: OK. We talked about strategic initiatives, we talked about those changes, we talked about the CEO. Gosh, wrangling that kind of executive in terms of what they can commit to out on the road.
Have you ever communicated a feature and find out it wasn't going to work or come to market? How do you deal with that conversation with customers, where you've said explicitly, "I'm going to do this thing," and it didn't materialize? How do you pull that back as part of a roadmap conversation?
Craig: It's hard. It's not a fun conversation to have. My best feedback is be frank. ”We tried this, we explored, we invested time and we're going to keep working towards this.”
It's a journey. A roadmap very much is a journey, and your larger companies absolutely understand this.
You want to be clear of, "Are you abandoning this? Is this outright change? Is this a thing you may get back to in the future? Is this a thing you're continuing to work towards, but completely missed the time frame?" You want to be very explicit. One, know what happened with it. Know your own result and then communicate that in no uncertain terms. You don't want to leave it open, "We didn't get to it this quarter."
Rimas: That sounds pretty nice to be able to have that conversation with a customer, to say, "It didn't work out, that's great." Have you ever lost any sales because of that? Or has anyone cancelled because you've canceled their subscription, or however you were paying for or pricing your product?
Craig: I'm thinking on this. I don't think I have. That honest conversation, especially when a customer gets a roadmap, this is a larger customer and they know it's a journey. In those situations I don't think I have. It's been more of when I didn't communicate that and they saw it not delivered, and they got frustrated over time so that a big part of the roadmap is that open to a conversation. But I'd be curious on your experiences.
Rimas: I've been in the same boat, in which I don't think I've had folks cancel, but it is one of these things where trust erodes over time if you continually miss. You may be able to flip back and forth, but that erosion of trust over time is the thing that gets it. I will say though, when I think about features and how I've missed, one of the things that I do to enforce the fact that this is a roadmap ahead of time with customers to make sure that they're not buying on future promises.
It's in my communications, which typically is a slide deck, to put the forward looking statement in there. After the first title slide just say, "Look. This is a roadmap. Things can change. There's market conditions, anything that you talk about past this point that is forward looking is a forward looking statement, and frankly we need to commit to it but we'll make every best effort to."
Craig: Being a startup, do you put the full safe harbor forward looking text on there? Or do you just put, "By the way. This is forward looking. Don't make any searches on it." I'm curious on stylistics there?
Rimas: The style applies to the industry that you're in and how seriously you want to be taken based upon what you're doing. If I was a startup maybe in fintech, I would put the whole safe harbor thing on there because you want to seem like you're taking their concerns seriously.
Craig: If you are in a publicly traded company, you are legally bound to in certain cases. Don't give the idea that you can go start saying what's coming without implications.
Rimas: For sure, and for the startups that maybe could be a little bit more loose in the developer tool space, if you're not selling to bigger enterprises or you haven't quite reached those sizes of customers yet. It may behoove you from a stylistic perspective to make it a little bit more fun, in the sense of, "This is a roadmap. This is all forward looking stuff. We're going to make every best effort to hit it, but know that we'll be honest with you with changes that happen."
Craig: "If you like it, here is a term sheet if you want to invest in us, if you're a startup." Cool. We've covered a lot. I'm curious as we start to wrap up, what are some of your favorite tactics for creating a good roadmap? What are tips, tricks, tools that you use commonly?
Rimas: It depends on the size of company that I'm in.
For the larger companies, typically there's something that you are largely
a part of that you aren't necessarily driving yourself. What
I mean by that is, usually the organizational alignment tools that help bring the organization together.
Because those then by extension help you create those thematic roadmaps, or however you need to do those roadmaps in general. Tools like OKR's objective key results that came out of Google, or I know Salesforce does its V2MOM which is essentially the same thing.
The goal again is to say, "OK. From the exec level and from the bottom, what are things that we think we should be working on?" And creating those objectives and saying how you're going to meet those objectives from a usually metrics driven approach, to say, "This is what we want to do to hit that objective."
It doesn't prescribe the solutions in the same way that we've been suggesting all of you don't put features in your roadmaps, it's more to say, "Here's an objective that we have. We want to get there by measuring these things to say that we've succeeded."
That leaves wide open the opportunity to build whatever necessary to meet those objectives. So that's one way to do it in much larger organizations, and much smaller ones where I think about the startups and places that I've been, we've tried to implement OKRs and it can get heavy weight at times. Sometimes I like doing things with, I think you feel the same way, with the effort/impact matrices.
Craig: That's one of my favorite approaches. We have a first impact matrix and we go through a process. We do this about every two to three months, looking at two to three months out, but we've done it for a full year period where the team comes around. I do this directly with the engineers.
We write on sticky notes the project or the feature, or whatever. Then we go around the room, and on a three by three matrix we rank it on effort versus impact. This is super rough granularity, I've written a few blog posts on this, once you start to put things on a grid next to each other you can see, "This project is high effort," meaning it's a three month project.
Or it's longer than that, it's got to be on there. If you've got something that's longer than a three month project, this goes as part of a roadmap. It's a pretty big project, pretty big deal, and things naturally shake out. It doesn't get down to the granularity of, "This thing takes four unit hours of work, and this thing five and a half," but it gives us a good picture directionally. It also highlights where low hanging fruit is, and that's my favorite thing.
When things bubble up to that top right corner, and we're very explicit to have that top right corner be low effort and high impact. Anything you have, something in there, it's like Christmas morning.
Rimas: Are there any other tools that you use besides that, or is that always your go to?
Craig: In planning out my feature work, that's my usual go to. We used it at Heroku, we use it at Citus. It's worked wonderfully. I've seen other teams adopt it without the same precision, it's very much in how you run the process. We also put OKRs in place for broader company coordination, but team local isolated to a specific area of the business. That's my favorite go to tool. How about yours?
Rimas: I have a wide spectrum of tools that I love using. Granted, the last maybe few years I've been more in the enterprise context, where I've been--
Craig: So, the number one is Gantt Charts?
Rimas: 100% Gantt Charts. You caught me. No, I'm kidding. I really enjoy the OKR process to be quite honest, thee challenge with that process though is being able to align that all the way up and down the org chart. Because you've got your senior execs or your CEO, or whoever is creating theirs. And then you have to be able to snap yours to that one, and it can take quite a long time to run that up and down the org chart.
But once you come out on the other side you're able to, just like you would with customers, and hopefully your OKRs are around customer impact. You're able to bucket those themes that come out of that, so that ultimately you can go and talk to the customer about those things. Usually for me that's where I get marketing involved, we start building the decks that sales and everyone else can use as a result.
Craig: I very much love OKRs. The part that I hate about them from a roadmap perspective is OKRs are meant to be ambitious. You're supposed to hit about 60% of them. If I'm missing 40%, what am I communicating to a customer? That you're going to get two thirds of this stuff and I don't know which two thirds? Which doesn't help from some of the thematic of, "Here's where we're going."
I do love them from a, "Let's be aggressive and let's try to enact big change," whereas V2MOM is usually more of. "Here's what we're absolutely going to hit, and then we try to stretch on certain things." But it's more of, we set our goals, and we hit our goal. So you're expected to hit more of the 100% typically, on the V2MOM approach.
Rimas: What does V2MOMs stand for?
Craig: Vision, Values, Methods, Obstacles and Measures. And I did have to Google that to confirm.
Rimas: OK, I hear you on the OKR front and I think you're right, in that "Yes. They're supposed to be ambitious, they're supposed to hit 60%." I know one of the things that I like to do, and I espoused my team that reports to me, is that you should stack rank your objectives. Like, "This is the first one that we definitely want to hit. Second one, third one down."
As you get to about the third or fourth one, those are the stretch ones that you're probably not going to spend time on if you can help it.
Craig: And think about how you share those, or do you share those externally? Those things. That definitely helps when you know you're committed to certain things, and others are more ambitious.
Rimas: 100%. When I think about other tools that I use day in day out with a roadmap planning, usually that involves taking the pulse of customers, honestly, and working through themes that come out of that and problems to be solved. That's the bucket that I use, when you go and talk to customers whether it be the advisory boards or just in general, as a PM you should really be out in the field as much as possible talking to whoever you can.
Identifying how their processes work and understanding, "OK. Here are the problems that you're having," and what you can do is mind map all of those or at least heatmap them to some extent, and say, "Here's some of the general themes that are all matching." From that you can pull together broader themes for the organization to then rally around.
Craig: I would 100% agree. If you're not at the core of it getting input directly from your customers, whether that's in person whether that's surveys or whatever process it is, and then consuming, digesting and analyzing that, then you're absolutely going to miss the mark.
Rimas: We've largely been focusing on the enterprise buy right now, where we've been talking about the one-on-one customer interactions, which typically is emblematic of the enterprise sale and sales teams in that regard. Does all of this apply to B2D companies, B2C-type companies?
Craig: Absolutely. It's a different scale, and there's different tools and processes on how you capture that input. If you're a very transactional sale and a customer isn't talking to a salesperson, it's very different how you want to capture that. Surveys are a very common tool and do work.
Email works. Email and ask a customer, "How are we doing? What could we be doing better?" Some of my favorite things are e-mailing a bunch of customers cold, open-ended questions. Not prescriptive, not specific. See where they put your product and think you should be spending time.
Rimas: OK great. I wanted to make sure that we cleared that up so that anything that we said here is still applicable to whether you're a B2B Company, B2C, B2D. Because this is Heavybit, this is about developers.
Whether you're in any one of those selling motions, the advice still applies. It's just the scale at which you have to operate with the audience that you're working with.
Craig: The method where you capture that input should match to your overall product and sales approach.
Craig: We talked a lot about how we capture input from customers, how we communicated this out, how we have some direction. We lead off with how much we hate roadmaps, but I'm starting to wonder how much we actually do.
Rimas: We just hate the word "roadmap" and the implication of it being about features that you have to deliver within a certain time frame, so if anything my take away from this is start calling it something else. Call it product direction, call it product engagement. I don't know, whatever else you want to call it, just not roadmap. To create the implication that this is about a path that we're going down together.
Craig: A journey, if you will.
Rimas: It's the journey because again, it's about delivering your product as a solution to organizations. How is this going to solve the pain that they have, day in and day out, with their work or with their lives? So, definitely. It's about the journey.