November 3, 2015
Ep. #6, Use Cases for Continuous Delivery
In this episode, Edith and Paul discuss the spectrum of Continuous Delivery, and where Continuous Delivery will and won't work in software d...
At DevGuild: Enterprise-Ready Products, speakers discussed how they build products that are sold into the enterprise. For our finale panel, we decided to flip the script and instead asked enterprises what they look for when purchasing a product.
In this session, CTOs Kimber Lockhart from One Medical, Ralph Gootee from PlanGrid, and Valentino Volonghi from AdRoll, discuss the frameworks they use to diligence technical products, the core enterprise features they need to satisfy their risk profiles, and the nuances of selling into their unique industries and businesses.
Grant Miller: Thank you all for coming to our last panel, to tell everyone about what hipster enterprises really want. To get started, the best thing to do is have you guys all do a quick intro. Talk a little about your responsibilities and how your organizations are set up.
Kimber Lockhart: Great. I can go ahead and get started. I’m Kimber Lockhart, I lead the technology team at One Medical. For those of you who are not familiar with One Medical, we are the nation’s largest independent primary care system. What that means is that we’re doctors offices, and we have about 80 doctors offices in eight cities across the US. Our focus is on getting rid of all of the bad parts of going to the doctor, and replacing it with a high quality convenience, high service experience that ultimately helps deliver better care and cut the rising costs of health care. Within the technology team at One Medical we have both our own product development, we have engineers and designers, product managers that are ultimately responsible for building the software that powers the practice and our patients’ engagement with our doctors. Then we also have an IT and security group who are in charge of working with fine folks you, to select and deploy solutions that help us to do that all faster.
Ralph Gootee: Hello everyone. My name is Ralph Gootee. I’m CTO and co-founder of PlanGrid. We write beautiful, easy to use software for the $17 trillion dollar construction industry. A lot of people are familiar with GitHub here I’m assuming, so construction has a lot of similar problems as software development. It needs version control. If you have version control that means there’s issues happening on these blueprints that are being versioned, and that means you have to collaborate around the issues and do compliance track documents. We do all document control and collaboration for the construction industry.
Valentino Volonghi: I’m Valentino Volonghi, I’m CTO at AdRoll which is a marketing firm. What we do is that we offer tools for running marketing campaigns, both to B2C companies and to B2B companies. We end up being both on the buyer side and on the seller side quite a lot, because we sell a lot to other companies. Everything that has been said today is particularly interesting. But the technology that sets us aside is the ability to listen in on all of the traffic that is happening on the web, each individual day. That’s over a 110-120 billion potential transactions each individual day, and answering to those within 100 milliseconds, wherever they’re happening around the world. Collecting all of this information obviously can lead to a lot of problems and scaling issues, and if we can do it particularly well, for every dollar that our customers are giving us and investing in our product that we can give them back $30 or more depending on how effective their campaigns and setup is, and how effective our product is of course.
Grant: Perfect. I’d like to have each of you, you’ve heard us talk today all about the various enterprise features. Reporting, RBAC, SSO, audit logs, etc. Are there features that are must haves when you’re going to be buying from a vendor? That you really have to have that? Or maybe it’s the 99% and you have to have that?
Valentino: I can start. There’s a few. I would say there’s a few things that we don’t need. One of them is RBAC, because then we saw the same things that Alex was saying. It kills the conversation between different team members.
It’s fine that people have roles, but it’s not possible to determine who’s going to come up with a good insight or something interesting that they have found in the data set, or in some reporting or similar arm.
You never know when you’re going to be about to board the plane, and you’re the only person that has the ability to access that particular flag or alarm or report, and all the people that are left out. There is nobody in the plane who can do it for you. Usually we don’t care too much about that aspect.
Grant: So, on that. Do you look for more core screened? At least admin and user? There’s some level of roles.
Valentino: Yeah, so very core and team level. Saying the engineering team as a whole can access all of these things, and the sales team can only access this particular area, because we don’t want the sales team making modifications to our reporting or alarms, or things done by mistake or of any sort.
Grant: So it’s like sales team, and then everybody else?
Valentino: No. It’s like sales, then support and then solutions engineering. Sales is obviously–
Grant: It sounds like you have some roles that you look for, just not maybe super granular. Is that right?
Valentino: That’s about it. But all of the extra granularity that usually comes, that maybe enterprises ask for, is a lot of maintenance cost to them to keep your product up to date with how the organization is changing. We have people sometimes that transition in different parts of the organization. One thing that happens, even with our own product, where we have some roles involved is that people scheme underneath the surface of processes and conversations. To get access to higher powered roles so that they can do more inside of our own product, from our own employees, and obviously when that’s found out it’s immediately squashed out. But it’s weird that this weird incentive is created. It usually means that you’re not providing the right set of tools to the people that need that functionality.
Grant: Ralph, what do you think?
Ralph: We have a centralized IT team that purchases things like Slack, but mostly it’s decentralized by the different departments. I’ve never evaluated somebody’s authorization mechanisms or access control, because different products have different needs. I would say for PlanGrid we build most governmental buildings, we’ve built a number of universities, we’ve built prisons, we’ve built schools. We build internationally. That means fedRAMPs, SOC 2 type 2 come to importance for us, as well as personal identifiable information.
How a vendor deals with that would be a roadblock for us, because we have to comply to other higher powers, and we need our vendors to comply as well. That’s the only thing that really bubbles up to some universal truth. Otherwise, it’s just the department and the product.
Grant: That’s interesting. This is fairly common where someone has stringent requirements, and those end up a passing down the chain. Where it’s about processors and sub processors, and you are continuously chained to those requirements. Are there any other features that you would look at and if a vendor came to you and said, “We can’t integrate it. It doesn’t have this.” We have some inline integrations that don’t need SSO. It’s hard to summarize it. SSO is nice to have. I would be surprised that you could roll out a product that’s meant to be used over 10 licenses without SSO. But sometimes products are only meant to be used by small teams. We also, I should mention this, we evaluate the business of the vendors we’re doing business with. If you just fundraised your first seed round and you’ve got three engineers, we’re going to analyze that and take that into your risk profile.
Grant: Kimber, how about features you must have?
Kimber: From our perspective our higher powers, if you will, that tie our hands in what vendors we can select largely fall into two categories. We have compliance and regulatory. We’re in the healthcare space, so patient data, keeping it safe and secure is one of our most important priorities. Because of that and in addition to that, we need to be compliant with HIPPA in many of our vendor selections. On the other side, we directly sell to enterprise. Employers will buy One Medical on behalf of their employee population to get them access to a timely and also high quality low cost care. Because of that, those employers have realized that we also build software and we have access to that patient data. They vet our software not unlike they would vet the software of a traditional software vendor. Because of that, for some employers we have a set of pass throughs that then we must ensure that our vendors adhere to.
When you’re thinking about coming in, and what’s negotiable and what’s not, it’s helpful to understand where my requirements are coming from. Because I can’t change HIPPA, and I can’t change the pass through, but there are some other things that I can be flexible on for the right solution.
The other framework I’ll share with you, especially around healthcare data, is that we look at our services in three categories. On the far extreme is data that will definitely contain protected health information. Here’s some use case or workflow where we know for sure it’ll be there. The other extreme, we have systems that we control what data goes into that system, therefore it is almost impossible for that system to have protected health information. Then we have this middle category which is quite extensive, which consists of the services that might possibly have protected health information. We’re not putting any in there, but we’re opening it up to internal users who might. Doctors need to communicate with each other and they will. As we think about services for each of those different categories, we have different responsibilities in how we vet both their adherence to HIPPA and also their security.
Grant: Kimber, it sounds like you might have a bit more of a centralized IT buying process than maybe Ralph does. Would you describe it like that?
Kimber: It depends. For things that are in the category of having protected health information, either likely or for sure, a lot of that needs to go through at least some kind of central process. That’s captained by our legal team, but our security team and compliance team play a role in that process. For things that are unlikely to, those can be more distributed decisions. We have both but, probably a little bit more to the centralized side than what Ralph was describing.
Grant: Valentino, how about you? Centralized, decentralized–?
Valentino: We’re a little bit of a mix. If it’s engineering and if it’s software that goes to directly help building the product, or that touches data that is relevant to the product, then the process is mainly siloed inside engineering. Usually legal and security help power around trying to vet what the what the product does. Although we don’t send out questionnaires, because I don’t think the security questionnaire is useful at all. Because oftentimes as I mentioned earlier it doesn’t match the data that is going to be delivered over to the service. So you’re asking me questions about sensitivity of the data, but you’re never going to see any sensitive data, for example. This is what I say from when we receive questionnaires from our customers. Since we don’t see it, it’s not very useful when we receive it, so we don’t think that sending it out is particularly useful.
Grant: There’s specific data that you just won’t send to vendors.
Data that can leave the company is lot less sensitive.
Grant: How do you decide with those? When you talk about the data that’s not going to leave. When you think about the tools that you might need to interact with that data, APIs, something else. How do you decide between a build versus buy on those tools?
Valentino: That’s a great question. It’s a complicated process for us. We are relatively small. Well, not anymore. It’s small, let’s say small or medium size. We have 170 people on the engineering team. So we have a bit of capacity to build things that could be considered close to core. If something is not core to our business there’s almost no point in building it. Obviously we evaluate the price and performance of that particular item, but probably we’re going to buy it. If something is in a gray area and the vendor is making us pay a lot more than what it would cost us to build it, then the immediate thing that I got to think of is: You want $250,000 a year for this. OK. That’s like an engineer full time. So I could put an engineer to work on that thing, but I don’t need to work on this thing for seven years. I can work on this for three months, build the 2% functionality that I care of and then move on from it. There’s an old saying that said that Microsoft Word is only used for about 2% of its functionality by any user, but collectively they use 100% of it. That’s about how I feel about the various different softwares that we buy. Of all the totality of the features that they have we may need five or six, and then once those are covered they might be so much simpler to build on our own. That’s where we’re like, “Your pricing is out of whack.”
Kimber: I have to tell you that my opportunity cost is so high in making that decision. If I think about what else the technical team, the engineers could be working on.
If it’s in the gray area it’s so hard to make the choice to build it when we could buy it.
Grant: Yes. Kimber wins.
Valentino: She certainly has a strong point. We don’t build a lot of stuff internally, but the volume that we operate at where we have 100 billion data points per day. Sometimes during some days we have reached the 7 trillion loglines logged each individual day, and need to process all of those. You signed up for a service and their pricing incentive is not lined up with yours, then at that point you figure, “Maybe my opportunity cost, maybe I could have evaluated that a little differently.”
Grant: That’s great. Ralph, Valentino mentioned vendor security questionnaires. When we talked, you mentioned that you do send them.
Grant: But how do you perceive them? I think that one centralized part of our IT system is we do make everyone go through an IT security audit, and also a review of compliance, and whatever contracts people are sending us. I’ll say most vendors send us really bad contracts initially that have crazy shit written in there. They’re expecting no one to ever read it. We do read through the contracts and we do check legal and everything, and I liked the way Kimber described all the different stages. We have a similar thing. There’s certain data that for sure is going to contain sensitive information., and for sure we send out a security questionnaire. Anything in line, we do a security questionnaire on. Then for other products, say I don’t know, a design product or a design tool. Maybe we don’t send the security questionnaire for that. We do make some judgments there depending on our feeling towards the risk of the data.
Because of compliance, every piece of data we have has some severity next to it. If any of the vendor touches that piece of critical data, then they must go through a security questionnaire. When we send out the security questionnaire we tailor it a little bit. I do admit receiving a bunch of security questionnaires, I was joking with some of the other panelists back there, they’re asking us about our physical security of our servers. And it’s like, “Go to talk to Amazon about that.” But we try to send out one tailored for the issue at hand, mostly because of compliance reasons. We are expected to send one out and if we ever didn’t send one out, we could audit ourselves and fill compliance ourselves. Which then would have a knock on effect for our customers.
Grant: You mentioned in our earlier conversation that it was really about covering your ass when it comes to your vendors and making sure that– Because this might not be a totally loved opinion, but those vendor security questionnaires people fill out with half truths. I’ve seen them fill it out and fill them out and you’re like, “That’s true most of the time.” That’s a challenge. But the thing that you pointed out is that you’re getting this commitment from your vendor that says they do this, so it checks the box for you and you can go back to them if something would happen.
Ralph: I will say penetration tests, those are nice.
If you have the time and the need and the want to put a vendor through penetration tests, if they haven’t done it yet, that almost always brings good results.
Kimber: That’s a point I wanted to bring up around penetration tests. My perspective has shifted as I moved from one side of the table to the other. As someone selling software, it’s easy to be very resistant to buyers doing penetration tests or asking you for the results of your pen test or your audit. I get that. On the other side though, I can commit that we’re not using it as a binary mechanism to qualify or disqualify a vendor. Our security team genuinely wants to work with vendors to help them improve their security and to understand where things are today. So yes, if you do terribly in our penetration test, then that could be disqualifying and likely will be. On the other hand if you’re in the same territory that most people are, and that’s that you’re giving your best to security yet we find a few things, there’s a process to work with you to fix any security issues that we find so that we can feel confident about using your software system.
Valentino: I have to say, I completely agree with that. It feels like the transactions and the conversations with vendors, or even when we go to sell something, can sometimes tend to be a little bit too transactional. I have to go through this because this is the process that I was told that I need to follow, without necessarily understanding the backdrop of everything that is happening around both. If we’re even at the table right now it means we want to work together and we’re trying to establish a trust relationship between us. We don’t know how we work, we’re going to try to establish things like SLA or security questionnaires. “This is the way that we like to be escalated to. This is how you’re supposed to communicate with us.” Ultimately at the end of the day it really is about how we’re trying to achieve this particular piece of functionality. I don’t need you to tell me that you’re up 99.9% of the time, because most of the time as was mentioned earlier, two or three panels ago, “Go ask Amazon.”
They don’t even give me an SLA, I’m not exactly sure I can guarantee that too much, but what I need to know is that you’re going to care when my business is going to be impacted by the fact that whatever you’re building is going to be not available. Yet on the security front, I don’t really care personally and AdRoll doesn’t necessarily care that you are going through this questionnaire, per say. We have some for compliance but it’s relatively short. What we care about is that you do have a person that is there that deals with security, that you do know something about security. For certain types of data we’re going to want to sit down with your team and go through your infrastructure. I want to know how you’re going to deal with disaster, if there’s going to be a disaster, because if we put your servers in front of our customers websites and you have a spike of latency for three hours each day, our customers are going to leave us and we’re going to be stuck with you instead because we maybe signed a contract even. So the deal is let’s get to know each other as two businesses, and figure out how we do things and what we value, what are our core assumptions that we care for and then we can move on from that.
Grant: Sounds like a lot of work.
Valentino: We only do– How many contracts do we sign in the lifetime of a company? It might be like 12 or so years that we’ve been alive, it may be a sign that 100–
Grant: I’ve got to sign more than that. I’ve got to sell some deals.
Valentino: From buyers, I mean.
Grant: I know. But you’re buying, I got to sell faster than that. That’s my point.
Kimber: It’s important to realize that when we work with smaller companies we have an awareness of what that means. A desire to work together, and a desire not to be so overly arduous with our demands that it causes problems for the vendor. We hope that our buyers will extend us that same courtesy.
Grant: On that front it’s interesting. Just a quick survey. How many applications do you think are running inside of your organizations, from both a line of business and centralized total? Like, number of apps that are running. I’ll let someone take a stab.
Ralph: North of three hundred.
Kimber: I think I’m in the low hundreds.
Grant: Low hundreds?
Valentino: Other vendors, all vendors included?
Grant: All vendors, yeah.
Valentino: I don’t know. Not hundreds, but maybe 50-ish, 60-ish.
Grant: The stat from NetScope or something is that the average enterprise is maybe a little bit bigger than you guys are, but would have 1,053 applications running because of shadow IT primarily. People signing up for random things online. What do you guys do to manage that? How do you take control of that? Is there any tooling or thing that you do? What’s that process look like?
Ralph: We can try to control on the front end. That gives you a little bit of controls. You go through and you start saying, “OK. To buy a piece of software you got to talk to your manager.” Then it’s like, “Wait. That didn’t work out. We still have shadow IT teams.” Then you’ve got to talk to your director, so you get the front facing controls and then on the other side we catch it during financial audits. It’s not only financial incentives that help us control those, we catch them during audits. We run through every vendor, “What the hell is this?” We go track it down, talk to five people before we find the root of who purchased this thing and figure out why they purchased it. But it’s a lot of physical effort.
Grant: Anything for free tools? Things that people just sign up for free?
Ralph: Not from me.
Valentino: If it’s free, people can sign up for it as long as it doesn’t deal with our own data. Our Google Apps account is blocked to third party applications, there’s only a white list for apps. Even if they’re free you can’t add them to your Google Apps account. So, all of that and the app, and the product, and all of the rest of the data usually doesn’t have a lot of free stuff.
Computers are fleet managed. If we know that we don’t want you to install something, we’re going to blacklist it so there’s no way you can install it.
Grant: So you blacklist on laptops, but you white list on Google’s G-suite?
Valentino: It’s a balance to try. We have network monitoring inside the office, so if there is anything that is happening on any laptop it quickly gets isolated and it can be removed. But it’s the balance between letting people do their job and be productive with it, without adding too much bureaucracy. There is a risk profile to everything, there is a tradeoff. If I’m adding something to 500 people it needs to be really valuable, because if I’m diminishing their productivity by even 1% it’s diminishing them by a bunch. It needs to be, “This is something that must be done from the top down.”
Kimber: While people are our greatest vulnerability, they’re also our first line of defense. We do spend a lot of time educating our entire employee population about why these choices are important, and trying to help them make better choices. Because if we get too block heavy and we just start blocking a bunch of potentially useful free or paid services, people will find something else. They’ll go around every time. People are smart and they want to get their job done. For us it’s a matter of providing a set of tools that are actually really easy to use that enables people to get their job done, that are the official set. Then helping folks to understand the risk level in interacting with tools that are outside of that set.
Grant: That’s interesting. Do you prefer– When you think about adding new applications into your things that you’re officially using, is it better for those to be sold in top down where they’re talking to your team and you help spread it out? Or where they adopt, there’s this broad adoption and then it needs to get rolled up? Do you have a preference in how those come in? Does one make it easier?
Kimber: I came from Box. I was an engineering leader at Box before I came to One Medical. Amusingly right now I have a bit of a cloud content vendor sprawl going on within One Medical. The reality is that centrally we would slow people down if we went through a central process for every major class of application, and I don’t want to do that. At the same time we do hit a stage with various types of vendors when we have enough sprawl within the organization that we need to help provide a tool that just works. For those kinds of vendors it’s helpful for them to support SSO, so that I can make it really easy for people to pick the one that we already chose, as opposed to going off and picking one of the others.
Grant: What’s the best way for someone to get in your organization?
Ralph: Have a good product. Good products, normally top down doesn’t normally work for us very much. It’s almost always came bottom up. I can remember the time that Slack came out and we had HipChat. I was fine with HipChat, but for some reason our engineers weren’t. They revolted against HipChat and we had to go with Slack. I’m like, “This is all IRC.”
Grant: Same thing.
Ralph: But still, I really got pressured. I didn’t mind though, I didn’t really care. I just want tools. We’re not that big of an engineering team, we’re like 150 or so. Around y’all’s size. I would hate to bog people down with trying to pick everything, and I do feel a regret for letting Quip in. I beat myself up about that every day, because I let Quip in and we still use Google Docs. I never know where to find anything. I try to find a written document and it’s like, “Is it in Quip? I don’t really like Quip. Is it in Docs? Other people don’t like Docs.” I don’t know, maybe I should have had more control at that moment.
Grant: I hadn’t even thought about that, but we suffered the same problem with a team of 20. Shutting down people using random things like that, and it’s the advantage of having knowledge centralized. It’s not just the security implications or the sprawl implications. It’s like, “How do you centralize that information?” That’s another great point for your document sprawl, everything else. Where do you find these things?
Ralph: We’ve got document sprawl too. Do you have document sprawl? Did you figure it out?
Valentino: We used to have in the marketing department, for a little period of time the marketing department had five different email providers. Not really sure what they were doing with five. We were spending more in email providers than in sending the emails.
Grant: Like the content emails?
Valentino: Yeah. On the other hand, what I do with all of my engineering teams every month, I sit down with each team in specific. They know that they have a deck that they need to present to me that talks about their service level agreements, their service level indicators and objectives. They talk about their current infrastructure and the bugs, postmortems, the production issues, escalations. And they talk about what tools they use, their budgets and their forecasts. My team’s own their forecast. If they miss it they’re going to hear an earful from me about all of that. Especially if it’s lack of preparation, or stuff like that. But generally speaking that’s how those things get under control.
Grant: Importance in reporting there.
Understanding that your application is going to get used and you need to be able to report and create the next high level view, the executive level report, to service up for someone like you CTOs to look at and evaluate and know what’s going on.
Valentino: One thing that I do not like is when you start to use an application and– What AdRoll does usually to avoid some of these problems is that if an application is important enough then we may mandate, “Everybody needs to use it within a short period of time.” The downside of that is that the application might be useful, everybody in the company might love it, but it might not have any meta reporting about how the application is getting used. Usually meta reporting is quite complex because again, this was mentioned earlier maybe two panels ago about Slack, like the PagerDuty CTO. I forgot the name.
Valentino: About how Slack is refunding users that are not getting used. That is a way to remove the need potentially, to an extent, meta reporting. But there are other applications in which the user is tied to price, which is normally the case, but you can’t always control your use. If a team is starting to talk about any logging service out there, the centralized logging like SIEM, or I don’t know. I don’t want to name any particular company. Those are like, if a team needs to do a backup or it needs to do debugging of something in production, it might log a fantastic amount of data and we pay a ton of it. But if this is done without my knowledge, or the team is doing it because it’s more venomous, and they end up spending or running out of the entire budget then you don’t have reporting for it. How am I going to figure it out? Splunk does have the reporting for their own things, but not every product does.
Grant: I’m curious too, for companies your size, if you think about open source and I’m sure you use a bunch of open source stuff. Do you work with the commercial open source vendors behind any of those projects? Is that something any of you guys do?
Valentino: Yes. We do.
Ralph: Yeah, we’ve done that.
Kimber: At times. Especially if my only other choice is to deploy and maintain it myself.
Ralph: You normally don’t have a choice.
Valentino: We’ve done bug bounties for languages, because the language has the garbage collection issue, and they’re garbage collectors. So we gave $1,000 or something to someone to fix it. Or we become members of the industrial panel of the language, in this case it’s Erlang. We’re a member of the Erlang Industry Group, or something I think it’s called.
Grant: A very prolific group.
Valentino: Yeah. Erlang is a strange language.
Grant: Cool. The final question is just advice for vendors. You’ve won, you’ve all been vendors, and you’ve made that transition from being small startups to now you’re an enterprise. You’ve sold into companies and you’ve seen that, and you also now are buying software. What’s your best piece of advice? High pressure.
Be clear about what you do, and what you don’t do.
We’ve been working with a vendor here recently who throughout most of the sales process alluded that they would have the compliance standards that we need, and we later found out that only certain use cases were supported around these compliance standards to the point that it’s not useful. We can’t use the product. That’s put a bit of a bad taste in our mouth in terms of being able to work with this vendor in the future, even as they do mature their compliance capabilities. It’s a lot of our time to buy, as well as a lot of your time to sell. If we’re not upfront about what the product actually does that can be can be a big problem.
Ralph: I’m thinking through all of the recent negotiations I’ve been through, of a success and a failure. From the successful side we did have one vendor, similar to your point, who just said “I will not change my contractual language.” And we’re like, “Are you sure about that? You really don’t want to change?” We pushed them off, we made them wait, and they really were not going to change their contractual language. And that’s fine. We still ended up going with them because the product fell into a certain risk profile, and the contractual language was just stupid enough that we felt that we could get by with it. But it was helpful that they were upfront from the beginning. They weren’t trying to drag us on, it was us who dragged it on a little bit while we wanted to see if they would change it eventually.
The other thing I would mention I’ve seen, this is on more on the failure side. I’ve seen, it’s really challenging when you’re– I’m assuming most people here are selling to people like us, to technology people. We sell to construction so this advice wouldn’t apply there. I’ve had a challenge, like early stage companies that hire their first salespeople. It’s so hard when you hire someone carrying a bag around to work with a quota salesperson, even if you didn’t give them a quota. Their first interaction, if they’re selling to other engineers, I can’t believe I’ve seen this situation, but I’ve seen even if it’s a technical solutions person they’re trying to sell to our frontline engineering team. They’re doing okay, the product is good and the foundation of the company is good, but this frontline salesperson and this engineer, they hate each other.
Grant: Oil and water.
Ralph: Yes. Be cautious with the first salespeople you hire, be cautious with the profile. There’s a lot of great salespeople out there that can have million dollar quotas at some company, and failures. I would say in general, the founders have always been the best salespeople I’ve ever met. So even if you have a sales team, if you’re a founder out there selling it, good for you. That’s how you’re going to learn. If you have a sales team, check in on the customers.
Valentino: Both of these are great points. I would reconnect it to a question you asked earlier about, “What’s the best way to enter into a company?” A lot of our clients that want to target CTOs or C-level executives and what not. Not only is that not always the best way to go, practically speaking, as was answered already. But if you are determined to go down that path, make sure that you know exactly which person you’re talking to at that point in time. Because even if you’re talking to CTOs like us, some CTOs are a lot more technical than others. Some are more big thinkers, some are more customer facing CTOs.
There’s a lot of different varieties of CTOs and when you get in the room and you have a single deck and the person in front is not as senior or capable of interfacing with another CTO, but it’s maybe a brand new salesperson that just started at the company and doesn’t know the product very well, it’s going to be a train wreck. There’s no other way to put it.
The first three questions I ask are going to be met with, “I don’t really know.” It’s like, “OK. We have another 45 minutes.”
Grant: “This is going to be awkward.”
Kimber: Also, I get 40 vendor emails a day. Don’t waste your time on me. Find somebody who’s actually making the decision. I’m there to help make sure my teams are making the right decisions, but I’m not there to force vendors or products down their throat.
Grant: You’re all incredible CTOs.