July 10, 2020
Ep. #22, Designing for Observability with Jimmy Bogard of Headspring
In episode 22 of O11ycast, Liz and Charity speak with Jimmy Bogard of Headspring. They discuss maintaining balance for on-call engineers, wh...
Grant Miller: To get things kicked off, maybe give everyone a little bit of context around your work at MuleSoft.
Uri Sarid: Sure. I'll preface it with a tiny thing about my own personal history. I've been at MuleSoft for five and a half years, before that I ran the NOOK Cloud for Barnes & Noble. That was my only other exposure to a public company.
Before that I did six different startups, some I started and some I joined, and many of them were in the enterprise space. Before that, I did many years as a theoretical physicist, which I to think prepared me for the world of integration. Because it is craziness. At MuleSoft the driving problem is that software needs to talk to each other. That's the world of integration, and we'll talk a lot more about that.
The thing that we wanted to solve when I joined is, "How do we not become the glue that sticks all these pieces together and freezes enterprises in place, and doesn't allow them to evolve?"
Because what you do when you do these big integration projects is you have to then tear them all apart later when things change.
So, "How do we both make applications connect to each other, but do it in such a way that you can change it later?" That was the big problem to solve.
Grant: Interesting. For everyone else's information, too, let's dive into a little bit of why enterprise focus on integrations is such a key factor when they're evaluating different applications.
Uri: It gets to a little bit of what Eugenio was saying. The heterogeneity and the complexity that you have inside of enterprises is huge, but one way that it reflects it is that there are numerous systems.
Instead of thinking about, "What is it that I should integrate with?" You have to get into this mode of, "How can I become integrable? How can I set myself up for the point where I'm going to need to talk to 15 different systems that I've never heard of before, because some of them were written before I was born and others have come on the scene?" That's the landscape of enterprise integrations.
Grant: I know you prepared some slides because there's this concept of integrability, which is a word--
Uri: It is a thing--
Grant: And how that compares to integrations. So, you brought some slides to help describe that process.
Uri: You have to put yourself a little bit in the mind of the customer. The customer is somebody inside of the enterprise. Unless you are indeed selling to a five person team that could be at an enterprise or not, mostly you're selling to somebody who's deep inside the enterprise and lives somewhere under the CIO or the CTO. It's important to put yourself in the head of that person.
That person is faced with, for example just inside of marketing and just inside of the SEO part of marketing there's something like-- This is a couple of years old, but 100 different companies pitching different things. The picture behind that, what that actually shows you is that's part of thirty five hundred different companies that this group is tracking just for marketing automation.
You would think, "That's not a big deal. I'll take the top 10 and those are the ones that I'll evolve." But it doesn't work that way. These are hyper specialized.
They're really good at one particular thing, and if you don't take that approach of adopting lots and lots of these things, then you're going to be behind them in each one of these aspects.
In that hyper complicated world, what do you do? You have to be able, as a CIO, you have to be able to get good at piecing these things together. That's the world in which you're in.
We'll see what the next slide does, but the idea is that the world that you're facing when you're coming into this world is not this nice puzzle block of everything fits together, and you have to slot it in the right place. It's a whole bunch of systems that were built over time with all sorts of integrations between them.
Then there's all this shadow IT and mobile in the SaaS world that puts things outside of the enterprise control, so from a CIO's perspective, it's like, "My God. I can hardly keep up with what I'm asked to do today. How can I possibly adopt your new software that you're selling me?" That's the mindset that that they're coming in with.
That's the challenge that they have to solve, and their career depends on it. At some point we saw that the average tenure of a CIO was two years, because they can't solve this, and they always blame the previous person. So, how do you get ready for that landscape? You have to think of where that landscape came from.
The build that this was going to do is, the way that this happens is incrementally. It's not as though somebody meant to create a big spaghetti block, they just built it project by project. Integrated this in, wrote some custom code, integrated this in, bought an aside with 5,000 custom developers, did a bunch of stuff.
You end up with this thing over time. The old boiling the frog thing. It just happens under your watch. What we wanted to figure out is, "Is there a different approach?" Then we would build software for it that allowed you to not get into this state.
The idea here, again it's not building out, but what you do is you don't just connect with custom code all your applications. You expose an API, and then you connect to that API as if you're building a new application. So that thing that connects them is itself an application.
It's a tiny one. It's what we would all call "glue code." But that thing is now productized. Now you've got three different pieces that glue things together, but they should be regarded as applications. They have an SDLC and they're well managed, they're exposed. It turns out that because they sit on top of APIs, when you go and change those things it's easy to do, because those APIs are really products.
It's a different mentality but it brings to the forefront that integration is a problem of productizing APIs, and then building on top of them.
That creates new structures and you build on top of those, and it's that motion that gets rid of some of that complexity. It's those levels of abstraction. I'll skip past this, and you end up with something that we call an application network. Think about your computers at home. You don't sit there and say, "What bits should I be sending? What wires should I buy to send them?"
No. It's CAT 5 or it's Wi-Fi and it's over standard protocols, and everything is plug and play. How do you get to that state for applications? That's the world of integration.
Grant: How should a application vendor think about entering that world? This is the perspective of the enterprise. If I'm a vendor who's building one of these small pieces, what should I be focused on?
Uri: It's a good question, because if you start to think about your problem, "What other software do I need to integrate with in order to have a coherent offering?" Which is always the case. There's lots of software and it all changes all the time, so you end up with a never winning race of integrating with everything and every customer will want you to do it a little bit differently.
The only approach that I've seen works is figure out, "What is it that you do? What is your value proposition? What thing do you actually do?" And model that in the cleanest way possible. Stripe has been a great example of that. If you want to do payments, this is the way that you want to describe payments. In the simplest possible way and productize that API.
Once you've done that, if you want, you can then build pre-built integrations through that simple model. But that model is unlikely to be changing all over the map because it's a clean way to expose payments. It's likely to not have to change very quickly.
The integrations can change quickly, but the integrations attached to something that is very easy to understand. It's easy to build and change them later. That's what by integrability, is expose yourself as this clean simple model. It's then easy for people to put it together again.
Grant: You focus on API creation and integrability, but should you build any specific integrations into a system of record, whatever that might be? Some people say Salesforce is the system of record for a lot of these applications. Should you build specific? Or should you just have an integration or integrable API, or should you build into an integration platform? What's the best approach?
Uri: Obviously you should use an integration platform.
Grant: Sure. That makes sense.
Uri: The key is that you should separate out the two problems. One is, "Have I made myself integrable? Have I exposed? Have I thought through what's my value prop?" And so on. "How do I expose that?" That stuff. Now you go to market and you say, "What friction are my customers going to encounter?"
And if they're always going to be integrating with two or three things, then I want to also provide those out of the box. Sitting on top of my APIs, not pre-built in, not back wires. Nothing else. I sit on top of that. Ideally you would use a platform for doing that, because why should you have to reinvent that wheel?
Know that when your customers go there they might be perfectly happy with those integrations, or they might say, "I've got an integration platform that I'm going hook directly into your APIs and do it the way that makes sense for me." And that should be fine.
Grant: If you are working with an integration platform and doing that in a more abstracted way, how do you avoid that least common denominator problem? How do you get that special feature that one API only has? How do you build out around that?
Uri: For those who haven't been in the integration space, there's an assumption that what an integration platform does is it has pre-built connectors for everything. Therefore, if you're not going to build a connector for every software system, it could only do the lowest common denominator.
That's not how we look at the world and I'm thinking that other integration platforms are not looking at that anymore, but rather you look at the API specification that is exposed. That will have all the richness of that company because there's no other way to interact with that company other than through their API.
Their API has all of their capabilities. You read that API spec and you make it simple for people to connect to all the rich capabilities of that in a way that doesn't involve a lot of custom code. It's more that the integration platform should fit itself to any software that it finds out there, versus that it has to lower the bar for all the other software so you can talk to it.
Grant: How do you think--? This is a fairly technical audience. Many are re-architected or have re-architected their applications to use GraphQL as part of their API. How do you think, how does that change this landscape? How do you think that fits in the future?
Uri: GraphQL has lots of answers for that, because it's an interesting topic, but in particular it's an illustration of what we talked about before in terms of integrability. The driver for GraphQL, if you'd look at its origins and Facebook, it's like "I'm a mobile developer. I need a bunch of information."
It comes from a bunch of logical tables, but I only need this particular combination of that information. I don't want all the REST of that stuff for any number of reasons. I don't know that stuff in advance, and it's going to change twice a day, so how can you let me query for exactly the information I want and you on the server take care of serving that up for me?" The GraphQL solution for that is that you have to write these resolvers on the server in order to serve the right information, and the GraphQL server will put it all together.
Which means that you have to anticipate in advance and code for all of the things that the user might want. We're answering a valid request of an application, to go and decide what information it wants with a siloed approach of, "I'm going to have to anticipate all that and write it all myself." We need to answer that, but we need to answer that in such a way that it scales.
In other words, that it goes across lots of APIs. What we've been working towards is, how can we take the GraphQL query and an extended GraphQL query, and automatically federate it across a whole bunch of RESTful APIs. Because REST has solved a lot of the individual problems.
But what the user is asking for is, "Please aggregate them for me." If we can do that in an automated way we get rid of yet more glue code that tries to anticipate that and put all of that together, and we let developers expose their functionalities as RESTful APIs, and we let consumers consume that through GraphQL and we can get both the best of both worlds.
Grant: An important point here is talking about MuleSoft and what you guys are doing.
It's not just the product. It's also that you have open-source specs and other things that you've been developing in order to help facilitate this move for the world.
It's not just a buy into the platform, it's like "Try to find the common open source way to do this."
Uri: It's not just open source, it's also about open protocols. Eugenio mentioned if there's a standard way to do things, please let's all fall behind that standard, and if there isn't a standard maybe we should create some standards so that we don't have to be recreating this stuff.
As much as possible we've tried to create standards, promote standards, open source some of the pieces that we think are important for the ecosystem to have so it's easier to connect to each other. I remember at the beginning people were saying, 'If you make integration so easy you guys are going to be out of a job." I'm like, "Don't worry. There is so much work out there for integrating stuff that we'd much rather write on top of that than have to write a lot of glue code.
Grant: One of the concepts, one of the foundational concepts of "enterprise ready" is this concept of product assortment. Which is basically, "How do you offer different features and feature gate in order to charge enterprise customers more money?" To segregate who needs to be paying more and who needs different features, and keep simple versions for your startup customers. Do you think that API access is a feature that these vendors should think about using as a feature to differentiate different pricing plans?
Uri: API access should be-- The access itself is not an add on. It should be in fact, favored. You get a tremendous amount of adoption and pull and developer love by telling developers, "Use my capabilities. You may or may not choose to use my UI." Obviously, it depends on your business. But if you're exposing core capabilities make them as consumable as possible. These days consumption means APIs.
If the end result is that the APIs are hard, you have to pay for them, and so on. It will actually limit your adoption more than anything. I'm not saying don't monetize the APIs, in the sense of, if this is the way that you're consuming your service of course you should monetize them. But it shouldn't be as a friction point to generate more money, it should be an adoption point to generate more demand.
Grant: I agree. But at the same time, I also use what's going on in the industry. Like Salesforce, you have the $25 a month plan and then the $75 month plan. The $25 a month plan doesn't have access to the API, the $75 a month plan per user does as an add-on and then you get it at the higher level. It seems maybe it works at Salesforce--?
Uri: We just got acquired. Give us some time.
Grant: OK. So maybe someday, that API access--
Uri: I'm not saying anything.
Grant: OK. I get it. Cool. So, another area where these overlap and intertwine is in security. Arianna is going to dive into security more broadly in a bit, but I'd love to hear your thoughts about how to think about security when it comes to APIs and integrations.
Uri: Absolutely. I love that we had the talk on Auth0 before this. Security is one of those things that, I didn't have a chance to put it up in these slides but it's one of those things that you walk into an enterprise and if you're not ready in the security part, get ready for a grueling experience the higher up you go. Again, it may not matter for the first two or three users that come in. But as soon as this gets adoption and you're all eager to jump in and so on, the security is going to be a big deal.
The landscape on security today is such that you probably don't even know when you're getting breached. It's already happening and you will find out about it at the worst possible time, when the bandits are not only across the gate but they've taken everything and they're somewhere away, and they're now offering it for sale.
It's the kind of thing that you have to think about at the design phase.
The API approach gives you the structure to make something secure, in other words it's more securable. That's because you've declared your points of integration. You can't just hook into your database and have backends and so on. These are products and the products have security built into them.
The only way into your stuff is through that product, and it's monitorable and it's well-defined, there's no secrets out there. If you try to hide that I've got a lot of IoT stuff at home and it always says, "We don't publish the protocols to talk to it." The only people who know it are the hackers. The security people don't know. Your company doesn't know. So, publish that stuff. Make it explicit and then secure it layer by layer, and we can talk more about how.
Grant: Your perspective there is an API-driven company will be more security oriented, because if they're API first and their interfaces are going to be secured by default, versus making calls in database.
Uri: By far. The integrations will happen. If you don't productize those integrations, that's where they'll come in.
Grant: In the same vein of other enterprise-ready features, I'm sure MuleSoft, you're also someone who offers enterprise software. It's not like you just do integrations. So within your product itself, are there any specific enterprise-ready-type features that you think have been super helpful, or most interesting to your enterprise customers?
Uri: There's two kinds. One is there's a list of things you need to do, you guys have them on your on your site. Here was a great example. There are a number of other things that if you're already doing them, it's much easier to be absorbed into that enterprise.
"Are you doing logging in such a way that they can aggregate into a dashboard? Are you producing audit logs that they can use for auditing?" Depending on the industry, there's a number of things that you want to have checked off.
The other part is the structure and the deployment model, and the way you are anticipating where the future is. For example if you have software that's running in the cloud, this may or may not be as relevant but if you have software that runs on prem, and much of our software runs on prem as well as in the cloud.
How ready are you to be deployed into a Kubernetes cluster? That may not be relevant for 80% of your customers, but that will change in the next couple of years. Are you already getting ready for that? You might say, "Why should I do it now?" Because the customers that have the deepest pockets are the ones that are also already playing with that stuff, so you want to tell them, "You're not going to make it a mistake for me." These structural things that you're already ready for where things are going, like Open ID Connect for example.
Then there are the hygienic pieces that you want to be doing. When I joined, I built our first infosec group and we became PCI compliant pretty quickly, and that's a little bit odd. Because we don't store credit card information, why should we do that? But that was a mechanism to get us ready in terms of security posture for when we expand, so that we've already got all the checks and controls necessary that enterprise will look at it.
Sure enough, when we got bigger and they look at the PCI and other ISO 27001 on that we already had, they were like "The InfoSec conversation goes from a big barrier at the time you want to sell, to something where the infosec guys are rooting for you. 'I want those guys because they already passed all those requirements.'"
Grant: That's amazing advice. I'm getting the signal, we've only got a little more time. Is there any last advice you want to give to the audience? In terms of their applications are out there trying to get adopted in the enterprise. How should they be thinking about that?
One of the things that we lose sight of sometimes as technologists is there's a person on the other side making this decision. Understand that person.
Walk in being able to talk to that person. Again, like Eugenio was saying, if you don't know the terminology and if you don't know the kinds of things, if you don't know a little bit of older stuff it's hard to have to build a lot of face in that conversation. Understand a little bit the world of your customer, understand where they've been and then connect it to their concerns.
Are they thinking about agility or are they thinking about security right now? Is there some particular functionality that they're worried about? What is their pain? If you solve one of those pains, you're much more likely to get some urgency.
Otherwise you can have great conversations for two years and no actual buying happens. Figure out what that point of urgency is in understanding your customer and then try to leverage into that, and be ready to pivot. God knows we've pivoted as a company and some of those pivots turned out to be pretty good.
Grant: Uri, thank you so much. This was amazing.