1. Library
  2. Podcasts
  3. Don't Make Me Code
  4. Ep. #7, You’re The Developer Of Your Developer Experience
Don't Make Me Code
43 MIN

Ep. #7, You’re The Developer Of Your Developer Experience

light mode
about the episode

In this episode, David and Steve are joined by Matt Harris, Co-founder of Sendwithus. The three discuss the challenges of providing the same great developer experience across multiple different languages, how important DX is to building developer traction, and finally they cover what to look for in potential hires when building your developer experience team.

This episode features Matt Harris, the Co Founder of Sendwithus, you can find him on Twitter right here.

DX Before It Had a Name
What DX Covers Now
How DX Differs From UX
What Made the Heroku DX Team Work
Internal Developer Experience
Hiring for Empathy
Keeping Every Touch Point in Sync
Beyond Stripe-Level Docs

transcript

Matt Harris: It was a really big challenge for us on how to match that developer experience across different languages and different environments.

David Dollar: Trying to get the experience right for a Rails developer when we're all Rails developers is not that hard, but trying to get the experience right for a Python developer, there's just all sorts of things that you're just not familiar with.

Matt: The ask to install Pinterest is super low cost to the user. For a developer, you're asking, stop your productive work writing code, do something in the command line, install something on your computer, then start making API calls and doing things, and it's totally different.

You have to justify your existence by saving time.

David: If somebody who's applying for a job, I'm not even necessarily going to look at the code. I'm looking at their READMEs. I'm looking at how they respond to users and issues. And there's a certain level of empathy almost in getting it right.

Matt: If developers use your product, you have a developer experience. And it's up to you whether it's going to be awesome or whether it's going to sit there and fester.

Steve Boak: Hi, I'm Steve.

David: And I'm David, and you're listening to Don't Make Me Code, the bi-weekly series where we discuss developer experience and some of the unique challenges we face building developer-facing products.

Steve: Don't Make Me Code is brought to you by Heavybit, a nine-month program for developer-facing startups. For more information, visit heavybit.com.

David: And if you're interested in being a guest on this show or if you have a specific topic you'd like us to dive into, you can reach us at dmmc at heavybit.com or on Twitter at Don't Make Me Code.

DX Before It Had a Name

Steve: We're calling this episode You're the Developer of Your Developer Experience. And we're here with Matt Harris, CEO and co-founder of Sendwithus. And he is going to talk with us today about building a DX team.

Matt: Hey, thanks for having me, guys. I'm stoked.

Steve: Yeah, there is a lot to unpack here. And I think we should start with a bit of history. And maybe Matt, you could kick us off and talk about your history as both a developer and a co-founder and how you got interested in the topic of DX.

Matt: Yeah, totally. I think it's interesting because I'm relatively new to the term DX, but in retrospect, it's something we've been doing for quite a while. So I am a former developer, I still sling code sometimes. My worst experience with developer experience has been back in the day writing against XML SOAP APIs, and that's probably the absolute worst developer experience possible.

But then with Sendwithus, one of the things we did, it's a developer and marketer product, and with two technical co-founders who like to write a lot of code, we wanted to make it really, really easy to send email and do email things. And so that primarily meant making great API clients for developers and making great API docs, from day one focusing on how can we ensure that this is as easy to use and as pleasant to use as possible for a developer. And so that's kind of where we got our start.

One of the things I'm really interested in talking about and learning actually from you guys is more about the distinctions between developer experience and just user experience in general. And how to think differently when you're thinking about developer experience.

Steve: Yeah, there are a few really interesting points there. I too am new to the term DX. Even as of a few months ago, I had really never heard it before.

And digging into some history, it's been around a while, and then I was really surprised to learn that David had not only been practicing it for years, but there was an actual team at Heroku dating back three, four years now. So that too is some really interesting history. And maybe you could talk more about that too, dude.

David: Yeah, so the DX team at Heroku originated around the time of Heroku's Cedar launch. So that was, I think, 2011. I don't even remember at this point. But basically Heroku went from being just a Rails Ruby hosting platform to really trying to host any language. So we introduced things like buildpacks and support for various client-side things.

And it turns out that trying to get the experience right for a Rails developer when we're all Rails developers is not that hard, but trying to get the experience right for a Python developer, a Node developer, there's just idioms and all sorts of things that you're just not familiar with in these languages necessarily.

So we formed the DX team drawing internally on Heroku from people with experience in various things. So it was me and Terence, who maintained the Ruby buildpack and ran Bundler, and Phil Hagelberg, who was a strong Clojure contributor, and Kenneth Reitz, who is our Python buildpack maintainer.

And sort of just pull together these people with sort of deep experience in these various languages and frameworks and tasked each one of them with trying to sort of recreate and own the Heroku experience in the terms of their specific languages or frameworks.

Matt: I think that's so cool because we had a very similar experience to Sendwithus when we started. We were Python developers and so we had an amazing Python client. And we got our first couple of Ruby customers and we're like, we actually had to hire a contractor or a friend to help us write Ruby.

And then one day we had someone asking for a C# .NET API client and I was like, oh man, I don't even own a Windows computer, I don't know how to do this. So I went on Stack Overflow and Google and found code snippets on how to do a HTTP request, post request in C# and how to do JSON in C#. And I just put it together in a gist and I sent it over to the customer.

And they got back two days later. I'm like, that worked awesome. And I was like, really?

But it wasn't a matching of our Python developer experience. It was a code snippet to hit one API endpoint. But it was a really big challenge for us on how to match that developer experience across different languages and different environments.

Steve: That's an interesting perspective on DX too, because it sounds like the DX team at Heroku was only developers in the early days. Is that right, David? It was developers with specific knowledge of certain languages that kind of worked together to tailor the experience to those languages.

David: That's right, yeah. So basically everybody on the team was developing a buildpack for a language of some sort. So these were all bash scripters at their core and also sort of very intimately steeped in the knowledge of various package managers or HTTP servers in their languages.

Phil Hagelberg wrote Leiningen, which is the package manager for Clojure, for example. So basically these people that were invested really heavily in their various tools that they had chosen.

What DX Covers Now

Steve: Yeah, and I remember researching the history of the term. Some of the earliest references I could find to DX were people on the Drupal boards complaining about the APIs and wanting to improve the developer experience of the Drupal APIs. And for a long time, as far as I can tell, DX was synonymous with API design.

And now, just banging around Heavybit for the last year, and as a reminder to anyone listening who may not know, Heavybit is a accelerator specifically tailored towards developer tools companies, we've seen a lot of different flavors of DX now, and people that are developing IDEs and of course APIs and all manner of developer tools products that have so much more than just APIs or programming languages involved. And so we've really had to expand the term now to encompass a lot more stuff.

Matt: I think kind of hitting on that, again, when we started, developer experience meant good API design. And then eventually that meant good API clients. And now it's kind of ballooned into everything from our documents around the API to just how developer aspects are shown in the dashboard and in the web product. That in turn means expanding the team from just the early developers who understand a core tool or a core language really well into a designer who knows how to make code look apparent to a developer.

An interesting anecdote is when we launched our developers landing page, specific developers, one of the things we did, and this is really common now, but I think we were one of the first companies to do this, was we had a well, it looked like a code block, and it was a curl call, and you could input parameters, and although it looked like code, there was a little pop-up saying, hey, click here and change the first name parameter or change this other parameter, and you could click and change and type in line and change only those parameters, and then the bottom was the run command thing.

And then effectively in the back end we took in whatever you entered and reran the command for you and sent you back the result.

Steve: Oh, so you actually ran it through a shell.

Matt: No, it actually ran Python code, but it looked like a curl command.

Steve: Yeah. Nice.

Matt: But it was really cool. And we got a ton of awesome feedback back from developers, and it was a really great way to illustrate what the API can do in a web page. And that was all a designer and a front-end guy who built that.

Steve: Yeah, and it's that kind of stuff. We've talked about documentation and boilerplate code and all the things we have to do for developers and how that, as a part of getting familiar with a new product, is so essential that you want a new developer to your platform to get comfortable making something on it very quickly. And so having, I mean, that is a great example, it sounds like, of being able to see the boilerplate code there, customize it to their needs, and then actually run it, not having to take it outside, do a text editor, copy and paste it or whatever.

Matt: That's cool. You said the Heroku DX team got started around 2011?

David: I honestly can't, it's either 2011 or 2012. It's funny how time has gotten away from me. It's whenever the Cedar stack launched.

Matt: I'm just interested in, I don't know how many years you were with that team, but how did it change when you left? What was different about it? How had it evolved?

David: Let's see. I guess I was with that team for about a year. So I guess it really started, like I mentioned, sort of trying to own the language experience on Heroku. Really, oh, we just want to build a good Ruby experience, a good Python experience. And then it really started to expand into just, we want to make the initial onboarding experience for all of these things really great as well.

So not just using the product, but how do you get our CLI installed, for example. We had a Ruby-based CLI, but now we're onboarding Python developers and asking them to install Ruby to install our CLI. And so we ended up building installers and Windows MSIs and sort of all this infrastructure around all of our stuff that was tailored individually to various platforms, Windows, Linux, OS X.

And then, yeah, ended up incorporating the API design, CLI design, sort of every developer-facing touch point for Heroku ended up kind of under that purview.

How DX Differs From UX

Steve: Yeah, now we're kind of dancing around all the different issues around DX and what makes it a sort of specific subset of UX, and that we have the same goals, but we go about our business really differently.

And I always tell the story of going to, so I'm a designer, and I have gone to these designer meetups and had nothing to talk about with a lot of the people there because the daily practice of this business is so different. I've been working on DevTools for the last six years now, and my day-to-day routine is so much different from a standard UX designer working on a consumer product, and the considerations are so different.

And this kind of stuff that we've been talking about, having to write code and install CLIs and get keys to other accounts, there are so many different considerations. Just onboarding. I mean, we measure successful onboarding in minutes, if not hours, with the DevTools product, where you might expect 15 or 30 seconds to input your email address and get started with Pinterest or something.

Matt: Yeah. I think it's interesting too, you mentioned this before, but the level of ask. The ask to install Pinterest is super low cost to the user. I press the install button, I get this app, I know I can install it in 30 seconds, and the evaluation period for them is so short.

For a developer, you're asking, stop your productive work writing code, do something in the command line, which developers have varying degrees of familiarity and comfort level in, and then install something on your computer, which is harder to remove, and then, yeah, start making API calls and doing things, and it's up to 15 minutes or an hour of trying to learn and understand what something can do. It's totally different. Totally different design behind that.

Steve: Yeah, and not to mention that this may not even be the developer's own environment. It's probably their corporate environment, where trust is an issue and they have other people depending on them.

We've seen all kinds of weird behaviors where someone will sign up with a personal email address and I'll go and do five or ten minutes of research just to figure out who I'm dealing with, because they're signing up with their personal email, but they want to install it in a company account. And the whole sandboxing, they might put you in a test environment or something that is otherwise pretty useless. And so you have to figure out a way to show value in less than ideal circumstances.

Matt: Yeah, that's really hard. The really hard thing I see about developer products and making that onboarding experience really great is often when you have a set of APIs, you have to use multiple APIs in conjunction, multiple endpoints in conjunction, to actually understand, oh, this is the complete set of value. It's really hard to be like, here's one API, you use it, and the end result is that performs a unit of function which you're happy with.

One of the things I thought was always really awesome about Heroku, and we've modeled a lot after this, is how much you could do through the command line, how powerful the Heroku command line tool was in conjunction with the website. It made you feel as a developer like you were the most important user of Heroku, right? Because you could scale dynos up through the command line or you could use sliders in the dashboard. But you had all the power in your command line, which I think was really cool, empowering feeling.

Steve: I think I remember seeing that in your talk at Heavybit Summit, David, about how that command line tool evolved over time from just push master to having lots of iterations and functionality built on top of it.

David: So the talk itself was just about kind of breaking down some of those higher-order, really product interfaces into developer interfaces. So git push heroku master is actually a really great product interface. You can push your code out, but if you're a developer and want to hack and tinker and actually customize the way things are done, it's actually a little bit limiting.

So being able to break down some of these more magical things into their constituent pieces and then give access to them to people that know how to string them back together into whatever they want can sometimes be valuable.

Steve: Yeah, and at the end of the day too, touching on the API issue too, that you want to build up this level of comfort because these are professional tools. They're things people are using at work, and you want this to become part of a developer's daily life. This is part of their professional tool set. So you want them both to be comfortable and relying on this every day.

So there's this interesting debate about power versus simplicity and how for a long time, I think tools in our space have leaned much more towards power over simplicity. And I think a lot of the effective design that's happened over the last few years has been helping to kind of restore balance and make tools appear more simple, but also be very powerful for people's daily use.

Matt: Yeah. I think one of the ways we've looked at that, at least within Sendwithus and our developer products, is the question is, if you make an API call and something goes wrong and we can't successfully do what you wanted us to do, how should that be returned? Because the design component of, do you actually return a HTTP error or do you return a 200, oh, request was okay, and then a JSON payload saying there was an error? And at what level do you do that?

And especially, this is probably relevant to the work at Heroku as well, when you can do things in the dashboard or through an API. And what you do in the dashboard will affect how the API calls are functioning or whether or not they're successful. And it was being a really big debate internally about how we should do this and then what we should hide from the developer or how much we should give back.

And we've ended up going the direction of, on the technical side, if it's a good API call, we should 200 it and then include error messaging inside of the message. But never silently fail. If the developer makes an API call that was successful to accomplish whatever task they wanted. And that's kind of going for the simplicity route and trying to build that trust.

Steve: Yeah, and what it means to be intuitive in this space, what people are expecting and latching onto ideas that they're accustomed to.

David: Yeah, so I mean, the simplicity versus power thing is, it's just so hard to get right. Being able to access sort of all the underlying primitives of the thing is what makes your product even work at all for some people. But throwing them into the deep end right off the bat is also a really terrible experience for most people.

So there's usually a balance, and it's often time-based, I feel.

You want to start people off simple and give them the happy path where they don't have to make many decisions up front. But then later, as they start to gain a better mental model of what it is that you've built and how to use it, they can actually start to peel the onion back a little bit and expose a little bit more optionality.

What Made the Heroku DX Team Work

Matt: I had a question for David. My question was essentially, for the Heroku DX team, was there anything special or different about the people on that team versus other developers at Heroku? Or, in retrospect, what made for a really good fit on the DX team?

David: Huh. So let's see. The thing that made them special inside Heroku was that they were people that had stepped up to maintain buildpacks for various languages. So as we started to build out the multi-language functionality internally and really settled on the idea of buildpacks and that they were going to be these separate pieces of software that essentially would encode all of this knowledge about a given language or framework, the initial people on DX were the people around Heroku who had sort of volunteered to maintain one of those for their pet language at first and were just sort of doing that on the side, and then we pulled them all together into a team.

Matt: Did you mention at least one person was also a contributor to, outside of Heroku, one of the build programs for, I can't remember the language.

David: Right, yeah. So actually Terence Lee was our Ruby maintainer, also one of the main contributors to Bundler. And then Phil Hagelberg was our Clojure guy, and he essentially wrote Leiningen himself.

Steve: That gets to an interesting point about the team in general and the important roles on it. And so, clearly developers. As I was saying, I've been a designer in this space for a while, and I simply can't exist and do my job without developers and without them helping me and without them being a part of the developer experience.

And DevRel, people that are talking with the community too. I mean, the minute you have an API, you have people who depend on it. And so developer relations or someone that's actually communicating seems also to be a really important part of that. And I'm curious if there's anything we're missing.

Matt: Yeah, that's a really great question. I think combined with DevRel, there's a support aspect to this, where a very specific example is almost every developer product that we can probably talk about probably has open source clients of some kind that are on GitHub.

And the moment you have that, you have GitHub issues and support tickets coming through, which are outside of any other support channel you have. So there's an ongoing support cost to investing in developer experience.

Steve: Yeah, and particularly, I don't know if Sendwithus has any open source. I know Convox does, and I'm curious how you manage that, David.

David: So Convox is almost entirely open source, so we ourselves can kind of work out of the same workflows that our community does. So, yeah, we submit pull requests, our users submit pull requests, we kind of deal with it all mostly the same. For us, actually open source, especially being sort of all in on open source, is a real advantage there, in that we don't have this sort of two-headed beast to deal with. It's really all the same problem.

Steve: Yeah, and Matt, I think your point too is good, that once you put it out there, it's out, and what you decide to do with it next is really important. Like Heroku did, you can either gather a team of specialists to help make that experience better, and, like at Convox, you become part of a community and help it grow, or you can just let it sit there and fester. And somewhere every team is doing this, whether they made a decision consciously or not.

Matt: Yeah, yeah. That's exactly the point. Whether or not you're consciously trying to make a better developer experience, if developers use your product, you have a developer experience. And it's up to you whether it's gonna be awesome or whether it's gonna sit there and fester.

And we do have open source in conjunction with a SaaS product. And one of the great things, again, was we're not C# specialists, and we're also, at the get-go, we didn't have Ruby specialists in the house, and having a community come around and help us build out the clients and help us build out the developer experience was a really magical moment. So it was really cool, and one of the reasons I'd recommend investing in DX.

Steve: Yeah, and actually seeing people rally around something like that is so rewarding, when you have people contributing and feeling like a part of the community.

Matt: One of the things I was kind of hitting on before, asking about the Heroku team, was some of that I've noticed internally in our company, some of our best team members with respect to making a better developer experience are pretty active in open source outside of what they do with us or before they even started to work for us.

And I think it's, again, this is purely on the developer side of the DX team, but developers who care about open source and care about writing software that just writing code for other coders, kind of intuitively get this and are trained through this by open source, in some ways, on what it means to create a good developer experience.

David: It's interesting that you say that, and I'd never really thought about that before, but, yeah, every member of the Heroku DX team was a maintainer of an open source project of some kind. Yeah, Wes Beary had Fog, and Kenneth had Requests, and Terence had Bundler. I mean, they all had pretty significant open source projects too.

Steve: I think there's something to be said too for the culture around that and being comfortable. There's two parts to it. I think one you already hit on, which is, once you've been a part of an open source project, you know what it's like to write software for other developers and to contribute to a larger effort.

And then there's, second, there's this comfort with the politics and culture of open source projects and just being good at and experienced at navigating that world, and that can be valuable both as an open source contributor and to a company that's trying to build DevRel. You naturally will then look to a community to help you improve your own work.

Matt: I think that's interesting because if we're drawing analogies between open source and developer experience, one of the things most open source projects feature, bunny years feature, is a vocal spokesperson who's not necessarily writing a lot of code all the time, but is active on mailing lists in blogs or active in support tickets, just communicating about what the product should or shouldn't be doing, or what should or shouldn't be included in the future, what the goals around the project are.

These are the benevolent dictators. And just like in open source, that analogy stays true for developer experience within a company when you're designing for an end developer. You need someone who is not necessarily writing code, but is bringing other skills to the table and able to communicate and able to do the DevRel side of it.

David: Yeah, I mean, so open source has basically a developer relations problem baked into it, right? If you want people to come and not only use this thing that you've built and put out in the world, but contribute to it, it has to be pretty easy to get started and pretty welcoming to new people.

This is all volunteer contributions. It's not like somebody's getting paid to write this code usually. So you have to really focus on making it easy for people to get your thing booted up, make changes, and be confident that the changes they've made are good.

Steve: Yeah, and I think the cultural aspect is at least as important as the code itself, in that I feel like I've seen a lot lately about open source projects with non-benevolent dictators just shutting down pull requests and taking too controlling a stance on things and making their own values, putting them at a much higher level than the rest of the community, and really screwing themselves over that way.

Internal Developer Experience

Matt: Yeah. Something else that came up that I thought was really interesting is looking at internal versus external developer experience. And so I was actually on a panel in New Orleans a week ago speaking with the founder of CircleCI as well as the founder of a Heavybit company, Edith Harbaugh from LaunchDarkly. And I brought up the fact that we actually treat developer experience as an internal problem as well as an external one.

So an analogy would be, Heroku is famous for the Heroku command line utility. Internally we have the SWU, what we call the SWU toolbit, and SWU stands for Sendwithus, our company. And it's an internal command line tool that all developers install, and it simplifies the process of running and installing and doing simple tasks for all the different internal services we work with and run.

So it means someone from my front-end team can go and launch a database or an app service without learning a new stack or having to learn how to install different Ruby stuff, just by running the same command line he would normally run for his front-end project.

Steve: Yeah, that's a really good point and a big component of internal DX.

I'm really fond of the expression that every company builds two products, the one their customers buy and the one their employees buy.

And that experience is really interesting as a developer tool company.

How do you make things good for your own developers? How do you get them onboarded smoothly? How do you get them to be effective and writing code quickly? I mean, just like we were talking about with a user of your API or something, you want to provide boilerplate, ease of use, tools to get them onboarded quickly and comfortable writing so that they can be productive. And then I know that's gonna be a rewarding experience for someone new to the company and will get them comfortable with your team right away.

Matt: I'd never heard of that idea before, of you build two products, the external product and the product your own team works. What's special about the internal product? My initial thought is, is it important that the external product and internal product are very similar? The more similar the better? Yeah, what's special about them?

Steve: What's that? Is it Conway's Law or something, that your team is destined to build a product that reflects the culture of the team itself, and, I mean, team cultural generally, right? I mean, it says something, I think, how a culture is top-down usually, right? So I think it says a lot that the founders and the leaders of the company are taking strong positions on certain things like user experience or developer experience and making sure that that's a priority for the product and the company. And that it manifests in so many ways.

I mean, just thinking back to our own history, how we've iterated over our interview process and our onboarding process and how we try to integrate new team members really smoothly, and it sounds like you've done something similar, not going as far as building a complete tool set for new hires. Yeah, I think people see those efforts and appreciate them.

Hiring for Empathy

Matt: How important, if someone's looking at, whether it's a small startup or a large enterprise, and they're starting to think about developer experience, how important do you think is this open source component, having previous experience in open source?

Steve: You mean, do you look at whether someone has contributed to open source?

Matt: Yeah, as a qualifier, if I was like, I'm building a DX team, what qualifiers do I need to look for? What's special about a DX team member versus just a general developer on my company?

Steve: What do you think about that one, David? I think you have a lot more experience with that than me.

David: So I do tend to look for open source work, but it's not even because it's open source. It's because there's so much stuff available on GitHub. If I'm looking through somebody who's applying for a job, I'm not even necessarily going to look at the code.

I'm looking at their READMEs. I'm looking at how is the thing structured and how do they respond to users and issues, even just sort of all of the ancillary stuff around a project where you can kind of get a feel for how welcoming they are to people, I guess.

Steve: Yeah, what kind of developer they are, how professionally they approach their work, and if they're maintaining their own project, how maturely and professionally did they put it together?

David: Yeah, and there's a certain level of empathy almost in getting it right. Getting the experience right. You have to feel somebody else's pain to care about it.

And I really like to see evidence of that kind of empathy in how they respond to issues or pull requests to their projects.

Steve: Yeah, one of the issues we've had with academics, for example, is that many times, even if they're very talented developers, they haven't actually had experience building something for someone else. And for all of us that are trying to build products, I mean, that's all we do.

So how good you are as a developer almost matters less than your ability to empathize with the user and actually then take that to heart and make that a part of your mental model of what you're building.

Matt: Yeah, I definitely agree with that. We go so far as to some of our job postings as not a developer, but we just look for product developers and try to emphasize the product component as much as possible.

Talking about the other components to developer experience. Just as important to actual API design is the READMEs or the evangelism and the community aspect. How else do you qualify that outside of a GitHub profile?

Steve: You mean qualified in a person?

Matt: Yeah, or, yeah. Maybe I'll give an example of API docs and an anecdote. When we were designing our API docs, we thought Stripe was just the bee's knees, they had the best API docs. And so we wanted to model our docs after that as much as possible. So we went as far as copying full-on layout and color scheme and other things I won't mention.

And it was kind of just the accessibility of their docs and the ability to find things quickly, that was what was key to us. That's what we were trying to copy. I don't know if there's a good way to put that in terms of, when you're looking to build a DX team, those are the things to look for.

Steve: Yeah, I mean, all the things that it brings together. I mean, the Stripe docs are sort of mythological in the command. Everyone, I mean, I've heard the actual phrase Stripe-level docs. And how we have tried to do similar things, like inserting API keys automatically and everything that people now come to expect as part of a new developer experience.

And, yeah, we always seem to come back to documentation because it's so critical, and it's such a differentiator between what we do and what consumer products do, because many consumer products, it seems, just get by without real documentation. And for anyone writing code, that's just a non-starter. You need to do something. And, yeah, the docs, the boilerplate code, the interactive bits, what you've got, where someone can actually run commands with minimal customization.

That stuff is huge. But now it does seem to be kind of specific to the product too. Our product kind of rides on top of AWS, is a monitoring solution, and so big part of what we do is try to automatically discover what's in a user's environment and automatically create health checks for them and do things that we feel will offset their own time.

And that actually is probably the better general umbrella, looking for ways to save them time, because I think unlike a lot of other companies, we're competing with our own customers.

They could arguably go out and try to build any of our products on their own. And it's our job to convince them not to do that.

Matt: Is that the problem all developer companies face, is that you're building something so that someone else doesn't have to build it, so the individual developer doesn't have to build it? So you have to justify your existence by saving time.

Steve: Yeah, I think that's at least something we all have in common, that none of us could make an argument selling to developers. I mean, they could all go out and try to build whatever any of us are doing on their own, and it's tough to quantify the value of our product in terms of time saved in their own development team. And so we make every effort to quantify that in terms of the steps they would have to take using a competing open source project or something to actually get everything configured and set up, where with our tool, in a few minutes or whatever, they can get set up and running.

David: And you have to be up and running pretty quick and pretty flawlessly, or all of those other alternatives start to look pretty enticing.

Matt: Yeah. I guess that comes back to the thirty-second time window to evaluate Instagram versus fifteen minutes to evaluate a developer product, depending on the amount of buy-in required.

Steve: Yeah, you have some leeway, but the same equation applies. You've gotta give them a reward pretty quickly and with minimal effort. And I think there's this kind of conversation back and forth, where if you need to ask for more, you have to do it in a sort of chunked-out way, where you give a little and they try something and you get something back, and you kind of balance that back and forth until they're fully integrated.

And, David, you were talking about this earlier, how you kind of build up to full exposure, where you start down the happy path and you make a lot of decisions for them, and then you gradually expose the power of the tool to them.

David: Yeah, that's a tough thing to get right, because you almost want to hint that all this other stuff is there, but don't worry about it yet, but it's there when you need it.

Steve: Yeah. And there's also, I think this came up once before on the show, but I still remember my first experience with the Google Maps API and just how perfect their boilerplate code was. It was as if they knew all of the top use cases that people would have in mind and just had them laid out in this really simple map. You want to do X, here's some boilerplate code for it.

And there's definitely a product aspect to that too, in that if you really understand your users, you're not just going to have boilerplate code. You will have thought about where that will be used and what the most important use cases are for it. And you'll tailor everything from your docs to code and everything in between to those use cases, your whole onboarding experience should be tailored around that. How do you think people are going to be using the product?

Matt: Yeah. Heroku does a really great job of that, with, you can Google Heroku Python Django and you'll get dumped on a page which is, step one, Heroku command line client install is one line, and it's create your Python project, one line, and then, all right, make your first commit is one line, and then it's, okay, now you deploy to Heroku and you're done. And it's this Googleable use case you can find, and it's really well laid out.

Steve: And I think there's something to be said for building you don't immediately onboard people and hire people just looking for that as a quality, someone who appreciates that and will continually build toward it.

I've seen just in the past few months of our company developing, we are building this culture that, as we've experienced issues with customers, and there was a relatively recent one with security where we had to revamp our onboarding process because people were unclear about our security policies, how much permission we had in certain areas.

And so we had to lay things out and be more transparent, and it really took the whole team coming together around a solution to that and everyone wanting to do that. And that was something that it really took some time to ingrain in everyone's minds, that when we see this problem, we're all gonna stop and address it and make it part of our priority to do so.

Keeping Every Touch Point in Sync

Matt: I think that's really key. We've had some growing pains around developer experience, especially in terms of the fragmentation of the experience. So meaning that we've got things on GitHub, and the Python client and GitHub has a README which contains the API for that Python client. But then on our homepage we have our own API docs. Then we've got FAQ docs somewhere else.

And it's trying to keep all of this updated and in sync, it was fundamentally a cultural problem, in terms of, when we make updates to the API, it's on us to go and ensure that everything is up to date, and we're a startup, so the changes happen quite frequently. And we're doing a disservice to our developer customers if we're not going through and being very diligent about keeping this all up to date, because it's a really terrible experience to download a client and go look at the README and then try to type something into a CLI and then you get an error message, and that's just a killer.

Steve: Yeah, a few shows ago we had Dustin Larimer of Keen IO in here, and he was talking about a recent refactor of their docs and their APIs actually that they did, where it had become fragmented. They had different APIs for different components of the product. And they had become very fragmented. Different commands would do the same thing, different terminology for the same thing.

And so they did this whole exercise where they started by looking at their documentation and looking at the different ways that I did fragment it over time and going through a whole exercise to make that terminology consistent, to actually break out one massive library into a few different ones that they figured out how to compartmentalize after looking at the documentation.

And that was an interesting one too, how you grow that experience and how you refactor it. And it sounds like you're going through something similar.

Matt: Yeah, that's really interesting. I'll have to go back and listen to that podcast.

Steve: Yeah, he called it documentation-driven design, where they started by looking at how fragmented their docs had gotten and saying, shit, we need to fix this.

Beyond Stripe-Level Docs

Matt: So we talked about how Stripe's API docs are kind of the spec that everyone wants to try to achieve right now. What's kind of the next level of developer experience? How do you go above and beyond and beat what Stripe's doing with their API docs?

David: So I really think that one of the most important things is, I mean, we kind of mentioned it in the title of this podcast, is Don't Make Me Code, right? If I want to get started with your product, even if it is a developer product and even if you have an amazing API, it is really awesome when I go to use something like that and I don't actually have to write any code to get working with the thing. I can just push a button and put it in an API key or an OAuth or something where it just hooks up to my stuff and just starts working. And maybe I can go customize it later or do something, but really just that initial onboarding, getting working.

And Stripe does this sort of okay. Their API docs are amazing, but they have this really powerful API, but don't do a whole lot for you also. So I do love their API docs. I consider that a really great example of DX, but the fact that I can't just plug their thing into my website without writing any code at all, I think it's a deficiency.

Steve: Yeah, I'm starting to see this really interesting set of tools start to work together. From a Swagger, there is a tool that will automatically generate an API for you. And Apiary, I think, now can read from a Swagger and generate documentation from it. And there's another founder that I talked to that will take the Swagger and not only look at the API, but then generate SDKs for individual programming languages for it. And we just got off a conversation with a company that wanted us to then look at that Swagger and automatically create health checks for all the API endpoints there.

And so this whole round-trip thing where, from something like a Swagger spec or GraphQL, you get everything, your API, your SDKs, your documentation. Everything that you need. And so there's no code to write at that point, hopefully.

Matt: I think my issue with that kind of scenario that you just described is kind of what David said, is that what you get if you do that is a very technically correct API that, you can get this and then post it and then update it and delete it, and it all works, and here's all the documentation. But you lose some of the magic.

You end up with a Stripe-style thing, where a Stripe API is really great, but those API calls aren't really magical.

They do what they say they're gonna do, and that's it, absolutely it. And I think there's a question, when you're designing for developers and building developer products, of, I think the API should do some magic as well. And if it's just get, post, put, delete, I don't know that you can get that magic in there.

Steve: I'm still trying to figure out what you mean by magic there.

Matt: Yeah, so a typical API where you get, post, whatever. It's going to create an object or delete an object or maybe put an object somewhere. But it's not going to do a more complex operation. Like Stripe, you can create an invoice, add line items to that invoice, add a customer to that invoice, and then charge the invoice. But those are all explicit API calls you have to do, and it's really great at doing that.

So a harder thing to do with Stripe's API is, oh man, I need to, through the API, send out a reminder to all my customers who, their plan's gonna expire in a month. And you can't do that through Stripe's API. And that's kind of magical.

Steve: Right. So going a step further, not just with CRUD operations, but actually giving people what they really need from it.

Matt: CRUD operations is a phrase I was looking for, exactly.

Steve: Okay. Yeah, and so now I'm trying to think of some counterexamples. Is there anyone out there doing this really well?

Matt: I would argue Heroku's command line client does tons of magic for you. And that's not a CRUD API. But there is a CRUD API hidden behind it that you can use if you want to. That's interesting. So does the Heroku command line client then wraps up, I write one command with a Heroku command line client and it in turn does six to ten CRUD operations for me?

David: Yeah, I mean, a lot of 'em are gonna be one, but, yeah, in many cases it's packaging up a lot of stuff into just a single command.

Steve: So, David, you may feel differently about this. He and I, our companies are both very active on top of the AWS ecosystem. And I feel that AWS is a pretty good example of a company that's really taken, depending on the lens you look at this through. They do some really cool stuff with their APIs, and with tools like CloudFormation, you can do a lot with basically the click of a button.

Someone can create an entire stack inside their AWS account from our product. And we can tailor that to their environment with specific customer data and environment data that we have. And so I think, whatever you think of their UX as a whole, I think they've gotten really good at creating APIs.

David: Yeah, I mean, I think they have great APIs. I think it's sort of the same, hesitate to say problem, but it's sort of the same level of challenge, in that they have really amazing APIs with very, very extensive documentation. But if you come in and say, I want to host a website, there are a lot of steps between you and actually doing what you want to do.

And they are getting better about this. They're releasing other tools, and they're starting to add more web stuff and wizards and things like that. I think that stuff is really important.

Your API is only as good as people's mental model of your product, and if they don't even know what the pieces are yet, just throwing this giant API at them isn't helping anybody.

Steve: Yeah, and I mean, there may be a better example that they go beyond CRUD, but they still, for a lot of the things that we need, for example, I mean, entire monitoring companies have spun up basically to build on top of the analytics and monitoring metrics APIs that Amazon provides, because they basically give you min, max, and average on any metric. That's not very useful to someone who's actually trying to collect useful telemetry data. And, yeah, if someone could ask better questions of their APIs, then it would make things a lot easier for us.

Matt: So to rephrase that, or to say it back, there's developer products built on top of the Amazon APIs, whereby their entire goal is to provide a better developer experience on top of the Amazon API.

Steve: Yeah, David, that's what you're doing, right?

Matt: Yeah, I mean, that's essentially what Convox is. Yeah. Which I think is pretty cool, then that speaks to Amazon's failure for developer experience, and then why it means more than just having a great CRUD API and great docs or extensive docs. Sorry, I took us down a huge tangent there with magic and APIs.

Steve: No, it's true, but I mean, once your product gets big enough, you've got to start thinking about that. That, I guess, for whatever strengths they have, they haven't done a great job of consolidating their APIs and tools in a way that a typical developer can ask a question, like David was saying, how do I create a website? And that you've got to munge together, I don't know, 20 different tools to get that done.

Thanks again to our guest, Matt Harris, for coming by. It's been a great conversation, and we are all now excited to go off and continue building our DX teams.

Matt: Yeah, thanks for having me, guys. It's been a ton of fun.

Steve: That's about all we have time for today. If you're interested in being a guest on the show or if you have a DX topic you'd like us to dive into, you can reach us at dmmc at heavybit.com or on Twitter at Don't Make Me Code.

David: To learn more about Heavybit, visit heavybit.com. And while you're there, check out the library. It's packed with amazing talks from founders of developer tools companies and other industry leaders.

Subscribe to Heavybit Updates

You don’t have to build on your own. We help you stay ahead with the hottest resources, latest product updates, and top job opportunities from the community. Don’t miss out—subscribe now.