Ep. #3, Give It a Name: Why Software Needs a Third Loop

James Governor: So giving it a name is really important. That's part of why we wrote Progressive Delivery. There was a movie in the very olden days, and I don't think it's particularly well remembered or anything. It was called Things to Do in Denver When You're Dead.

It was about-- There was sort of mafia, and there were some contextual things. "Give it a name" was one of the themes that ran throughout the show. One of the things was boat drinks. So they would talk about boat drinks, and boat drinks was like, you know, you're the mafia, it's the end of your career, and you end up, you know-- Valhalla for the mafia is you in Florida on a boat drinking ridiculous cocktails. And those are the boat drinks, sort of Valhalla.

Heidi Waterhouse: Uh-huh.

James: One of the themes of this movie was: Giving it a name. A movie about language. And I, who work in the tech industry as an industry analyst, obviously protection rackets, giving it a name, I understand that. It's not generally something that we do, but there may be some firms that do like to give things names.

Anyway, the truth is, in tech we coalesce around concepts. It is great to be able to say, "Oh, that. Yeah, I get it. Progressive Delivery. You pull together feature flags and dark launches and the observability to close the loop and all of these other things. This is great. I get it. This is production excellence. Amazing. Progressive Delivery."

But the podcast, I think there's something--

Kim Harrison: Wait, wait, wait, wait, wait. James, let's pause. You gave Progressive Delivery a name. For those who may not have heard any other episode, why did you name it Progressive Delivery? Just in one minute or less.

James: Basically Sam Guckenheimer from Microsoft Talks about progressive rollouts, something that they did--

Adam Zimman: Progressive experimentation, right?

James: Progressive experimentation, sorry. It was something that Microsoft have done actually for the longest time. If you think about the blast radius, one of the reasons everybody gets pissed off at Microsoft when they do shit and you know, that may be updating the operating system or anything else, is the blast radius is so big. Right? You are touching so many people, so everybody knows about it.

And so they would do experimentation, they would try things for a subset, a specific cohort of folks before they rolled it out more broadly. And that was, at the time, Azure DevOps. That was what Sam was working on and that was one of their precepts. But a lot of the things about Progressive Delivery is--

I mean look, smoke tests are not new, canarying is not new, blue-green's not new. But packaging altogether with the capabilities of the cloud from an abundance perspective, amazing automation coming at it from an alignment perspective, that's Progressive Delivery. So naming things is just super important and enables people to understand stuff.

And that was-- The progressive clicked. I've been thinking about all this basket of things and I didn't have a name for what CICD looked like if you'd done the Internet for 15 years, right?

Adam: Yeah. This was where you know like when you and I started talking it was, we both had the same problem. We needed something to call it so that it was consistent, as much as consistency happens in tech, to make sure that when we said something we didn't have to go into the hour long explanation.

James: 100%, and we said, going into an hour long conversation.

Heidi: Haha. I also want to say, I had been talking about the difference between release and delivery and that putting something out was not the same as people accepting it.

James: 100%. I think we sort of, as time went on, there were times when Adam became a bit worried. "Wait, we call it progressive? I mean in this post woke era, will it be okay?"

But I think A, fuck the haters and B yeah, so the idea of progressive rollouts, so being able to limit the blast radius, it came together really well. And, and yeah, I think naming is important and Things to Do in Denver When You're Dead, think up Progressive Delivery. I mean, you know, like that's a boat drinks, it's a Valhalla. Right?

This is the amazing thing that happens when you improve your culture and improve the platforms you use and are able to deliver the right product to the right person at the right time, the right user at the right time.

Kim: So I think that brings us really neatly into the naming of this podcast. Right? Everything the three of you have said--

James: It's almost like we had a plan, Kimberly. Haha.

Adam: So another kind of side quest is that one of the books that I really enjoyed, kind of fantasy fiction was Patrick Rothfuss, Name of the Wind. If you like fantasy books, great book. But the general premise, or one of the underlying currents is: Knowing the name of the thing gives you control over it.

James: Oh, yes. In Magic, that is absolutely a thing.

Adam: There's this certain aspect of this that I think kind of resonates with me in terms of how we thought about naming, and when it came to the naming of the podcast. Right? I think there were a couple things that converged on that name. Right?

James: I mean, we can definitely talk Ursula Le Guin and naming things and giving them power. I'm ready for that conversation. 100%.

Heidi: Madeleine L'Engle. Yeah.

Adam: Yeah, exactly. I think, persistent theme.

James: Names do have power.

Adam: Yeah. And I think that when we were thinking about this and when we were working our way through the book, there was definitely this aspect of, to Heidi's point, like, I think this was something that Heidi kind of came back to again and again that forced us all to think a little bit more about-- You know, in DevOps, we'd been talking about the separation from deployment and release. Right?

And that was a huge part of the notion of how do you get something built and then how do you get it so that it's running and operating? Right? And that was the segmentation between developer and operator.

But the realization was that there's actually something that happens between the running of a thing and the acceptance or the adoption of a thing. And I think that that was where we came around to this idea that there's another loop.

James: Wait, are we gonna name the podcast, Adam?

Heidi:

We're gonna call it the Third Loop because we feel like when we broke down the wall between Dev and Ops, it was transformative, and it gave us so much insight into how the other half lived. But as we did that, we came to realize that there was another wall that we needed to break down, and that was between the software makers and the software users.

And if we can break that down and say, people who use software are an essential part of the software experience, like, we are not giving them this from on high. This is a collaboration, and if they don't use it, our software means nothing.

So we need to break down that next wall between people who make software and people who use software. And the way to do that is to be able to actually not only listen to them but to see how they're using the software.

Adam: And I think an interesting point about that Heidi, is that something that we've talked a lot about is the idea that when software was isolated to a power tool that certain segments of the population would use to accomplish very maybe career or job specific tasks, there was an aspect of natural alignment that took place between the software developers and the software users in terms of, it was a specialized thing.

I think the thing that has accelerated and exacerbated the need for this breaking down of this third wall is the fact that software is everywhere now.

And so all of a sudden-- I think every single one of us has the experience of someone in our lives that is completely non-technical to the point where they would not have a phone or any type of digital devices if they didn't need to, to participate in society. And they've been confronted with a challenge where something has changed at the software level that they have been ill equipped or incapable of being able to navigate themselves.

James: Are you talking about healthy people, Adam?

Adam: I'm talking about healthy people. But you know, this is the thing is that when software's in your car, when software's in your house--

Heidi: Tractor.

Adam: Your tractor. A ll of a sudden you can't avoid it. And I think that the challenge there is that when the software maker is inflicting updates on their users as opposed to thinking of it as a coordinated kind of relationship and sharing, you run the risk of doing real harm, physical harm to your users.

James: So for me, yeah like I don't think we fully realized this when we started off on the journey, but it became more and more apparent to us that actually this third loop, including the user in the process, including the user-- You know, if you think about product, which is what we're doing here, it just became more and more obvious that actually the Progressive Delivery and the inclusion of observability, so you have that feedback loop, that Third Loop, I think we all got really excited about that.

And to me now, Progressive Delivery. Yeah, the Third Loop's a great way of talking about that, that so much is about understanding the user experience. That's, what are we doing here? We are trying to create delightful experiences that people will get benefit from and I think that, yeah, Dev and Ops, awesome. But how about Dev, Ops and Users? That is the Third Loop.

Kim: I want to ask you all some questions about names that almost made it. And I'm going to start off with one of my favorites. We're going to pick on Heidi again because I think we all loved this, but it just didn't quite hit, only because Third Loop was that good. But Never Finished was one of the almost titles.

Heidi: Yeah.

Kim: And there's some others that we got to get into.

Heidi: So Never Finished was an earlier iteration of this idea that:

We don't burn software onto CDs anymore. And so because it is ubiquitously deliverable, it's never set, it never stops. There's no real good release point. And we need to accept that software isn't ever finished in a web world because we keep tweaking things. And it's mostly good, but sometimes very difficult.

Adam: Well, I think that the other part of this that goes along with Never Finished is the idea that our world doesn't stop.

Heidi: Mhm.

Adam: And in the context of software, especially now that software is in all the things. Right?The world is constantly changing and there are external factors to our software that are constantly moving and changing and updating around us, whether it's interdependencies through other software products and services and applications or the way that people use them, t he shapes of the physical things that they're fitting into--

Like all of these factors, one of the amazing opportunities with software is that it gives us the ability for these updates and these changes. But at the same time, this has also put us into a situation where it has given us the burden of having those changes just accelerate and move faster and faster around us. Right? And we are impacted to an increasing degree by those changes.

Kim: Another title idea. Adam, sorry, it's your turn: Radical Delegation.

Adam: So I think that this was something that came from--

James: I think you should do this in a pirate voice, Adam. Haha.

Adam: Or at least Lord Captain Commander?

Kim: Yeah.

James: Well, that's a whole set of British poshness, which I do look forward to hearing you try.

Adam: Ahem. In the course of naval battle, you need to make sure that when you give a command, it's followed to the fullest extent. And sometimes those commands are given one month and it is many months later in which you are needed to reach your objective. Therefore--

James: Oh my God. I never want you to talk in any other voice, Adam. This is your Third Loop voice. Haha. This honestly makes the show so much better. And now you are only allowed to talk in that voice.

Heidi: Okay, but let's discuss what he's saying. Haha.

Adam: I guess first I need to know, James, whether or not that was even close to the appropriate regional accent or not even?

James: I mean, I'm not really the right person to ask, but listen, that was amazing.

Adam: So I'm gonna table the caricature for now, just so that I can actually formulate my words on what I actually want to get across. Ultimately, this was the notion that we include in the book, it came initially from the Navy. And this was in the context of when ships were commanded to reach an objective. In the early days of the Navy, this was something where we didn't have, you know, satellite communication. We didn't have the ability to reach vessels anywhere at any time from any type of, centralized command.

So the objective was given, and then it was up to the captain of the ship to be able to coordinate and reach that objective in meeting the parameters of the mission. And so this was the idea of radical delegation, where you were giving complete control of the objective to the captain, and the commander of the overall mission wasn't even there. It was just like, no, this is your part.

And similarly, we found that one of the core tenants of Progressive Delivery was how do you actually separate the control and the objective of your software from the delivery of your software? And so part of that was, how do you make it so that you can actually delegate who gets to see what and when from a feature functionality perspective from the person that built it, or even from the person that is operating your environment down closer to the user. Right?

And this was something that we talked a lot about in the context of not only feature flags and feature delivery, but also in the context of observability. Right? When do you know that a user is ready for a new feature, a new workflow, new functionality? How do you control that? How do you make it so that it is controllable down to either an individual user perspective or a group, a cohort, some other criteria that you've identified?

You know, whether or not it's-- You think about it from a perspective of one cohort could be users who log into your service multiple times a day versus users who log in once a week versus users who log in once a quarter. Ideally, what you're doing is you are delegating that control as the feature matures to a individual or group of individuals that are closer and closer to the end user as you go forward in time.

And this gives you the ability to ensure that your users that are the slowest to adopt have the most control in their adoption. And this is really giving them agency to be able to then say, "hey, look, I'm now ready." And this is something that I think that we've seen kind of vacillate back and forth with things like phone updates. Right? I think that's the one that is probably closest to most people more so even than operating system updates these days, computer operating system.

I think that phone operating system updates is something that a lot of people, they definitely have contacts for. And we've seen some of those where there have been wins and there have been losses, both for Android and for iOS in terms of how they push those and how they actually make sure that users are ready for them.

One of the things that I haven't seen with phone updates in a long time is the ability to actually change back. Right? Which is, you know, sure, complicated and hard to do, but that would be something that I think would be helpful for a certain segment of users.

James: Well, I mean, you mentioned agency. I mean, agents-- Or not agents, but LLMs, oh, my goodness. We crush on LLM personalities and we get upset when they change. I think that's one of the really interesting Progressive Delivery frontiers, forgive the pun, is that, where people are like, "but I actually like that other personality."

Adam: Well, sometimes to their detriment. Right? Because, I mean, I think that there was, you know. What was the recent--?

Heidi: The obsequious nature of them was-- Yeah.

Adam: Yeah, we're removing this one because it actually led to poor mental health for a number of our users and the number of users that were just like, "yes, but I enjoyed my poor mental health because the robot spoke to me nicely."

Heidi: I think that one of the things that's interesting is we as users have very poor delineation between the different types of software that we're using. It's just sort of all our life. And in a more corporate environment, you know more about where you are. And I think the agents and AI is making that significantly worse because we're poorly stitching together a bunch of things with underthought middleware that does not indicate where information comes from or what it's doing.

And I think that that lack of visibility is sort of the goal of Progressive Delivery in a way that we want people to not have to think about like, am I in this little section or this little section? But it's also a real problem because, who gets the blame if something doesn't interoperate properly?

Adam: Well, I think it's not just about blame though. This is where I think we were very specific or you know, when I was talking about radical delegation and why I felt like the term fit was it forced me to also articulate the distinction between delegation and abdication. Right?

James: Mhm.

Adam: And for those of you that you know, like, haven't, you know, kind of explored that or you know, don't know what I'm talking about, the greatest difference is one of responsibility.

When you delegate to someone, you maintain the responsibility for that outcome. You are just allowing the control to that individual for how to get there, but you're still responsible for the overall success or the overall outcome, whatever occurs.

As opposed to abdication, where you are basically saying, I am removing myself from this, it is now your problem, you are now the responsible party. And I think that this is where we've seen unfortunately a lot of abnegation in the context of relationships with the user, from software vendors, where it's like, "oh, I'll give you that control, but now you're responsible."

Heidi: Mhm.

Adam: As opposed to thinking through and actually paying attention to what are the possible outcomes and how do I ensure that the user doesn't end up in a place that I can't help them recover from or they can't recover from on their own. And I think that that's the difference between, you know, where does that responsibility live?

If I give you the option to update your phone and it turns out the update bricks your phone and I say that's your fault, that's not really delegating to the user.

Heidi: Yeah, I think an interesting way to think about that is the rise of age verification. The government is abdicating their responsibility for what happens online. And like they say they're delegating. They're like, okay, platforms need to make sure that users are protected from seeing things they don't want to see. But the way they are enforcing it makes it impossible to do it in an ethical way because it's one size fits all.

It's not understanding the nuances of different communities. And the people running the sites-- Like the big one coming right now while we record is Discord. Like you can either have a adults discord with verification of your identity or you can have a teen discord, but you can't be an anonymous adult. And that's a real problem. It's going to cause a lot of harm. But because the people making the rules don't understand the nuances of that or maybe do and just want to enforce their views, we're going to have a lot of people more in danger than they were before.

Adam: Yeah, I think that this is a topic for another episode, which is, frankly, technology has moved faster than our governments know what to do with.

James: 100%.

Adam: And our systems of government and process and rulemaking has not kept up with the pace and the reach of technology.

Heidi: Absolutely.

Kim: Well, okay. To be fair, Adam, and I think this is called out in the book, it's going faster than our culture can handle.

Adam: True.

Kim: Another potential name: Jerk. We talk about this in the book, right? I think we all have a bit of whiplash of how much has been bestowed upon all of us and are we ready to accept it or not?

James: Jerks driven by jerks. Is that--?

Adam: Yeah, I mean, sometimes. Sometimes not, right?

James: No, no, that's true. But we just-- Yeah, there are definitely some jerks in the system right now.

Adam: Oh, there are definitely some jerks. Don't get me wrong. But I would argue that the greatest technological jerks that are felt by users are done with, maybe not altruistic intent, but at least non-intentional desire. People do not want to make things horrible for their users, by and large. They recognize that the users are their source of, whether it's revenue or, you know, kind of awareness or usage, however, you're measuring the success of your software and your service.

But I think that for many software vendors, there's an isolation that happens as you get deeper into the software that you're building that actually takes you further away from your user base. And I think that there's also the challenge where the people who talk the most about technology are the ones that use it the most. And therefore they're the ones who are most ready for things to move forward or change or ask for differences. And therefore the voices that you hear are from the people that are most ready for change.

That doesn't mean they're always ready for all the changes you throw at them, but I think it means that they're the ones that are kind of pushing for things to move faster.

Heidi: I think that's true. And I think that every culture has neophiles and slow adapters, and we need both of them. I was reading an interesting thing that says, like, picky eaters are essential because they will tell you when something is only mildly toxic and they will keep driving, like, less bitter.

It turns out Brussels sprouts 50 years ago were really actually very much grosser than they are now. I'm like, I like Brussels sprouts. Why did everybody hate Brussels sprouts? Cause they're different Brussels sprouts. We've kept breeding them so that they are less nasty.

James: Yeah, no, they're amazing now.

Heidi: Exactly. But in the same way, I think that we need both kinds of people. We need the enthusiastic early adopters, "I will cobble this together on a raspberry PI and a dream." And we need people who are like, "What? My phone is five years old and still works. I replaced the battery. Thank you. Don't touch it."

James: Yep.

Adam: Yeah. James, I'm interested in your take on the notion of technological jerk as you talk to a lot of large organizations. And I think one of the things that I would love for you to kind of talk about a little bit to give some context to our listeners is how large organizations are grappling with this increase of software everywhere and the increased rate of change.

James: Yeah, it's such a great question, and I think it's beholden to all of us in this game is to understand there's different rates of change that you've described because--

It's so easy to assume that everyone is living in the future, but they're really not.

So Progressive Delivery as a concept, one of the things, and you know, the Third Loop, once you include the Third Loop, it becomes very clear that, or I guess overarching there is that software delivery is inextricably connected to product management. Which sounds obvious, but let me tell you, go into most retailers, banks, telcos, they don't have a sophisticated product management culture and they don't have a sophisticated software delivery culture.

So in talking to organizations that are fairly traditional enterprises, there's a big delta between our sort of desired state of good and what their reality is in delivering product. I'm talking to an E commerce company right now about their journey to progressive delivery. And then they're not using feature flags. You know, they're like, no, we have canarying, so we're most of the way there.

And you're like, okay, who's the user? Are you understanding what things look like for them? Just a very fascinating conversation. And they're like, the user's the developer. And I'm like, okay, from the platform perspective, that Is true. But what about the user of the developer? Right? Who are they building for?

And yeah, we're not terribly sophisticated and this is the whole reason we've done what we did. Why did we write the book? To get people excited about closing that Third Loop, thinking about the user using some of the modern platforms and modern tools, modern methods and so on so that they can get there. But no, I mean we're still so far from good.

You know, me as a consultant, I remember sitting down with one of Google's software leads and they literally looked me in the face and said, "oh well you know everyone has automated builds now," and you're like--

Heidi: Hahaha!

James: God bless. You know, you maybe don't get out much. Because they do at Google, but yeah, no, that was not in any sense the state of the art. And there's so much sort of low hanging fruit stuff that needs to be automated. And yeah, the idea that everyone has automated pipelines across all the software delivery they do is just not true.

So I think we've all got a lot of work to do and that's where the four of us wrote the book. That's why we're doing what we're doing is saying like let's reset what "good" is. Let's understand that there are these incredible new platforms that previously were not available. There are a set of practices that are amazing that people have established. You know, you've got the tooling, you've got the platforms, you've got the practices. Let's start this.

And let's also not forget that I'm sort of being slightly skeptical about like our enterprises ready for this. I mentioned an E commerce company, you know, hello, we've definitely got the companies we would consider the most sophisticated users and builders of technology on the planet and they will roll out config changes globally without being able to roll those back consistently and easily.

So you know when the Internet falls over and you're like what? I mean companies that should know better-- But yeah, so the hyperscalers, they make stupid like network changes too that get propagated across the entire network. So we've all got work to do.

And enterprises shouldn't, I don't think they should feel, you know, oh my God, you know, the Challenger bank is so much better than us because, or oh my God that you know--

We've just all got to get better at serving our users, the Third Loop. The most important thing isn't really the competition, it's delighting the customer and building amazing things for them. But yeah, there are so many lessons we can learn and things that we could do that will help us to better serve customers.

Adam: Yeah. And I think that like a subtle aspect of that is the thing that we talk about also in the book is the notion that when you say get better at serving the customer, a very important aspect of that is recognition that not all customers want the same thing at the same time.

James: Mm mhm.

Adam: So how do you create systems and services and applications and features that do a better job, you know, it may not be perfect, but do a better job of actually meeting the individual customers where they're at?

James: Banking is such a good example there. Like we finally have got rid of the tellers and what a fuck up that is for so many people that just want to go into a bank. Like the idea that I can't go to the bank to do my banking. Now some people love telephone banking. That was a big innovation like in the 90s or whatever. This is amazing. I can call my bank and get my banking done.

But then the idea that everybody wants to do their banking on an app, that's just not real. That's magical thinking. That's not putting the user first. By all means, if you can find a journey for them to feel comfortable banking online, using an application, great.

Heidi: And be as safe as in person banking where you know your banker.

James: Right? So I do think that there is a sort of responsibility, to your point, Adam, about delegation comes with a responsibility to think about the user and to think about accessibility. I don't think we talk much about accessibility in the book. That's definitely an oversight.

Kim: That's definitely something we're going to address on this podcast.

James: Yeah. I think I'm sweating now feeling like, "why did we not."

Heidi: We discussed it some and we decided that we needed to be more focused. But we are going to get to it because universal design and accessibility are really very core to how it works.

Kim: Yes.

James: 100%. But the universal thing is interesting and yeah, look, we need banking, health care, government services, everything to be as accessible for everybody. And I think that's where offering different affordances and giving the user control of when they're ready to make that change, to offer them journeys, it's so important. And again, that's Third Loop thinking.

I think the respect for the user and how they want to engage with you is so important and modern business doesn't really do enough of that.

And government too. Government's like, oh, everybody uses the Internet now. And it's like, well, I don't know about that. Health care, the UK, the NHS has done a pretty good job. You just dial 111 now and you can get health care advice.

Adam: Yeah. Don't talk to the Americans about health care.

Heidi: Yeah, we're having a feeling.

Adam: Hahaha!

James: Yeah. The NHS is not perfect, but the idea that I can just call up and get real medical advice for low-- You know, it's not 999, it's not an emergency, it's just this is worrying me. It's a phone call. Sure you can use the Internet for this, but most people are pretty comfortable. And maybe not everyone, but, but yeah, a phone call can be a really super important affordance.

And yeah, it's pretty amazing. Like literally any time, day and night, you're like, something's worrying you, it's not an emergency. One, one, one. And yeah, some of it's automated, some human. But if the answer was just, "use an app," that would be a bullshit answer.

Heidi: Yeah, yeah.

Adam: And unfortunately that's, I think all too often in the States, you know, because these systems and services don't exist, we've got issues where people are using ChatGPT for medical advice.

Heidi: Mhm.

James: Well that's gonna happen anyway.

Heidi: But we are not going to solve the world's healthcare problems right now. Were there any other names that we considered?

Kim: Honestly, James, I think you had the most reasonable. You kept calling it The Pod, the Progressive Delivery Pod, which, hilariously, I think we had ideas for book titles and you just kept like, "say what it is."

James: Yeah I think there's a lot to be said for that. Yeah. The power of giving it a name. That's how my company worked. You know, when RedMonk was founded, it was founded on based because I worked out what the name was and I went to my now business partner of a couple of decades and was like, "dude, we're doing the company now."

And he's like, "why now?" I'm like, "well, because I finally worked out what to call it."

And yeah, the magical power of a name. So when you've got it, say it again, keep saying it, get people to hear it. So yeah, I'm a simple, simple man that believes in giving it a name, repeating the name. And so yeah, this was a definite discussion we had, but I like Third Loop.

Adam: And I think Third Loop, when you think about this podcast being the follow on or the extension of the discussion that we started with the book--

James: Oh my God, we're going to have so many incredible guests.

Adam: Yeah. And I think that the Third Loop is really where we want to make sure that we're continuing to focus this conversation.