Ep. #5, Free, Like Puppies: The Real Cost of AI Code

Adam Zimman: So this episode's topic, I think I was the person who said we should do this because as the team member that is resident in Silicon Valley, I think that I was looking for some different perspectives maybe that could be brought to light with regards to the kind of current and future trajectory of AI.

Because I've been to a number of conferences and meetups recently where I'm surrounded by individuals that are so over the top, enthusiastic. They are the kind of multimillion token per day spenders swearing that AI has, we have arrived and you know, I've heard the statement on more than one occasion, "code is now free."

Heidi Waterhouse: Haha!

Adam: To which I always come back to yes, except for the cost of the tokens. Now I will say that I did see a post recently where a team was actually looking to hire junior devs because they realized that they could accomplish certain tasks cheaper paying a junior dev salary than having the token cost associated with a senior developer doing those tasks with AI.

So we have kind of started to come full circle. I feel that there is a recognition that bottom line cost does come into play, whether you are using humans or the robots.

Kim Harrison: But they want the junior dev babysitting the AI.

Adam: So they have realized that the junior devs do not use as many tokens because they don't have the same sophistication with regards to all the agent, multi agent coordination type stuff and that they will have the ability to be able to double check the output without wasting the time of the senior devs who can be working on more complex problems.

Heidi: And this is even before we've hit the end of the AI subsidies. Like tokens cost, but they don't cost as much as they should yet.

James Governor: You know, we just had GitHub move to usage based pricing and everybody's freaking out. Haha!

Heidi: Yeah.

James Governor: Oh my goodness. People are all like, "wait, what are the subsidies now? No!"But there is absolutely no doubt. I mean, Anthropic has had a pretty bad few weeks. OpenAI is in a very interesting position. We've had articles in the Wall Street Journal talking about "They're no good. Oh, dear me. We had a terrible Q4 because of," as the Wall Street Journal said, "massive growth in Gemini adoption." They had to drop Sora. So yes, Heidi, code may be free, but tokens most definitely are not.

Adam: But I also, I do want to get the perspective from you all in terms of stories that you're hearing of success. I still get the impression that a lot of the success with agents and using AI, it still feels a lot like motion and not as much like progress.

I mean, I think that there are people that are making progress that are actually using AI effectively. But I feel like the vast majority of people, they're still excited in the "I can play with something and create something cool," but not so much "I can play with this and create something useful."

James: It's a great question and there are many opinions, as you say. There are the absolute true believers that live and work south of Market or in King's Cross in London, which seems to get more and more exciting by the minute as a sort of a AI hub. And you know, then there are the people that use Bluesky who say that AI does not work at all, that it is a stochastic parrot, that AI has and will never have any value.

And that's without even getting into questions about all the other stuff they don't like about AI, such as the people that use it, such as the energy usage, such as the questions of copyright and the fact that the AI bros are explicitly saying we are here to get rid of all employment. Those were all the things. So yes, there were those folks who are not so AI positive.

But there were also someone like Mike Masnick is on Bluesky and he's a pragmatist. And he's like, "I want to rehost my website and this is how I did it."Or in fact Kate Holterhoff, my colleague, her benchmark for AI is indeed I would like to rehost this website. And she has an academic website that our dear, dear friends at Heroku some time ago said that could no longer be free. So she's always on a journey to re host that platform and we'll use AI to sort of do that.

Heidi: There's Cat Hicks who's doing a really interesting educational series on how to learn AI and how to learn how to use it.

James: Mhm. But yeah, so Rich Holman, good friend of mine, got a small consultancy, WordPress consultancy called Dogwonder and he decided to use this new Claude design tool this weekend. So it was an interesting experience because it actually did a good job. He gave it a design language, and he said it did indeed speed up the, the design of a nice looking website, refreshed the website and did it in a very short space of time.

And he said well that was great because it took about you know, a lot less time. On the other hand, getting it deployed. Oh boy, that took like at least five times as long trying to work out how to get the thing deployed. I think that I guess what I would say is there are different buckets of people. There are some people that are kind of living in the future and not thinking about the consequences in all cases.

And you know, sometimes they're having their production databases deleted and then being surprised that the root provider was doing the backup volumes in, in fact, the same place as they were doing the most recent backup. So there are the true believers. There are people that are like, I will never touch anything created by AI. Kill it with fire. And then there's the great sort of middle.

Heidi: All right, I want to back up. We titled this Free as in Puppies. And I want to back up to where we came up with this phrase. And this is many years ago when we were having a build versus buy discussion. And there's a common open source thing that when we talk about how free software is like, how is free and open source software, is it free as in beer or free as in liberty or free as in lunch, which doesn't exist? What kind of free is it?

And I'm like, well for the most part all build projects are free as in puppies. You can get free puppies by the side of the road, but to own a puppy you have to invest in vet care and you have to accept the fact that it's going to destroy some stuff in your house.

Kim: Potty train it.

James: You have to clean up a lot of shit.

Heidi: Yeah, you have this enormous time commitment.

A puppy may be free initially, much like software where you can adopt it for free, but if it is not something that you are paying for as a service, then you are responsible for potty training it, for keeping it inside and outside and safe, and writing the documentation and doing the updates and all of the work that comes with running software.

Adam: I'd just point out, Heidi, I'm still looking for the open source project that actually is good at snuggling.

Heidi: Yeah, I know. Wouldn't that be great?

Adam: I haven't found that one yet.

Heidi: Haha. I think this applies. Like, we need to have that foundation before we talk about why AI is like, is code free? Well, no, because like, okay, we've designed a website and through much trial and tribulation we've managed to deploy it. How updatable is it? Like, how consistent is its use of XML and markdown so that you can go in and change things once it's up?

Adam: So I think this is actually, this gets to the root of the conversation that I was privy to where I kind of brought up the fact that we should talk about this. And one of the assertions that was being made by some pretty senior developers and experienced folks was:

Are we kind of reaching the convergence point where if something is broken, we are not actually having a need for an experience or a learned engineer to like evaluate, assess, fix, but instead we're just having giving the direction to the AI to go and fix the thing that's broken and recognizing that that may mean that it's completely rewriting the code in a almost one shot capacity to be able to fix the thing that was bugging you?

And it goes back to this idea where code in the context of a human is actually writing the syntax is I think the argument that was being made of why.

Heidi: Uh-huh.

Adam: You know, and I think that this was being extrapolated from we don't have humans writing machine code anymore. Right?

Heidi: Some.

Adam: Some. Very few that would, I would argue that we do still have humans that actually have to decipher machine code, but very few actually are writing it, especially for any type of like production usage. But I think that there is that notion of like, is this the next iteration where computers or robots are going to be writing the code and it's just the humans that are going to be continually refining the definitions or the criteria for that code.

And I don't know I think this is where it, it is an interesting idea that like, do we need to just understand what the problem is, or you know, what the needs are of the computer. And are we getting to a point where the agents or the, as Kent Beck likes to refer to it as, is the "genie" good enough at figuring out how to make the bits do the things that we want them to.

James: I think that's exactly right. Intent is super important. But to this question about the puppy thing and the question about like, what we put into production and whether we should put it into production basically what you're doing with AI, as opposed to more traditional software development, is you're just generating a huge number of puppies.

Kim: Yeah.

James: And if we put them into production, then I'm in danger now with my metaphor of talking about killing puppies.

Heidi: Puppy murder?

James: We don't want to kill puppies.

Adam: Right.

James: So there is a question about like, what we put into production, because once it's in production and it has users, then there are a whole set of considerations that come into play where indeed the puppy does need the care and feeding.

And so the beauty, I think of Progressive Delivery, thinking where we're going to understand the system in the context, having tested it thoroughly before deployment and making sure we had an automated system for the build and getting it out there to a managed cohort to understanding that behavior, I think that we will ideally be killing fewer puppies. And that I think is inarguably a good thing.

And so when we look at the core principles of Progressive Delivery, yeah, it is about understanding what has value and what we will need to sort of maintain over time and understanding its sort of value in the context of the user experience. So yeah, less puppy deaths, people.

Heidi: Well, so let's think about another very common metaphor that we use. We say, "cattle, not pets." Like we take care of the herd as a whole, but we don't have attachment to the individual program. We have attachment to the system. And as these cattle move through, they don't have names, they're easily replaceable. This is sort of the container theory. Right?

And this sort of feels like what Adam is saying about if a program is misbehaving, why don't we just reinstantiate it? Like it does not matter as an individual, it only matters as an instance.

James: Absolutely. But the fact is that I think applications have character.

Heidi: Is that desirable?

James: I mean, it just is what it is. Just look at how attached people get to particular models. OpenAI changes the model. It's just a fricking model and they're like, "what did you do to chatty?"

So we're sort of, in our applications, we absolutely become very attached to-- And this is big part of the why we wrote the book about Progressive Delivery sort of understanding, people get attached to particular experiences and you change those experiences. They get upset about that.

So I think applications, I mean I think the cattle versus pets, we were really talking about the infrastructure. I think applications, once users use them, are rarely cattle.

Heidi: So we're making them real like Tinkerbell, like once we believe in them then they're real as a user.

Adam: But I also think that like this is something that is an interesting trend that I've seen with a number of my AI optimistic friends. You know, one of the common things that people are using agents for and are starting to like do is they're using AI to actually act as like a centralized controller for a lot of things. You know, I think a common example is the like IoT or like home automation where you know, you think about it and like we've come a long way.

There's so many IoT products on the market and the reality is that there's like an app for every single one of them. And a lot of people are just like, "I don't want 47 apps to be able to have my home function the way that I want it to. It would be great if I could just have like a single chat interface."

So like I think of like Adrian Cockroft, we were talking at a conference a few weeks ago and he's done an amazing job of like having everything feed in to-- He can now created like an iCloud account for his agent and he just has iMessages chat back and forth of what he wants it to do and it is connected to all of the home automation in his house and it will just go off and do the thing and he doesn't have to think about all those individual bespoke apps.

Heidi: How is that different than like seven years ago, my dad yelling, "hey Google," and getting all of the lights and everything to change? Like that didn't take AI.

Adam: Well, I don't think it took AI. What it took was that I think he had some systems that didn't have that interface with Google or with Apple Home. And so what he used AI for was, he told the agent, "hey, go and build this interface for me."

Or, "go and make this work." Right? So it was that gap technology of you know, automated automation. Right? It's like how do you actually make the things stitch together the way that you want them to as opposed to having to wait for the people who built them to get around and give you that functionality.

Heidi: So glue code.

Kim: Yeah. So I think this is interesting to me. Are we talking about AI or are we talking about automation?

Adam: I think that it's a good question. And I mean I think this is one of the assertions that I've made previously is that--

AI, at the end of the day, it is automation. It is using computers to do things without having to do them manually. It's more sophisticated than kind of programmatic automation that is hard coded. But it is still, at the end of the day you are using the computers with predefined patterns. It's just patterns that you didn't define in it's ones that were defined by the model or that were assimilated by the model.

James: So we have a couple of things. You know, as Heidi mentioned, we had home automation. There actually probably was some AI in that. It just was not of the LLM--

Heidi: I think there was some ML.

James: Yes. I think we now, so we tend to use "AI" and "LLMs" quite interchangeably. And what I do think and where this is an area for the industry at large to get sort of its arms around is certainly it's about deterministic versus non deterministic behaviors and automation.

And so if you're using LLMs to do everything, you are sort of brute forcing-- You brute force everything, you sort of try and drive some deterministic behaviors into the system. And you know, when we look at Claude Code, it's hilarious because using regular expressions for sentiment analysis in the code base and you're kind of like, "what is going on here?"

But yeah, I think the question is really about deterministic versus non deterministic automation. And if we rely entirely on LLMs, then we're always in that world of non determinism or less deterministic systems and costs go up, the puppy gets more expensive and we should be thinking about, "hang on a minute. In order for the puppy to be sustainable, we should be at least-- Yeah, like whether is AI automation, I think you're right, Adam, it is. But for me the critical question is are the applications that we're deploying, are we taking advantage of sort of the non deterministic style AI in the right way or are we just not only burning tokens, but just creating extremely inefficient systems."

Adam: Yeah, I mean, I think that this is the thing where it's like there are times where non determinism and the way that current models are built is extraordinarily advantageous. But oddly, a lot of those have nothing to do with building an application.

Like I think of this in the context of more on the bio side of the house where looking at using large language models to figure out what types of drugs should be explored or should be actually synthesized and tested or looking at it from a perspective of better understanding of protein folding and like these are things where it's like the, the numbers are so large with regards to the possibilities.

Having a non human automated system that runs through these variations in a non deterministic way is actually leading to some really amazing outcomes. But when I think about this in the context of systems automation or even like predictable outcomes from a user expectation, I do think it's harder to see how non deterministic behavior is actually being helpful in--

Other than like. it really is like Gene Kim and Steve Yegge in you know, their Vibe Coding book, they talk about, they equate it to like pulling the slot machine, right? Pulling the lever on the slot machine. And I think that the question is: Is that good? I think it's correct in terms of an analogy, but I don't necessarily feel like that is the most efficient way to get to the outcomes that we may be looking for.

James: Well, it's possibly good every 150 times.

Adam: Or depending on the odds on the machine, that could be 2,000 times.

Heidi: Or every 3 times. Intermittent reinforcement is absolutely what we are wired to respond to. This is why gambling works because as mammals we're like, if something is available all the time, if it's consistent, we're not excited about it either way. If there's a reward every time, we eventually get tired of the reward. If there's not a reward, we'll eventually stop pushing the lever.

If there's an intermittent reward, we will push the lever until we fall over from exhaustion. And this is what non deterministic software building feels like to me. It's like sometimes you get a great hit, sometimes it is amazing, many times it is not. But because it is so dopamine inducing, we'll keep doing it. And I feel like there's a real troublesome overlap with gambling in non deterministic building.

And I thought this when I was Watching Steve, I'm like, you are so excited about this because the reward is not predictable. Like software wizards have existed for my entire adult life. You could install Lotus123 with the software wizard and it would just walk you through the steps and say, "where do you want to install this? How do you want it? Like what do you want?"

And I would get a predictable outcome but saying like, "hey, fix how my lights work." Might work, might not.

James: Yeah.

Adam: And look, I think that like even Steve came a little bit full circle and was just like, had this realization with one of his more recent articles of saying that like, hey, look, the problem with this kind of addictive behavior is that it's harder to step back and touch grass and breathe, and you end up in a situation of rapid burnout, potentially faster, even though the robot is supposedly doing more work.

Kim: Mhm.

James: And again, I mean which brings us back to this, as Heidi says, the question of sort of novelty. What is sort of interesting is quite often the novelty. It makes us grumpy. You know, we see that repeatedly in software. Bluesky introduced a carousel for photographs the other day and everybody was hating on the devs for ages afterwards.

And it was it's always interesting the degree to which people will get angry about things they get for free. So to your point, we sort of do want to make our users grumpy from time to time. I think that's one of the questions in Progressive Delivery. Progressive Delivery is not about like, we're never going to change the system because some users like it, but being able to enable users to have some, perhaps some control.

It was very interesting to me. A lot of people were like, could you not have given us a toggle to turn on this carousel or not? There was a feeling of like, why is this not a feature? Like you might have had a feature flag, but we wanted to be in control of this. Now I think a lot of people would probably move to the carousel. I don't hate the carousel, but--

Heidi: I think the safety people had some interesting points about carousel, but yes.

James: That is quite interesting to me in that I evidently managed to miss that part of the discourse. So yeah, I don't know whether you want to fill us in, Heidi?

Heidi: Just that it's very difficult to evaluate any feature on a safety basis unless you have somebody who has basically been in the trenches of safety. So what they said and I would never have thought of this is that the carousel enables people to put nasty surprises in the third and fourth pictures.

James: Mm.

Heidi: And basically if you see a thumbnail, it is much less affecting than if you get a full size picture of something gross. Right? I would not have thought of that. But you know what, there are people who have been doing safety on the Internet for a long time and they are aware of "meantime to dick," which was a Second Life metric.

James: Yeah.

Heidi: Like how long did it take a user base to start drawing dicks on things? The Lego random build people found out the hard way that that is still a human impulse.

James: Yes. I mean there did used to be more dick on main on Bluesky than there is these days.

Heidi: Mhm.

James: I can see the carousel being used in that context. But anyway, thank you for that explanation.

Adam: Haha!

James: But we are indeed still on this path. Okay. We want things to change. That makes us as humans engaged. But we want some control of that process. We do want to understand how that looks. Am I just a shill? I think I'm a shill for big Progressive Delivery. Haha.

Adam: Well, I mean, look, I think that one of the things that I've been kind of contributing to the conversation, and maybe I have the same problem as you, James, is that like I've pointed out to a number of, senior devs that, that are on this "code is free" bandwagon.

I'm just like, okay, well if code is free, then don't you think you should be spending more time talking to your users and understanding what it is that they want? And oh, by the way if you were to actually use your free code to be able to like start doing a better job of monitoring and you know, thinking about observability of how users are using your system and you can start to do that instrumentation with zero or low cost in addition to, whether you're using feature flags or some other type of personalization to be able to provide people the experience that they are going to be most satisfied with.

Like, why don't we use that free code for the good of our users as opposed to just--

Heidi: Feature factories?

Adam: Well, yeah, I mean, how do we measure our engineers? How do we measure our productivity?

Kim: Okay, so this gets on a question I want to ask. What is the future of the developer if we have AI moving in and changing how they do their work? Because everybody has feelings about that. And the truth is I don't think it's going away.

Adam, I think you're getting at something that I want to hear you all talk about. What is the future of the developer? What should they be focused on? What should they be doing? Are they feature factories anymore?

Adam: This is something that I think is a existential crisis for I think a lot of senior devs that are heavily realizing that AI is really useful to them. You know, these are the folks that are doing like multi agent coordination and they've got it where they're know, blowing through their backlog of all these things that they wanted to build.

And I think there's that realization of, "okay, well, at this point in time, like all I do is kind of give direction." You know, it's almost like a more of a managerial role than an individual developer type role.

But I think that you take that a step further and you say, hey, look for a lot of these things that you may be building, they are in service of the user in the sense that like the user has requested them, the user has indicated this is something that they need to be successful with your application. You know, it begs the question of like, well, if you just create the one feature that allows the user to say, "can you add this to the software," do you need the developer there?

James: I think maintenance-- It goes back to what do we say at the beginning of the call are there still some people that are writing Assembly, right?

Heidi: Haha.

Adam: Well, interesting metric on that is that I saw there was an article, I think in the Register a few weeks back stating that like, over like high 80s percentage of projects that were started to be able to move from like Z series or mainframes to x86 using AI have completely failed and been abandoned.

James: Oh my God. Yeah, there was a huge amount of garbage about that. And when Anthropic was talking about that, I'm like, "you have no idea how mainframes work, if you think this is going to be a thing."

But you know, this idea that, oh, we'll just have a very smart product person and they will not need any developers because they'll just like prompt stuff and then everything will be nirvana. The puppy needs to be looked after, there is maintenance there.

And yeah, as soon as you have users then you have to maintain those users. And there has to be some consistency of behavior. And it turns out that the people that actually know how shit works are called software developers.

And so whether It's Adam talking about, like. And I'm, I'm very interested in the story of juniors being back in fashion because the tokens are too freaking expensive. But yeah. Are we going to say that observability no longer matters? Are we suddenly just, "Oh, we could just get the machine to make a new one. Well, just give us a new puppy and I'm sure we'll love the new puppy."

Adam: I mean, I don't think that that negates the need for observability. You still need something that's doing that, like monitoring or reserving of--

Heidi: But who's acting on it?

Adam: And I think that this is, you know again this is the current state versus future state. Right? Like, I think that optimists' future state is, well you are having the agent proactively monitor and then if there's an issue, it's going in and coding a new solution or fixing it and to meet the parameters that are defined by, you know, Jack Dorsey, who apparently is going to have 6,000 direct reports--

Heidi: Haha.

James: Oh my God, I mean, can you imagine wanting that? What an extremely weird aspiration. "Please give me 6,000 direct reports."

Adam: What could go wrong? And each one of those direct reports, obviously will have like, a flotilla of agents that are doing all the work for them while they sip Mai Tais on the beach.

But to my point though, Heidi, is there a future state where we do have some type of automated kind of remediation.

Heidi: Self healing systems.

Adam: And I say this as someone who, you know, has been to more than one keynote over the past almost 30 years that have talked about self healing systems, haha, only to know for a fact that on the back end, that was a lot of Photoshop. Haha.

Heidi: There ain't no such thing as a free lunch.

Adam: So, I mean, I think that I have seen the demos, but when you pull back the curtain, you know, the reality is, there's still people and we're not--

I would argue that we are able to operate at greater scale than we were 10 years ago, 20 years ago. But the reality is that as that scale increases, it doesn't negate the need for humans. It just means that the humans have additional tooling to be able to achieve those scales.

Heidi: I think that may be correct. I think one of the things that I'm finding interesting is how we measure value. Evidently, because I don't know, tech bros have never read a book, they thought it would be a good idea to set up leaderboards for who was burning the most tokens. several companies thought this would be a good idea.

Adam: Yes.

Heidi: It is not a good idea.

Kim: That's something to show off?

Heidi: Yeah. Well, because they're engaging with AI so much more than like-- "Are you learning AI? Are you using it? Well, I guess we'll incentivize token usage."

Adam: "We'll measure engineering productivity through token usage."

Heidi: Yeah.

Adam: Because obviously lines of code is so passe. Haha.

James: Yeah, it's really, it's really bonkers.

Heidi: It turns out that's a terrible idea. Who knew?

James: Mhm.

Adam: Anyone who has watched a childhood cartoon where a cartoon character is shoveling money into a furnace probably knew that that was a bad idea. Haha.

Kim: But I could automate that. Haha.

Heidi: But I think that it's a really interesting reflection on the problem of if we can't tell because of a shared project how much you're accomplishing as a developer, how do we measure your output? And it's a really Taylorist way of thinking.

Adam: Well, but I think that it's not only about measuring output. I think you brought it up. I t's measuring value. Right? Because I think that there's a difference. Right? And I don't think you're saying there isn't.

Heidi: Yeah.

Adam: Again, I bring this back to why I wanted to have this conversation on this podcast. Right? We're talking about the third loop. We're talking about this notion of user adoption. And typically speaking, not always, but typically speaking, user adoption has a direct correlation with user value. Right?

People aren't going to adopt something that doesn't actually do something for them, at least not willingly. Right? And so when you think about this aspect of user adoption, this seems to be something that still isn't necessarily part of the conversation and the context of whether it's AI generated code or AI tools, ss we've seen with the significant amount of backlash that comes from large user bases where AI is being kind of thrown into everything.

Right? Or being used to rapidly enhance that feature factory for things that people maybe didn't really want or need. And so like, how do we balance that and where is that headed?

One of the things that I know has come up in conversations for me is this notion of taste or, you know I think that we talk about it in the book in terms of knowing what good looks like. And it's this idea of: Is that the role of the senior developer to be able to articulate, "oh, well, I understand the problems that we're trying to solve, and my experience tells me this is what good looks like. This is what is going to be desirable."

Heidi: Uh-huh.

Adam: Are we getting to a point where can we get closer to the users for a better read on that, a better kind of measure of that?

Kim: In the creative world, that would be a creative director. That would be the person who understands. Right? And can channel the talent, whether that is a machine that automates parts of it or other people that do parts. "Do you have the vision? Do you understand what it is you're delivering?"

Heidi:

One of the things that I find most fascinating about marketing in software is how far we are behind marketing in pretty much every other field. The grocery people are infinitely more sophisticated in their understanding of users. And there's no excuse for it. It's just that we haven't cared yet.

But it's so interesting to see that friction start to really show up. Like, I think that anyone at Clorox could have told you how the Salesforce super bowl ads were going to go, which is to say, like, baffled hostility.

Like, not nobody watching the super bowl by Salesforce, but like, as a percentage, the people who can make a Salesforce investment, and the whole of the super bowl watching public, is disproportionate.

And that sort of weird assumption that everybody cares about what you care about is so interesting when we see-- Kim, we were on the bus in London and there was, like, a Galaxy Phone with AI and like, nobody cared. That was not going to solve any of their problems.

But I've been in the meetings where we pay for a billboard.

Kim: It's a lot.

Heidi: Haha. It seems like a good idea from that side. So when we're talking about what is it that we think AI is going to do and are we listening to users? There's so much taste making that needs to happen before users are ready to accept AI.

And we haven't done that work, and not just for the general public, but even for internal consumers, even for software people, we've said, you should want this. And everybody's like, okay, this is like telling me to buy a gas oven without Julia Child promoting gas ovens. Like, that whole show was totally sponsored by natural gas and what Julia Child could do made gas aspirational.

And in that way we moved a lot of gas stovetops. But until you have that visualization, that notion of how a technology can serve you and can make your chicken better, it's not just resistance, it's hostile bafflement. Like, why are you trying to make me buy a thing I don't need? I feel this way about air fryers.

Adam: Yeah, fair enough. Haha.

Kimi: That's real.

James: No, I'm not going to have any air fryer slander here. Come on.

Adam: I mean, Julia Child would have loved the air fryer.

Heidi: Right, but like, I have a convection oven. I think we're okay.

Kim: I have a convection oven slash air fryer. And I still don't know why I need both features in one. And I think you're right. Why? Tell me why. What is this doing for me?

Adam: Look, I think that this is the interesting thing of the way that AI has been brought to market also, where I think there's general acknowledgment that even the people who are building the models and gaining success there, they don't really know what the optimal usage or use case is for this. Right?

I think that they're kind of inclined to say look, the world is full of nails and we build hammers and you know, just bang on stuff and things will happen. But I think there's the realization that we don't yet know where this is going to actually be useful and where it's not.

I think there's still a lot of exploration going on. So I think that to your point, Heidi, like having a specialist or having a known or recognized expert that comes in and says, "this is how you use it in a way that is productive and repeatable" is what I think we're kind of looking for, waiting for. I think the challenge there is what we were talking about earlier is, you know, deterministic behavior. Right?

It's really hard to tell people how to use a non deterministic tool in a repeatable way.

Heidi: Yes. And so that goes back to what do we think developers are for? It's for determining what has to be repeatable and what can be variable.

Adam: Interesting.

Heidi: So what is the guide rail that you must put in and say, I need to be able to trust that you will not delete my production database. That has to be predictable and it can be fluffy and stochastic about how it interprets other commands like I want you to build me a website with a purple gradient background.

Okay. There's a lot of different purples, there's a lot of different gradients, but it still needs to have a header and a body and predictable things. So I think we're going to need developers to really sit down and think about what is core to behavior and what is user configurable. And the user can't do that because the user doesn't understand the stack.

Adam: Yeah. So there needs to be a combination.

Heidi: Yeah.

Adam: The idea that like users can ask for the things that they want or need, but there still has to be some type of parsing or rule set in place.

James: Users have terrible ideas.

Heidi: They really do.

James: Terrible.

Heidi: Bless them.

James: User tyranny. It's not, it's not good. Oh God. And we haven't even got to like AI and the, the role of open source maintainers. But one of the most important jobs is to say no. And now you got people with agents expecting that anything deserves to be accepted. You know, taste is kind of saying no. And honestly that's one thing that LLMs are very, very bad at.

You know, they do not like to say no. And you can invariably even if they're instructed to say no, you can persuade them for a reason why they will say yes. So they both love to glaze us and tell us how awesome we are. But no, they are terrible at saying no. And I think arguably, and again, this Progressive Delivery thing, let's understand the user behavior.

But I mean any product that we are building, we need to be thinking about actually saying no to the next set of feature requests. Think about companies. You know, you accept every enterprise feature request and you ruin your product.

Many a good software company has basically become unusable because they accepted every request. And I think that great developers, designers, product owners say no a lot.

Adam: Yeah.

James: And that's, I think, where developers, whether they're junior or senior, should be saying no a lot. And again that's part of the problem with LLMs. They just don't know how. In fact, I think this is my next blog post: LLMs don't know how to say no, and that is one of the ways in which they are deeply problematical.

Adam: Well, I think that this was also directly, I think related to how we talk about automation in the context of Progressive Delivery, as one of our forays where automation-- You know, we talk a lot about the fact, not just automation on the like builder operator side, but automation in the context of the user side. Right?

Like, how are we being as software builders, you know, kind of tastemakers and saying, okay, we're going to automate a lot of the things from, for the user so that they don't have to click 37 times to get the outcome that they want and that we're instead making the smart choices for them, giving them optionality where needed if they need to escape out of that, wizard like workflow that you talked about, Heidi, but you know, having that kind of like easy button for the repeatable actions or behaviors and making that something that like we actually use observability.

We use our, you know, kind of metrics to that over time we're continually converging along that most likely path for the user and making it easier and easier for them to get the value from the system. And I think this is the thing that like, if you were to apply some AI with your observability to be able to look for those types of patterns, to be able to look for ways that you could make sure that you're not asking a user seven times for the same piece of information or the same input, instead you're actually making sure that that context is being carried forward--

Those are the types of things that all of a sudden, like we can actually have more predictable outcomes and use these automation tools to be able to drive better outcomes. But you know, to your point, Heidi, maybe that's machine learning and not AI.

Heidi: One of the things we should be thinking about more is consent. And by that I mean informed consent. So like, do you really want to delete that thing? Like, this is a dangerous activity. How does it know that? We have to figure that out.

But consent should be enthusiastic, informed and ongoing. And that is for both the LLM and for us.

We should be able to say, don't use this. We should be able to say, I'm giving you this set of permissions, don't go beyond them. We should be able to say, I trust you with this. And those are degrees that we haven't yet taught AI to have.

But AI should also have the concept of consent. Like how do you want me to talk to you? How much power do you want me to request? And are there things that I should stay out of?

And I feel like if we gave AI a consent based model, first of all the tech bros would all die and it would be a different world. But second, I think that we would end up with fewer horror stories because we would have more humane checking.

Adam: So you want the inverse of OpenClaw?

Heidi: I do.

I don't see a value in my robots having a social system. I do see the value of them communicating, but there's no reason that they should be doing it as facsimiles of humans. They're not humans. If I could change one thing about AI, it's that it would never refer to itself in the first person.

It would not say, I shouldn't have done that or I'm helping you with that.

James: It does confuse people about what they're talking to. We see the "I" and it-- Yeah, no, A hundred percent. I think it's intentional.

Heidi: Oh, it is.

James: And it's very powerful.

Adam: Well, it also like from a psychological perspective, it actually encourages additional usage and engagement.

Heidi: Mhm. But we didn't consent to have these machines simulating humans. Some people did, and I guess they can turn that on. But I think that as a default, it's very dangerous.

James: Yeah, people use them. If you don't want to use them, you don't have to. People share all sorts of interesting and weird things with robots. Again, because robots don't say no. And whilst I think that it's an important role of a friend or other human, a friend, a manager or parent or coach or employee, we should all sometimes say no to things. I wish I was better at saying no to things.

Heidi: It's so interestingly difficult because we are, for good reason, socialized to go along and get along.

James: Mhm.

Heidi: Because otherwise society would be an endless hell of brawling and spats and bad feelings. But the ability to considerately and thoughtfully say, "would you like a sandwich before you make that decision?"

Adam: Well, the way that I used to put it when I was running product teams is I was the ambassador of no.

Heidi: Yes. Haha.

Adam: My role was to say no in a way that did not upset people. And I think that there's that aspect of it, James, where it's like, how do we make it so that saying no is something that people recognize as, and I think this is like the consent model, Heidi, it's saying no is something that you recognize as mutual respect as opposed to opposition.

Heidi: Mhm.

James: And again sometimes if someone offers you a puppy, you should sometimes say no to the puppy.

Heidi: Yeah.

James: Are you ready to look after the puppy? Are you saying no enough? I guess that's, for me, the lesson I'm taking away from our podcast today is say no a lot and mean it if you want to truly please your users and build sustainable digital systems.

Heidi: Yeah.

I think that in a world where code might be free, we need to consider both the cost of maintenance and the extrinsic costs. Like free for whom? It's not free for people who live next to a data center. It's not free for people paying more in electricity.

Adam: I also think that when things become free or things become automated in that context, I also have a growing concern that they become very homogeneous and everything starts to feel same. And I think that there's a aspect of us as humans that crave variety. Different things work differently, are interesting and useful to different people.

And I think that there is a concern that all of a sudden the more we use the same models and the same robots to build things, all of a sudden things start to get really boring.

Heidi: Sebastian and I were having breakfast at a hotel and he's like, "oh, no, this is the Cisco sausage gravy. It's perfectly acceptable," because he'd had it at school. And I'm like, yeah, Cisco owns like 80% of food service, pre-made stuff, which is the vast majority of what you get at a restaurant.

And so even though they're different restaurants, a hotel, a college food service, it's still the Cisco gravy.

James: I can't do food slop. I prefer AI slop to food slop. I can't be doing that. I'm definitely saying no to that.

Adam: There you go.

Kim: I'm going to drop one in here, because we do need to say no. But I think some of us need to rethink what this change means for us because I think some people are going to have to stretch themselves in new ways that might feel uncomfortable. Is their role changing?

James: Oh, no. You know, are you saying I gotta say yes to things?

Kim: Is your role changing?

James: That I want to say no to? You got me there, Kim.

Kim: What does the future of a developer look like?

James: Oh, you're right.

Kim: Saying no, I think is important. But are we going from feature factories to creative directors, tastemakers?

James: That sounds aspirational. Not for everybody, but sounds pretty good.

Adam: Yeah.

James: Okay. Say no, say yes, say more, say less.

Heidi: Listen to the user, but not too much.

James: Haha! Listen to the user, but not too much. That's the new manifesto for Progressive Delivery. We got it right there.

Adam: Haha!