Ep. #7, The CTO’s AI Playbook with Peter Bell

Joe Ruscio: Welcome back, everyone. Super excited to be joined today by Peter Bell, founder and CTO of gather.dev. Today we're going to talk a lot about CTOs and what it means. Welcome to the show, Peter.

Peter Bell: Joe, thanks so much for inviting me. I'm excited to be here. I've been a listener for a while so it's really fun to be on the other side of the microphone.

Joe: Yeah, yeah. Well there's a lot to talk about today. As some listeners may or may not know, prior to joining the investing side of the table, I spent a first career as a founder CTO. So a lot of the things you're working on are near and dear to my heart and I'm very interested to get your take on them.

But before we dive super deep, what's your origin story? How did you come to become a CTO and found Gather?

Peter: So I built a bunch of companies. After I left university I did a degree in computer Electronic Systems engineering. But I was writing software for a while. I started building websites back in 1997 when like people were just coming out of boardroom meetings saying, "we need a website."

So I built a bunch of websites. Then I kind of got bored building the same thing. So I effectively built something like Squarespace in 1999, a set of domain specific languages for generating custom web applications which, they all wanted content, community, commerce, the same kinds of stuff. I enjoyed that.

I spent a lot of time getting deep into DSLs, domain specific languages, and generating code. I then realized that I wanted to have more impact so I took some roles. I ran engineering at General Assembly (GA), which was an ed tech startup in New York and a bunch of other startups. You won't have heard of. And then somehow I realized-- So my GA role, I think I did a bunch of things right. But I managed to get fired after six months. I'm like, oh, guess I'm not a very good CTO after all.

So I decided maybe if I could learn from other CTOs. So I helped to co-organize something called the New York CTO School for a number of years which was great, but it was a not for profit. And I realized that the basic principle was it used to be in New York. Back in 2010 if you wanted a CTO, you basically got to the point where you raised an A, flew to California and persuaded somebody that there was better places to live than Palo Alto. There wasn't good CTOs out East.

Joe: Yeah, it's maybe hard to imagine today if you were spending time in New York City, but yeah at that point in time in New York City, I mean Datadog may not even been founded. I'd have to check the exact date, I think maybe it was being founded in 2010. MongoDB was a tiny, tiny NoSQL database company. Like there really wasn't any New York tech scene to speak of.

Peter: Exactly. And there was a New York CTO Club which was great and that still continues to exist, but there really wasn't this kind of resource for the next generation of how do you become a startup CTO. The problem was I did that for a while as a hobby, but I realized there was so much more you could do in helping CTOs to learn from each other. And so I tried to turn it into a day job.

I started running CTO summits on the east and west coast and a bunch of other places. About end of 2019, I'm like, hey, this is really working. At the time I was a senior director at Flatiron School, another EdTech startup that had got acquired by WeWork. I built a team of 50. I fixed up and supported the curriculum team in some changes and they didn't really have anything for me to do.

So I'm like, hey, this is a really good time to go full time on running these full day single track conferences in person. That was about a month before COVID. Turns out it was not the best time.

Joe: Yeah, tough timing. So these CTO summits were in person get-togethers where people could share learnings and talk about the role. I mentioned, given that you've spent so much time working with CTOs and helping level them up. The thing that's always fascinated me and particularly as a, I guess the former CTO is almost every executive role distills down to a pretty quantitative set of numbers. Right?

A saying I'm very fond of, talking with our founders, particularly later stage companies, is that a VP is always on a pip, right? And meaning that like when you hire a VP into your company, ultimately you're handing them a budget and they're delivering you some numbers.

And the CTO is sort of a weird exception to that in that it's, I think it's a very fuzzily defined thing. What do you think makes a good CTO? Like zooming way out.

Peter:

So zooming all the way out, a CTO is somebody who helps a company to figure out how to leverage technology for business advantage for that company and for their customers or clients. At the highest level you look at new technologies and you understand them sufficiently well to identify opportunities for the business and then you ensure that you have an organization working with you that is capable of delivering on those opportunities.

Joe: Right. And often you're doing that in, I mean typically not scale organizations where the, I guess the day to day management of the engineering workforce is more than a full time job. You're doing that in a, I don't want to say adjunct capacity, but more of an adjacent capacity, if the day to day management of the engineering orgs, if you have like a global VP of engineering or whatever the title is working with you.

Peter: I run communities of CTOs at gather.dev. and the thing that I've really noticed is one of the big drivers of what a CTO's job is day to day is the size of the org. So for example we have a community for founding CTOs. That's basically you might have just left your job and you're hacking on something. You might have a team of like seven, eight people. Basically you are somewhere between an IC and a team lead and your job in that stage, this is kind of like pre-Series A days, back before you could get a half billion dollar pre-seed raise.

And the idea is that your job there is to minimize cycle time, right? How can you quickly validate learning and iterate into product market fit? That's your job. Then I look at two other broad groups. The next one I call startup CTOs, which is anybody who's moving from a team to an org. They got maybe 8 to 10 engineers all the way up to say 85 and they're building the processes and systems.

How do I hire my first EM? How do I hire a VP of engineering to keep the trains running on time? How do OKRs apply to engineering teams? How do I get visibility into my roadmap so I can start to provide that to my sales or revenue orgs?

And so you're trying to build all the infrastructure and then CTOs at scale, VPs CTOs running orgs are like 100 plus, 85 to 100 plus. It's primarily about alignment politics, repeating yourself for a year and a half till people actually hear the three simple points you said. It's much more about large organizations and directing them gently but effectively.

Joe: Yeah, I think you've hit on something that I've always found interesting is the way the CTO role changes and like what peak CTO performance looks like at the different stages of the business. I mean in particular, I mean I think typically early stage startup CTOs like you were talking about, I mean it tends to be oh, it's the technical founder, I mean historically it was like, oh it's the person who's just like a code creating machine.

And I think the very first transition point typically of early stage founders we talk to is this notion of like as the CTO you need to transition as you start scaling your team from "I'm the person who's in every critical coding path because I'm the cracked, best engineer here" to "how, as CTO, do I make sure I am never in the critical coding path."

Peter: Exactly. Well, the good news is these micro transitions. The first thing is, oh, I'm not the only coder here. So I need to spend some time sharing context with a couple of buddies, a few people who can then deliver and help them to be productive, taking it all out of my mind and making it as shared context, which hopefully if we get to AI stuff, it's exactly the same on the onboarding problem. Right?

If it's in humans minds, if it's around the water cooler, if it's not written down, it's not available to onboard new teammates whether they are human or agentic. And that's really incredibly frustrating having gone through that a bunch of times, like damn it, I could have spend all this time writing down all these obvious things and slowing down my performance to help other people to be slower at writing code.

But the truth is that if you get over that hump, once you've got two, three, four, five, six other people delivering in their own areas, you now genuinely have more velocity than you could have shipped as an individual.

And then I feel like this kind of keeps going and you get to the point where like you know, you get to about 20, 25 engineers and usually that's about the time where maybe you've hired of maybe let's say that you're a hardcore coder, you've hired a VPE to kind of like give you a hand with making the trains run on time, the people management, the org structure, stuff like that. By 25 people I think is time when the VP is kind of silently like, "okay, we're going to remove his GitHub access. Like he gets read access only. And that's."

It's a blessing, it's the right thing. Because at that time your job, it varies by company. It might be to evangelize. I mean I think of people like Charity Majors. I primarily know her from the stories she tells. But that delivers huge value to Honeycomb, if that was the only thing she did.

You might be deeply technical. You might just be really good at building the team and the org structure and aligning the people. But you have to find the things you can do and make sure that you complement yourself with VPs who can do the other stuff.

Joe: Right, right. Yeah. And you just hit there at the end, I think that's the other kind of really interesting transition point which as you said, I think tends to happen a little later stage. There comes a point as a company scaling and succeeding where the CTO actually becomes a very public facing role.

Like in your first description you kind of alluded a bit to helping the org, adopting the right leading technologies and then mentioning also clients and customers. But increasingly as the business scales, the CTO is a critical public facing role both in terms of evangelism, in terms of working with key customers and it becomes in some ways almost a bit of a sales role. What's your perspective there?

Peter: I think it depends very heavily on the domain you're in. Anything in dev tools, dev infra, it's a no brainer. You having a charismatic--

Joe: Yeah, I am biased to be fair.

Peter: Exactly. Like, I mean if you're providing tech to change the grocery stores, like I mean maybe you need to have an occasional partner conversation, but nobody's like, "I shop at Kroger because of the CTO." Right?

Joe: Right. Yeah, that's true. I don't know who the CTO of Kroger is, no offense.

Peter: And then there are others where you effectively become not quite a forward deployed engineer, but somewhere between that and a field CTO where it really is about building trust and credibility with 5, 10, 15 enterprises that are signing multimillion dollar contracts with you.

And then sometimes it's just about being the public and it's more like a PLG marketing kind of vibe where you are just sharing wisdom and people want to buy the tools. I remember I'm thinking about Edith Harbaugh back in the day from LaunchDarkly.

I can't remember how many years and how much money they spent until they finally got the tagline: decouple deploy from release. I feel like that probably cost $30 million in venture money to actually land there because I saw all the iterations as she was going through it. But at the end of the day I wanted to buy from LaunchDarkly because they created the category and they were bringing that kind of wisdom and experience to the space.

Joe: Yeah, they were a Heavybit company. So thankfully not $30 million. But yeah, you're right, it was not a overnight thing. You know, the other really brilliant, and it is related to CTOs and tech CTOs, but one of Edith's really brilliant innovations I thought was show them how to build it.

You know, one of the early things that she did there at LaunchDarkly, was just put out like very kind of detailed posts like, oh, you think you can build your own feature flagging system that performs the same way ours does. Like, here's all the pieces, right? And this is everything. And oh, you didn't realize you needed a CDN for a robust feature flagging system? It turns out you do.

So yeah, I think it's a great point too that even from vertical to vertical, the CTO role can kind of shift in a way that, you know, at the opposite end of the spectrum, I don't think like the VP of Sales role, that the VP of Sales role is incredibly challenging in its own regard. But it is pretty-- It's like, what's the number? Do we close? What's our pipeline win rate?

So gather.dev, we should talk about that a little bit because it sounds like you've had a long background in helping to educate CTOs and even starting to bring them together face to face in your summits. Then the pandemic. So how did that all lead up to gather.dev, and what's the kind of founding principle mission of that organization?

Peter: So I've been really lucky to learn from so many great engineering leaders around the world. I like present at conferences, I ran conferences, ran events, host the O'Reilly CTO Hour. I run the executive summits for CNCF at KubeCon twice a year. So I get to meet all these amazing people. I'm like, what would happen if they got to meet and learn from each other?

And so the vision behind Gather is to say, take these very thin niches: Founding CTOs startup CTOs, CTOs at scale rather than just like, oh, engineering leaders, where, you know, and I love them, but there'll be a bunch of team leads, a bunch of VMs. Some people will be like, how do you fire your first person? Or how do I do a one on one? Or you know, how do I do my performance reviews? And other people are running a team of 1500 at an enterprise and they've just got very different challenges.

So the vision with Gather is that this is the time, right? The good practices aren't changing every year or every decade. Now they're changing every week. I mean, heck, a week and a half ago, the agents didn't have their own social network. Agents can now hire humans. There's a website to do that, and there's even a place for agents to be able to persist themselves with durable resources in case their creators shut them down. Now, whether or not they have the agency to choose to engage with that, like most of those things didn't exist a few months ago.

And so the question is, how do we get the insights from everyone who's way too busy to write blog posts and write books and all the rest, make it super easy for them to contribute, and then load that all up into a high trust context where it can be available within the community and then the good practices can be synthesized and shared outside so that we can improve the way that we build and manage software and build and manage engineering orgs.

Joe: Yeah, I do think we're at this interesting moment that you've hit on. I mean, interesting for a number of reasons, but particularly for the timeliness of something like Gather. Having been around a few loops on the IT innovation cycle, we have these moments and you can think of like you're talking early original.com, I think relevant to our audience like cloud and mobile in the late aughts, early teens. And now again with AI, where the frontier is moving so fast that even if you can get people to write down blogs or record podcasts, like the rate at which those are not as relevant to the latest information and really the only way to really learn at the frontier is engaging.

I mean my last startup was one of the earliest cloud native monitoring companies. We were in San Francisco, and much like today, it was like going to meetups every week and meeting with other distributed systems engineers who at the time are all trying to figure out how to make the cloud more resilient, how to push elasticity with the primitives and build on them. You would learn things. It was fun to go to those.

And I think the same is absolutely, obviously happening with AI right now. And until the best practices really start to emerge and codify, that's going to be the case. We'll definitely put a link in the show notes. I think any aspiring or current CTO should check out. Gather.

You mentioned you have a history with O'Reilly. You're working on O'Reilly book, I believe, which is a callback again, I think. For the younger listeners, there was a time where O'Reilly books were, I mean, still great resources, but the only way my very first programming job was professionally writing Perl, and I had three books from O'Reilly that were the only reason I could do that job.

Peter: No, I started with Visual Basic, ASP classic code fusion back in the day, and a little bit of Perl, which is, as everyone knows, I think write once, read never. You can do way too much with Regex in a single line of Perl.

Joe: Yeah, yeah. And so what is the upcoming O'Reilly book? What's the title? What's the focus?

Peter: So it's called Scaling AI Adoption in Engineering, and it's specifically targeted to CTOs and engineering leaders who probably who've got larger orgs. Right?

If you've got three people on your team, you don't need to scale AI adoption. You're just kind of like, hey, here's my new skills.md. Right? It's pretty easy. But once you start to have 100 humans, 500, 1000, how do you think about managing stakeholder expectations? This is the first time where the CEO is way ahead of the technology.

Like they're on board, the money is available for the tokens, but now you have to deliver, and it's just hard to do that. How do you manage stakeholder expectations? How do you deal with culture? What does hiring look like in the AI world? And what kinds of things do you hire for? And then how do you solve this continuous change management problem?

You talked about the cloud, which was a really good example. Right? Moving from on prem to the cloud like maybe 15 years ago. And it was great. You'd have a center of excellence. You'd have a program manager, a couple of project managers, you'd have dashboards and you'd have these boot camps people would do.

You'd go one repo at a time. You'd have this incredibly consistent process for moving 700 repos from on prem to the cloud over a 24 month period. So now we need all of that. It turns out if you just say, oh, here's a Copilot license, figure it out. That's not enough. That was fine for the first half of 2025. If you're doing that now, you're behind.

But you need to go beyond that to say, how can I create channels of learning and change that will allow the best ideas to naturally bubble up from experiments, to be filtered appropriately and then to be distributed efficiently. You are teaching with a curriculum that changes every week. And how do you create the appropriate mechanisms within a larger org to make that work?

So those are some of the challenges I'm thinking about and writing about.

Joe: Yeah, yeah. And I think one thing you hit on that's super interesting to me or has been over the last two or three years, is this is one of the only kind of IT innovation cycles I can think of where the impetus to adopt and benefit from it is coming so strongly, literally from the CEO, down to the CTO. I mean, historically, it's typically, I mean again, if you look at like cloud or, or some of the other, or virtual machines or other technologies at the frontier, leading technologists kind of see potential advantages that outweigh all the potential risks and they start adopting it.

And at some point you have to go back to the rest of the C suite and say, hey, we're going to do this big migration project and it's going to cost a lot of money, but don't worry, it's going to be worth it. And they all look at you sideways and say, "really? We're going to spend all this time and money and nothing's going to change in the product?"

You know, you just have to like, draw on your political capital you've built up with them. Like, "no, no, you can trust me." And AI is-- And you know, and understandably because it's a tactile thing. Anyone can use ChatGPT. Anyone can start thinking about like, "oh, wow."

But yeah, but it's coming, in some ways almost too fast where it's like, "hey, what are you doing with AI? Like, I need to tell the board in a month that we're AI revolutionized."

Right? How, as a CTO, do you balance that? I guess other than just buying an enterprise wide license for Cursor or Claude or whatever.

Peter: So there's so many things. Firstly, our first takeaway is you got to do it yourself. Unlike I feel like cloud and mobile, you could hire a really good senior director or VP and they would take care of it. And as long as you used good management practices, you could kind of like sniff test it. There were some benchmarks you could check in with people. You didn't really need to be an expert on Kubernetes to move workloads to the cloud. You could hire somebody to own that process.

AI is not like that. If you are not, sure you should have a daily driver for chat, you know, Claude, you can go with ChatGPT, you can go with Gemini, you can go with Grok. There's so many options. You should have a chat, but you should be-- If you're not using something like Claude Code, Codex, augmented code, you know, one of those systems, you're not doing it right.

I finally got off my butt about a week ago and just started prompting Claude Code and now it tracks my projects, tasks, to-dos. It has all the events, it has the SLPs and now I'm starting to build lightweight scripts. I've used it to ingest like 300 CSV files for historic communications.

So firstly, as a CTO, you need to do it yourself. You have to get real hands on experience with the technology. And then secondly, I think you have to be realistic about where your org is.

I was speaking with Laura Tacho, the CTO at DX, just got acquired by Atlassian. And there's no question that if you don't have good engineering efficiency metrics, if you're not looking at Dora, Space, DX, Core 4, whatever you use, you're not going to be successful. If, "great news, we've got 4,000 agents generating code. Bad news, Sami still has to SFTP it to the servers and Bill and the team in Chicago needs to review the QA." You've got to fix that.

And I think you also need to be realistic about the risks and opportunities for your business and commit to a path that makes sense. It's like any other TALC, the technology adoption lifecycle curve, right? Innovators, early adopters, early and late majority and laggards. With the possible exception of laggard, there is a story for any company to pick any one of those lanes as long as they're consistent.

If you run gyms or ski resorts, you know what, people are still going to be skiing in 10 years. Yes, you'd get more value if you adopted AI tomorrow, but you don't have to. If you have a SaaS software, especially like a website builder, you better be adopting AI, otherwise your industry probably won't exist in 5 years.

Joe: Zooming way out. My perspective has always been that if you look at the history of IT, right, over 60 years, it's just a repeated set of innovations that remove whatever the current bottleneck is, increase productivity and leverage. I mean even hence the name of this podcast, right? Like I've always been thinking about, hey, how can I increase the leverage of the humans that are working in this organization?

And AI just these tools are the latest version of that and also arguably the single biggest version that we've ever seen. But I always do look through this lens of like, okay, how are these going to, first of all, remove the bottleneck? And then most importantly, once you've removed the current bottleneck, there are new bottlenecks behind it and they've just been hidden from you because they were dominated by the--

It's sort of like Amdahl's law with the serial and parallel parts of the program. So a few things are interesting. I mean one, I do think you hit on measuring, you know, as a observability person in my background, you know, I'm a strong believer that automated systems require observability to manage correctly and to their potential.

And if the process of writing code is going to be mostly automated, that means we are going to need a way to measure that and understand improvements, regressions, any changes you make in your process, what they have. I mean we have a, shameless plug, a Heavybit company Milestone we've invested in that's helping teams with this right now.

The other thing, and I want to get your opinion on this because you hit on you know, starting to use Claude-- I mean I think you're right and recently the good news is in the last, literally just the last three months, I think these tools, you know, they keep hitting these stepwise improvements but the one that's really gotten me and it's a little long winded, but I want to frame the question for you properly.

I feel now there's been with these last three months like three shifts and the first phase were the Copilots or even like you know, Cursor where it's like, hey, in the IDE we can start auto completing, we can start helping code build pieces. Helpful. Although I do think a lot of Java engineers correctly were like, well that just looks like what JetBrains does for me. So I'm not sure what the big deal is.

Peter: Autocomplete Plus.

Joe: Yeah, Autocomplete Plus. And then when I think when Claude Code came out that was the first big shift for me. And I think most of the senior principal staff engineers in my network, they're like, oh, wait a minute, I can be in a terminal or I can tell an agent to go do a piece of work for me. Again, that improves the productivity. And so most of those senior engineers where they were like now giving Claude or which, whichever agent these chunks of work to go do, a pull request would come back, they'd review it, commit it, rest of the process kind of looks the same. They could just like, I don't know, 2x3x5x the amount of pull requests they were generating because they weren't writing the code.

I think specifically with Opus and Codex in the last few months, you just can give it a pretty high level task. Say, "I want a piece of software that does this" and it'll go and whir and come back with a thing mostly working. And you look at it and you say, okay, well tweak this, tweak that, or whir a bit more and it's done.

And for the first time in the last three years, the thing that struck me, was that we're moving to a place where there's going to be a whole lot of code running in production that no human has ever laid eyes on.

Peter: Yes.

Joe: And that I think up until this moment everything's been like, oh yeah, human's faster, human's faster, human's faster. But still it's the old way, human's faster. All this code running in production that no one's ever reviewed or maybe ever will. I think that changes a lot of things from a CTO perspective. So long winded setup. But I'm curious like what your perspective is there?

Peter: If I had the definitive answers to this, I'd have raised a lot of money and would be working on my own startup to evangelize it myself. But I'll share some of the patterns on learning things.

I'm seeing one story that people use is, is this a leveling up in abstraction? Right? Is it? You say that there will suddenly be a lot of code running in production that no human has viewed. When was the last time that you look at the just in time omitted assembly from the JVM? There's no question that most of the code running in production at that level is never reviewed by humans. And I'm old enough to remember a time when people were like, "huh, C, that's not for real programmers. If you're not which registers you add from. Like if you don't know the 6502 assembly instruction set off by hard and you don't know the hexadecimal, you're not a real programmer."

So there's an element of that, but it assumes a lossless abstraction, that there's no leaks in the abstraction. And unfortunately English or any human language is not a terribly precise programming language. So then we have the challenge that if you're just like, oh, just go build me a commerce site, it will. But firstly, it's probably not the commerce site you imagine. Secondly, it's probably architecturally not the way you want it to be. And thirdly, if you just say it again, you'll get a completely different commerce site.

So now we have the challenge where we're removing some of the accidental complexity and the knowledge of syntax, but we still need to reliably at very least define verifiable rewards. What's the good outcome It can converge towards? And there are multiple dimensions to this. This is security, this is, you know, linting plus in terms of code best practices or architectural good practices, there are design patterns.

And I think one of the things we're working on now as a community, as a profession is writing down all the stuff that we know but never bothered to write down with sufficient precision. Right? Like, you know, you look at a bit of code, if you've programmed for a while, you're like, that's pretty good. This is going to be maintainable, single responsibility. You've got all the things we're now trying to get better at documenting that in a way that the agents will reliably firstly do it, so it's prompting, but then secondly that they will reliably verify it.

Whether it's pre commit hooks or whether you wait till you know your CI pipeline's running. I see both and I think they both have a place. Because you want to have agents that are responsible for reviewing for security, reviewing for architectural choices, reviewing for naming conventions, because you could argue, well, it's just assembler, no human's ever going to look at it. But it turns out that even agents have a hard time maintaining crappy code bases.

So I do think that for long lived projects, not for shims and MVPs and prototypes, but for long lived programs, code quality continues to matter, but I think that's just a matter of improving the specificity of our requirements gathering and persistence and how we teach the machines how to do a good job of creating, verifying and deploying great code.

Joe: Yeah, yeah, I think, fair point. Like the class of software discussing today, long live enterprise software systems that make real money. One of the things that I've been thinking about is you just hit on like English is very imprecise language. I mean the role of a software engineering organization, I think at some level has always been to, I think it's helpful to frame it as reducing risk to the business in the sense that the business has some very high level, ill-specified in this terribly imprecise language, Goals.

Like you said, a goal could be like, hey, we want an E commerce site. And the product and software engineering organization's jobs are to go from that high level directive to a working E commerce site. I think it's a relevant thing for CTOs. I still believe, particularly in these enterprise software systems, like ultimately there will be humans that hold the title of software developer, software engineer, moving forward in the future.

And it's just in terms of what I think for CTOs the challenge is understanding what that role is changing to along with timescale and how to start moving your organization. Maybe that's a good question for you. What do you think are best practices for a CTO to start that transition? Like you made the point on cloud with all the repos. What's the AI version of that?

Peter: So people are still figuring out the details, but I've seen some really good examples. 2025 was the year of the prototype. It went from, you know, Q1 it was like, ah it's autocomplete plus Q2, maybe this has promise. Q3 you were saying, oh, for certain repetitive tasks and generating certain things or upgrading versions or Greenfield apps is becoming plausible. And then as you said, the world changed with Opus 4.5, Codex.

Like November of last year was when we're like, oh, this thing's for real. And you look at fever dreams like Steve Yegge's Gas Town. I wouldn't want to use it myself. I'm not quite that competent or crazy, but I'm like, oh, we're all going to be running one of them. It's just without the Mad Max references and it will be a little more sane and it'll be six months or a year, but that's clearly the future.

So as a CTO with a large organization, firstly, you've got to let people know we are doing this. It's not just autocomplete. Yes, there are problems with it and our job is to figure it out. If you've still got people in your org, and most people doing large orgs who are like, "nah, I can code better or faster myself. I spend more time fixing the prompts," it's like, yeah, that's because you're prompting wrong. And even though there are areas where it's harder than others, figure out how to prompt it right, even if it takes longer because that is your job.

And the job of an engineer going forward is a meta job. Our job now is to build the specifications that allow the machine to turn well formed specifications into software. Our job as engineers is not to write code, it is to improve the systems that we have built that will generate, verify and deploy the code for us. And if you don't want to be an engineering manager to a team of agents, I hate to say it, but this is a good time to open a coffee shop. The role is changing and I don't see it going back.

Joe: Yeah, you know, it's interesting in terms of like where you will use code. I mean, I'm a huge fan of Andy Grove, High Output Management in particular, because there's just such a couple of like succinct takeaways. And so one which up to this point was relevant to like leaders, you know, CEOs and CTOs and executives. But the notion that as an executive and a leader, your actual output, your personal output, is actually the output of the organization that you manage.

And hence the title High Output Management, which if you haven't read it, it's a trope to say this in Silicon Valley, but please go read it. I t's a short book, but one of the things kind of fascinates me is first and foremost that feels like that maybe is going to become a lot more relevant for the future of the quote unquote human IC. In that like your output is going to be the output of the systems you manage. Right?

And historically the bottleneck for that was how many hours a day can you bang your meat sticks against the analog physical device used to communicate the important valuable things you've conceived of into existence? And that bottleneck's being removed. So, what's the new bottleneck?

Peter: I also want to clarify too. I think it's impacting multiple bottlenecks. I was speaking with a senior leader at a large telecoms company at KubeCon at the executive Summit a few months ago. And he highlighted that their biggest win was they managed to reduce the cycle time for their PRs, like from here's an idea to here's code in production. They managed to reduce that by about 60% by improving the requirements gathering process.

Basically they had an agentic system that would engage with the business stakeholders and ask them the tough questions that the EMs or team leads or ICs didn't want to ask and then they'd get such good specifications. I think they were just using augmented coding at the time, so the coding was a little faster. But you've really got to re-envision the entire SDLC and understand that there's no particular reason that an agentic system with human in the loop touch points can't go from a corporate strategy to OKRs to proposed experiments to epics to tickets to code, to deployment and to experimentation and run the feedback loop and run experiments.

So I think you need some humans in that loop, but there's no reason to believe that it's just coding that's going to be changing. That's going to be the smallest part of the transformation.

Joe: Yeah, I think that's a great point. Like you said, looking at the SDLC holistically and understanding like, hey, where can we bring these techniques to bear at each point? I think it also means like mapping that process. I mean it was always implicitly a state machine, but I think understanding that it is actually a state machine and now it's a state machine that's moving at a velocity that, that needs to be managed in software that understands like where it is in the state machine, what the transitions are, what the certification points are, what the gates are.

And as someone who's always been a process, product geek at heart, going back to the earliest day, and that's probably part of why I ended up in the engineering leadership track, is because I liked tinkering on the machine of the set of humans, and how can we make it better with software? If you look at a lot of the things that actually make AI and agents perform really well, they map a lot to a lot of the practices that top decile software engineering teams have been adopting for the last 20 years in terms of like, hey, know your specs, hey, incremental development, hey, have good tests.

Peter: And even the things we learned about remote management where it's like write it all down, prefer public Slack channels over private channels or emails. Well, guess what? That is exactly the knowledge base that you need to use to update your onboarding resources automatically so that you're capturing the intent.

I mean, context is a whole other fascinating question. And on the one hand, there was a recent piece talking about this idea that we need to capture the context graphs not just of current state and changes in state, like we changed our strategy, but also why, what inputs were required and weighed in what manner to decide to change strategy from X to Y, to change your ICP or value prop or whatever it is.

And on one hand we do, I think, need to build those rich context graphs. Then the other engineering challenge I don't think anybody has solved yet is we then need to figure out how you can take those and effectively treating it like operating system where-- Context is a memory management, allocation and access problem. Right. It looks very much like orchestrators, look like schedulers, compacting looks a little bit like some of the memory management techniques you use for your writing to disk. In this case, it's a lossy choice.

We've got to get much better at both capturing intent so that we can bring the richness of understanding to our agentic, hopefully co-workers, not overlords. But we also need to figure out that the real technical challenges of what subset of that context can be pulled to allow to make better decisions. And I think we're actually going to bring, a lot of the learnings are going to be from organizational psychology, from organizational structures.

How can we take the learnings from those and use them to leverage existing frameworks to help the machines to make better decisions by understanding our value, our cultures and our constraints?

Joe: What's interesting too is that problem is especially like the larger scale of the enterprise or CTO of just by definition, the bigger that problem is, the more context there is, the more customer history there is, the more different departments there are, the more stakeholders there are.

To me, one of the most interesting things that's the last two, three years of advancements is as much better as the models have become and as more capable and what they can achieve, the one thing that stayed kind of stubbornly the same is the size of a really effective context window, which is still around like I think 200,000 tokens as of this recording.

Peter: Right? I think that's true. And also I think that it again goes back, as you said, to good practice from before. Thinly sliced units of value, lots of little kind of pieces of software designed to apply clean transformations that can be composed. I think there are lots of things we can learn about managing complexity to help in terms of that kind of context engineering challenge.

Joe: Amazing. Well lot of stuff to think on today. It's been a super great conversation, Peter. I really appreciate you coming on the show. And we'll put links to gather.dev and the upcoming book in the show notes. Thanks so much for joining us today.

Peter: Joe, thanks so much for inviting me. This is a lot of fun.