Ep. #17, AI Native Software Factories with Solomon Hykes

Brian Douglas: Welcome to another installment of Open Source Ready.

John, you're here again with being co-host, man. It's 17 episodes.

John McBride: I know. How have we done it?

Brian: Yeah, a lot of Google Calendar and sharing calendars and identifying when we can actually do this stuff so.

John: Yeah, busy days.

Brian: Yeah, I'm all about optimizing for workflows, which I could maybe get feedback from our guest, Solomon.

You've been going to this workflow and you showed up, and you're in this meeting and in this podcast. So how are you doing?

Solomon Hykes: I'm doing great, yeah. Thanks for having me.

Brian: Yeah, yeah. Well, Solomon Hykes, some people like know you from Docker 'cause you helped create that and sort of led that ship or steered that ship, and now you're working on new thing called Dagger.

Solomon: Yes.

Brian: Catch us up on what's going on with Solomon and what's taking up all your time?

Solomon: What's taking most of my time, you know, and sort of competing head-to-head with my two kids, is this new startup Dagger, yeah.

So we've been working on it with my co-founders for, I don't know, maybe four years, maybe going on five years now.

It was our COVID lockdown tinkering project, and then it became more than that over time.

It's taken a lot of focus and energy, but yeah, it's a lot of fun. You know, I'm doing what I love to do, building stuff.

I have the privilege of having users who actually use it and tell me all the things they hate about it.

And then we go fix that and, you know, they use it some more and complain some more, with love.

And, you know, those rapid iteration loops, that's what I love doing so, yeah.

John: I'm actually a fairly early adopter of Dagger and the whole like, ecosystem.

Solomon: Nice.

John: We even looked at using it at Open Source to replace some of our GitHub actions, and CICD, and stuff.

And funny enough, you and I actually have a funny connection with some of the Concourse people way back on like Cloud Foundry and like how Concourse ran and stuff.

And that was my initial instinct when I first saw Dagger is I was like, "Oh, this is so cool. It's kind of a new hot take on Concourse," which for those who don't know, it was kind of a early days container-based CICD.

That's obviously not what Dagger is, but I'd love for you to paint the picture of what it is today and how that's like evolving.

Solomon: Yeah, and by the way, you know about our shared DNA with Concourse, right?

John: Yes, yes.

Solomon: Okay, so it's probably worth saying that the creator of Concourse is a core engineer at Dagger, Alex. And he built several critical components that you can experience if you use Dagger every day.

Some people are just obsessed with a certain kind of problem and we eventually find each other, and we can kind of commiserate on how things are still broken and go and try and make it better together, so that's exactly what's happening at Dagger.

John: Yeah, shout out to Alex Suraci I worked with him on a bunch of the Cloud Foundry pipeline stuff that was like deep Concourse, but yeah, it's funny how small tech is, right?

Solomon: But getting bigger.

John: It's getting bigger. That's true, that's true. Anyway, so shared DNA. You know, maybe some roots in like CICD, but what is Dagger today?

Solomon: So--

Dagger started as a solution to a very specific problem that turned into a quest to solve a larger and larger ball of interconnected problems, but the initial problem that we focused on solving was what we call push and pray.

This is the problem that you encounter if you're in charge of builds and CI, you know, if you're the designated DevOps person in any software team, and you end up working on these CI pipelines a lot, and you find yourself in the loop, right? It's all about iteration loops.

You make a change and then you run those changes that change through the pipelines, and then the pipelines take forever to give you a result because you have to go through Git, you have to push, then you have to wait for the push to trigger a series of events that will dispatch some jobs and the jobs will do mysterious things on a mysterious server.

And then eventually you'll get an email, or you know, you'll be refreshing a browser and it goes red. I'm like, "Oh, what is it this time?"

And oh, you forgot a tab because to configure this pipeline, you have to combine a weird YAML format that only works on this one platform plus a bunch of shell scripts.

Plus probably a custom tool that someone in your team made to glue it all together. And that's your life.

And these very slow, very brittle loops, one, they hurt productivity, and B, they just sort of burn you out as a DevOps person and you find yourself in the situation where you're the bottleneck, right?

Devs are like blah, blah, blah, blah, you know, shipping features, it all works on their machines but eventually, the I in the CI is for integrate, at some point, all the pieces have to be assembled and then, you know, the new version has to be out there.

And there are so many steps to automate to make that happen these days because these apps are so complicated that the DevOps person, usually vastly outnumbered by the devs, is kind of holding it all together and is getting ground down by this problem of brittle, slow, maddening iteration loops.

So we focus on solving that. And then, when you pull out the thread of how to solve that problem, you find yourself understanding why it hasn't been solved before?

Because it's not a problem that a traditional product can solve, you can't just drop a SaaS product or drop a tool and boom, problem solve.

Because it's about integration, it's about gluing all these different things together that form your stack. All your build tools, all your test tools, everyone's favorite stack, and then tool chain, it all comes together at integration.

Sometimes people try to say, "Well, okay, throw all of that stuff away. Let's start over with a monolith," right? A monolithic platform that just takes care of everything. Problem solved. Because it's all homogenous.

But in reality, no one can do that. So you have to find a solution that can glue everything together, but somehow just be more efficient than the current mess.

So we just went on a quest to solving that and we built this engine that I think solves the problem, you know, and now we're building a community of people that are using Dagger to solve this problem of push and pray, and just generally integrating all the different parts of your stack into a consistent application delivery platform.

John: Yeah. I can sort of see the like bigger picture even as it, like, you know, associated with Docker. Because sometimes a container feels like you're, you know, stitching a lot of those pieces together to get to that end result.

Even a Docker file sometimes can just be like many layers of a thing that end up, you know, spitting out a binary that then you run in the container.

Was that kind of part of the iteration where at Docker you saw this problem? This like stitching together integration problem?

Solomon: Yeah, I think before we actually pivoted to doing Docker, we were trying to solve the whole problem of application delivery. And at the time, the way you did that was with platform as a service, right?

So we had .cloud, which was an attempt at solving the whole problem with a monolithic platform.

You know, and it was us versus Heroku, versus later Cloud Foundry, versus, you know, whatever, Red Hat OpenShift. And we just ran into this problem of customization and modularity.

Like, my analogy is factories, right? It's like a software factory is what we call it. You got the prototype, you got the design, and then you got manufacture the thing at scale, right?

Like no one buys a factory of the factory store, right. Like Tesla has their unique factory that they co-designed with the product. It's part of the product really.

Every great product works that way, a physical product. And software is the same, right. Like your scripts and like that whole mess, that's your mess.

There's no other software factory like yours, right? So you can't solve it once and just, you know, build a perfect factory and then just kind of mass produce it. It's not how it works.

So then we realized, "Okay, we're missing a building block here. Like there's no ecosystem to build on."

You know, the common denominator for building a platform, like a system modular enough is Linux basically. That's all we have. It's Linux, and then shell scripts, and that's it.

And so we just found ourselves by chance, et cetera, building Docker itself, which kickstarted creating an ecosystem.

And now we're kind of back to the starting point of, "Okay, now that containers and that whole ecosystem exists, now we can go back maybe to solving the problem we were trying to solve in the first place, but just now get it right, make it modular enough that we can actually solve this problem of making software factories just great, instead of consistently a source of pain and waste."

John: Yeah. I think I've said this on the episode before, but when I was at AWS, I was shocked how much bubblegum and duct tape there was on everything.

Basically all of AWS is compute and storage, EC2 and S3, and then everything else, all the other services are just that with bubblegum and duct tape.

So I definitely hear that on like, you know, the software factories and having that be a little more integrated and grokkable for those DevOps people.

Solomon: Yeah, and I think, you know, one thing we do is we talk to a lot of teams about their software factories and we cast a very wide net, we talk to tiny, and giant teams, and everything between.

And so it's really interesting to realize, "Oh, wow, everyone is building this out of duct tape." And you know, it's just a pragmatic thing to do.

You got a product to ship and you have to balance like, "Okay, I can't boil the ocean here. We got to ship, ship, ship."

And then you're on a treadmill and you're like, "Okay, I know I need to replatform this," like this needs to scale, there's too much duct tape.

But you know, I can't just close up shop and tell developers, "Can't ship, come back in six months. We're rebuilding the factory," right?

And then so developers, they're just, you know, they're coming to you relentlessly or, you know, managers, or business units, or whatever, and it's always, "Just one more feature, just one more script, just one more deployment."

And so you're in that treadmill and it's hard to get off, you know? And it's really interesting to see everyone's in that situation and everyone feels bad about it, so we do calls and we're like, "Hey, tell us about your, you know, CICD, your dev environments, your scripts, whatever."

And they're like, you know, it's like when you visit someone and they're like, "I'm so sorry about the mess. Please ignore the mess." They're like, "Don't worry. They're like everyone, it's messy. It's always messy. It's crazy."

Brian: Yeah, so it sounds like to like summarize like what you guys are doing, you're like... There's like playbooks out there on how folks are building and shipping the production, and Dagger is making that more accessible for folks to, you know, build on top of...

Maybe a month ago, John, we were talking about this sort of this infrastructural layer where we mentioned the Linux part. And everything's built on Linux and that's like a given.

And then we have now today Kubernetes, which is like these evolution since Docker containers where that's kind of a given, you've got this orchestration layer, but then when it starts to like, "Okay, I've got to do scale infrastructure and databases for folks on their phone to generate images of their emoji faces or whatever."

Like, there's not a clear playbook on how to do that. There's a lot of people like slinging from the hip. And being like, "we've got to demo by June. Like let's get this going."

Solomon: Right, yeah. I mean, it always starts with a workshop, like an artisanal workshop. You don't start with a factory in day one.

What happens is, it's a startup or an internal project or a team. Day one, it's a workshop. There's like three of you anyway, you're just doing everything yourself.

And then eventually, if it lives long enough, it'll grow. And then the workshop gets bigger and bigger, and then at some point it becomes a makeshift factory, right.

That's just the evolution. And then that's why eventually you have to deal with a technical step. But like our approach is...

I mean, there's a million startups trying to help solve this problem. Our approach is to provide a solution at the system layer, because there's a lot of tools, there's a lot of frameworks, and there's a lot of infrastructure, and there's a lot of, you know, solutions and best practices but the system primitives that we have remain the same and they have been for basically 50 years.

Again, it's POSIX. Like it's all built on the same primitives and they're very old primitives. And you know, the systems we've built, they're not the same.

And the systems we were building when even Linux, let alone Unix were created, right? And so we've been basically filling the gap at the tool and application level, right?

And so what we're trying to do is build like a new system layer that you can actually rely on, almost like an operating system, when building these systems. So it's a lot of system design, platform design. It's like designing the Lego brick.

Okay. How many different shapes do you need for Lego to be perfect? Is it five? Probably not enough, right? Is it a thousand? Okay, that's too many, right? That's system design.

So that's what we're doing. So it's a hard... We're focusing on a hard engineering and design problem, which we think unlocks getting to the right solution.

Because there's just too much fragmentation, too much one-off work for this thing to scale.

John: Yeah. Where do you see the failure modes? Like, when you're evaluating and or I guess looking at these companies or talking to people with, you know, their messy CICDs, and the bubblegum, and duct tape.

Like where's typically the failure mode? Is it in how the treadmill got too fast and it's hard to get off?

Is it, you know, just that there wasn't enough time and energy spent into like making sure the systems are reliable to actually deliver software out of the factories.

Solomon: Where things break, usually there's a few things. One is just the inability to run the whole thing locally is just a major problem.

Because that means there's a whole class of work that you can't do without kicking off this insanely complicated infrastructure.

You know, it's like, there's this bald and chain that you're carrying around. So just bringing back the simplicity of this is your build, this is your, like not just a piece of your build, the whole build across maybe five tool chains, whatever platforms.

This is your official build and it's here, you can run it on your laptop anytime you want. And also later you can run in CI, it'll be the same one, you know.

This is your end-to-end test environment. It's here, take it, you know, you can run it right now on your machine, in my machine, and in CI.

So it was just this concept of separating the actual logic of your workflows from infrastructure, especially proprietary infrastructure.

'Cause these are not VMs, you know, I mean, there's VMs below, but it's like proprietary everything. Proprietary APIs, proprietary event dispatch, proprietary configuration files, so you're really locked in.

And so yeah, local execution, just making that portable is the biggest... The lack of it is the pain point and enabling it is just...

Everyone who is using Dagger starts there, I think, just running the thing locally and you're like, "Oh, I'm breathing again."

And then the second one is just complexity of the system. You know, it always starts small and simple.

Just one script, just one YAML, just one Docker file. And then yeah, you leave, you come back a year later, and predictably the complexity just kind of exponentially worse.

And no one knows how it works anymore. There are areas that literally you can't touch 'cause the person who designed it is gone, you know.

Is this the script? Oh, no, no. That's just... There's another script that generates this script and then you got to concatenate it with this other system, but don't touch it, you know.

So that's the other failure mode. Just crippling exponential complexity because you're not building, like it reaches the point where your software factory, right, like all these workflow that ship the app, they are themselves an app. It's complicated.

You know, that's why we call it a factory. You know, it's complicated. So it should be software, it should be proper software running on a proper platform, but it's not.

It's a bunch of scripts and config files running on makeshift infrastructure. And so, that's the second failure mode just, okay.

And the solution is, make it proper software built on a proper platform. And those are the two problems that we come in and propose solving, yeah.

John: That is one thing I love about Dagger is I can write Go, for a lot of the systems and the integrations, right?

Solomon: Oh, so you use the Go SDK.

John: Yeah, yeah. That's the one I've used in the past. It's been a little bit since I've looked at it and I want to revisit it, but maybe this is a good point to kind of jump into, like how LLMs integrate and the way you see that right now.

Because, you know, the first thing I thought of when you mentioned like, you know, the piece of the pipeline that nobody can ever touch again and we've all seen this code where it's like, you know, it's got these big comment brackets like, do not change, we don't know why. Like, it's going to break.

Is there a feature where like LLMs or even like LLMs doing DevOpsy work start to unlock more productivity or efficiencies, or like what does that feature start to look like?

Solomon: Yeah. Finding the answer to that question has been a big focus of ours for the last three months and we got pulled into it, 'cause for a while we sort of made a conscious decision to ignore the hype.

Of course, at an individual level, you're an engineer, you're going to play with LLMs, right, of course, but as a company, we didn't feel any urgency to say, "Quick put the word AI in the website." You know, "Anywhere. Just put AI," you know.

Because it feels forced and fake, and our whole business is built on an open source, like a hardcore systems engineering community as a growth engine, right. Like, so it has to be authentic, otherwise the whole thing falls apart.

So, but eventually what happened is those users, those systems engineers started applying Dagger for building AI agents.

In other words, integrating software with LLMs to do interesting things. And in particular for tool calling. So, you know, the kind of phase two of AI where it's not just send message to LLM, get message back, which was mind blowing enough.

But now, you know, send messages to LLM and also LLM calls, tools, calls functions to actually do things. And you know, obviously, that's what like the tornado we're in now, everyone wants to do that.

And some of them were Daggernauts, we call 'em, you know. They identify as part of the Dagger community and they're like, "I'm going to do this with Dagger."

And they did it and they liked doing it and they came back and, you know, we have meetups, we have a live stream we do every other week with the community so they started showing it.

And you know, of course, from there we just kind of got pulled in and we took a few months to really understand why and how and what the opportunity was.

And I think now we figured it out. And it boils down to two things. One is architecture and the other is actual use cases like applications.

On the architecture side, like AI agents fundamentally are not... It's not a software market. It's not a new software market. Like, "Oh, there's people who build pre-agent stuff and then there's going to be this new market of people using and building agents."

It's not a new software market, it's a new software architecture. It's like client server or microservices. It's a way to build software. And it's an architecture that's going to be everywhere soon enough, you know, sometimes for the wrong reasons, like you have a hammer, you see nails everywhere, but fundamentally it's very powerful. So eventually, all software will have some agentic dimension to it.

The way that dimension is relevant to Dagger is we discovered with our community's help that, agents have a very different architecture because they're built around workflows.

And so it turns out the architecture that we designed Dagger for, which is workflows, build workflows, test workflows, data processing workflows.

And we have a very specific systems engineering opinion of how you should run your workflows.

Like, you know, again, it's an OS, it happens to be a perfect match for how agents are architected. Because again, agents are mostly workflows and then a little bit of front end and infrastructure, right.

And we can go to why and why that's cool, but it just makes Dagger a great match for a system layer for building agentic workflows.

Okay, and then the community was excited about that for purely, nerdy engineering reasons. Like, "Look, how cool is this?"

And we've always encouraged a long tail of uses of Dagger regardless of use case. In other words, the whole time we've been talking about push and pray, CICD, making it better for the software factory, et cetera.

But in reality, the fundamental tech can run any kind of workflow as long as it fits this architecture.

And we've always had like an 80, 20, 80% straight up fix your software factory, platform engineers, mostly at medium and large engineering teams.

And then a 20% of all sorts of crazy, fun applications of Dagger and agents were just one more of those. So that's for the architecture, however it's relevant to Dagger, basically Dagger can run agents really well.

The second dimension is used cases, like, are agents relevant to the software factory? Because that is our business, you know, that is the 80% for us.

Like do agents matter? Or is it just like a side thing? Like, a pure technology side thing.

And we realized, okay, actually it matters enormously because one area that agents are disrupting, like in a major way, not the only area, you know, there's also like lawyers and video editors, I guess half the planet is like, "Oh, shit."

But yeah, software engineering is one domain where agents are just changing everything, right.

Because now they're going to write a lot of the code and we're saying, "Oh, well. Your AIs will be software engineers."

Brian: So I'm glad you brought this up 'cause I wanted to actually ask this question because like, we're seeing the world is cyclical. Like, I was just talking to someone about IBM that didn't know, like, IBM's incept in like started in 1911. So like they were doing, you know, they were business machines, but like-

Solomon: Typewriters, yeah.

Brian: Yeah. They've gone through different types of computing and the computers and system mainframes, but what I'm getting at is like, yes, programmers are at risk of, like, maybe are we losing our jobs or are we, like, accelerating our jobs where we have less programmers?

But the question I have for you specifically, like, where you sit in the space is, with Dagger, we're building factories, but, like, we know about 200 years ago we had factories, tons of them in the US.

Well, at least where I live in the US we had a ton of factories, but then manufacturing went offshore. Like, we figured out how to speed this stuff up.

We could also like, you know, build components in Taiwan and then bring them over to finish, do sort of Made in America.

We've accelerated that at the speed of, I can push one click and I can have something in my Amazon, you know, box outside my house.

Like what's your take on that for software? Like, especially where you fit right now with Dagger, 'cause you're now enabling... What I think is this next wave, like, we had this OpenAI, API, you know, consumer testings out, and now we're actually building some cool things that have runtime for agents, but the scale part is not quite figured out just yet.

Solomon: Yeah.

I think like my personal take on how agents impact, you know, the job, it's kind of a bottle half full, bottle half empty decision, I think. Because more automation always leads to some things you're doing are not worth doing. And if it's all you were doing, then maybe it'll be harder to find a job doing that.

But, you know, there are all other tasks that are now more valuable and you know, all the topics around that, like, okay, but it's not always easy to go from doing these tasks to those, how do you train people?

What about people actually lose their job? I don't have any qualified opinion on that. I think in the world of software, specifically in practice what I'm seeing, is that every engineer now basically has 10 interns as a multiplier to them available or will soon.

You know, it's the equivalent of that. Like it's not just an army of humans doing the work and with a bunch of scripts.

Each human has its own little army or will of agents to do their bidding and like do all the work that's better done by an agent, and then that frees you up to do the rest, right.

I believe there will always be the rest because that's the whole point. You know, you move on to the next level of task, right. So we want to enable that.

I mean, I want to benefit from it. I have a lot of ideas that I want to build and you know, we're a team of 25.

It seems like a lot of people, but it really, it's not when you want to build so much. And so there's always like, at least--

I don't live in a world where there's too many people and not enough work, it's the other way around. Like, there's too much stuff to do and there's just not enough of us and so, you're telling me we can farm out more of the work to agents? Yeah, sign me up.

And I think all the smart teams are going to be doing that and they're all going to need a platform to do it.

So this is where it connects to this community that we serve, which is platform engineers, people building these software factories for the teams that they serve.

It's their job to deliver this 10x multiplier to make the engineers they serve more productive. It's going to happen via this platform.

And we want to help 'em do that basically. And then from there there's a bunch of questions in the details, like, what does that mean in practice?

Like short term we're focusing on two applications. One is adding agents in your CI.

Like you have workflows, they're already triggered by events, they're already scheduled on these clusters, they already have access to really valuable data, like your source code, your secrets, your deployments.

You can drop agentic capabilities in these workflows. Not all over the place. You got to be careful how you do it.

Like you can insert QA in the CICD pipeline, right. You can insert, like, a first pass at generating docs. We're doing this for us.

We're finally going to have an agent write our change log or at least a first pass of it from the git history instead of always having, you know, forgetting, "Oh, no I didn't write the manual thing, but change log is out of date."

Like there's so many tasks that can be embedded in CICD as we know it, like nuggets of agentic work. So that's the first use case that we want to enable.

Then the second one is, I mean, Cursor, Windsurf. You know, the agent in your editor writing code for you and that market is like going crazy.

Every engineer I know is either using one of those or evaluating it. And not just in startups either, large companies, like they're rolling out these evaluations.

So you have an AI and your editor that can edit, that can generate code now but what about the dev loop, right? Like how could that agent know if it's doing a good job or not?

That's sort of the bottleneck now. These models are so smart, they can generate any code you want as much as you want.

And you know, it's only getting better, but, you know, they need to know if they're doing a good job or not.

And right now it's pretty primitive how you do that. Like you have to look at the code and say, "Okay, approve." And then it doesn't build and you say, "Yo, you broke it, here's the error."

But you want them to be autonomous, right. Like you want them to have a dev environment. And guess what? Dagger provides environment.

So, we want agents to be able to just plug into your existing Dagger platform and just make use of all of it exactly like a human would.

So then you elevate the coding agent from a sidekick in this little island that is your IDE to like literally a colleague.

Brian: Yeah, and I want to ask one more question before we transition over the reads.

Solomon: Yeah, sorry, I give long answers 'cause I'm excited about this.

John: That's great.

Brian: No, I appreciate the excitement and I appreciate... I mean, I've known of Dagger for quite a few years and even when we chatted like a couple years ago, John had a used case, but I didn't really have a used case for me to leverage Dagger.

But I've been really on this, like, tear of trying to figure out, like, what's this next wave of AI? And, like, I do think it's the infrastructure thing.

I think it's platform engineers like getting the green light from their higher levels to say, "Let's go, scale the stuff. Let's invite our new agent, coworkers, to work alongside of us." And I see Dagger enabling a part of this.

But you were just at KubeCon EU, and I didn't get to catch your talk and I watched a bunch of videos in preparing for this conversation, but I also went by your table and I saw a ton of people, like, chatting with you and the team.

So I was just curious, like, what's been the feedback? Because now you've been building this tool for building factories, your platform engineers.

This AI thing is now enabled because your community said, "Hey, this is something that's working." But I'm curious to broader ecosystem.

I've got the feedback of, like, folks a little concerned around using like a copilot, or a continue, or these other assistants but this is a different world.

Solomon: Yeah. Well, it's interesting because at an event like KubeCon that is focused on, you know, platform engineers, AI is very present, but so are core platform engineering topics like CICD and since now we're talking about both, you know, we're communicating, positioning ourselves as relevant to both.

When people come talk to us at the table, they end up talking about both, but it's not an equal mix.

So we have some people that are basically only vaguely interested in the agent side or you know, maybe they didn't really connect that we're doing agents now, you know.

Because we've been around for a few years and we're known in the platform engineering community in the DevOps community as this tool that makes your CI better, right?

And so most of the conversations start there. And then a surprising number of platform engineers are very interested, or very scared, or generally want to talk about agents.

Not scared, but just sort of, they got questions. You know?

John: There's a hesitancy, like, I think in the industry at large, right.

Solomon: Yeah. But what ends up happening is that we have sort of layers to it. There's the base layer, which is continuing to do what we've always done.

So continuing to help platform engineers modernize their software factory and it starts with one engineer getting really excited and joining our Discord, using it on the side, inserting a little bit of Dagger in one little piece and then gradually expanding and trying to get everyone else on their team excited.

And that's a pretty long journey because it's never a priority to modernize the factory. It's rarely a priority because it's foundational work. It's replatforming.

And so it usually takes, I mean, we've had success stories including companies that are using Dagger at large scale and we're trying to get authorization to use the logo, but they're using it.

And it took like a year from the first engineering showing up to the Discord to a manager saying, "Okay, we're rolling it out," like thousands of pipelines we're converting, you know, very gradual.

So that's sort of the baseline. And then on top of that, you've got agents.

And the way that starts is, an individual engineer just playing with a tech because you can build little agents on top of Dagger very quickly that do something useful and then it's like a Lego construction, you can keep it on the shelf and then you can bring it back later to combine with others and kind of build your world.

And so you take an hour, two hours, it's like an hour, an hour and you got an agent, like a micro agent almost.

And then it's the way Dagger works, it's very encapsulated. You got inputs and outputs. It's like a function with an AI inside and that function you can bring back anytime.

And then from there, that I love that we're tapping into this sort of fresh, nerdy, excitement, collective excitement. Like we're all beginners with this stuff.

There's hackathons, meetups, like friends getting together and showing each other cool toys. Everyone's building toys and we love that.

You know, so our Discord is super active with agent randomness and it's been great news for us. You know, it just perfectly fits what we're good at.

And to be honest, it's kind of what we're missing because modernizing software factories is very important and valuable, but it takes time and there's just not the same urgency, right.

John: How, you know, and this question is very abstract. But how do you define DevOps? Like, what is it?

Solomon: Oh, man, I tend to be very flexible on the words. Because words change faster than reality. And so I tend to focus on the reality and sometimes the word for it changes. I'm like, "Sure, okay, we're calling it something else now. I don't care."

"You fight amongst yourselves what you want to call it. But like we do it and it works."

So, you know, the problems remain the same. I like using software factory whenever I can because it's not a trendy buzzword, it's an analogy.

You know, I kind of dread, I'm kind of hoping it doesn't take off like a Gartner thing or whatever because then people will argue over what it means.

So yeah, DevOps, I mean, I'm in the pragmatic, not sure exactly, it's fuzzy, it depends camp.

John: Yeah. Just don't start calling it AI Factory.

Solomon: AI Factory. I heard that one being used by Nvidia actually.

John: Oh. Interesting.

Solomon: We've been known to say things like the AI, you know, the agent ready software factory or you know, something like that but yeah.

You got to market, right? You got to do marketing.

Brian: Excellent. Well, Solomon, thanks so much for catching us up about Dagger.

And honestly, I'm looking forward to digging into the docs and some blog posts as well, and also trying to find out some of your community using Dagger as well.

I'm going to definitely take a comb through.

Solomon: We're all about community and I would highly recommend for you or anyone who's listening, who's interested, if you do one thing, just join the Discord because it's very active, very friendly, very welcoming.

And it's like a support group for DevOps people. And now agent, engineering people. And you'll get a taste of, you know, the vibe and the values.

I'm just really proud of it. I enjoy hanging out there all the time. And we actually, when I say hang out there, we don't have a separate Slack or private something like...

So as a team of 25 people, we work out of that same discord. We just have a few extra channels, but we're literally there all the time.

So yeah, come visit us.

Brian: Cool. Yeah, well thanks for the invite. I'll definitely be showing up soon. And with that said, are you ready to read?

Solomon: I'm ready to read.

Brian: So John, you've got some reads and specifically these are reads things that we sort of see around the internet.

Usually on topic, sometimes off topic, but yeah, actually it seems like you've got two on topic ones.

John: Yeah, two on topic ones. I thought the first one was kind of crazy and maybe a sleeper announcement from Mistral.

They announced and shipped their product, Le Chat, to the enterprise, which in theory just, you know, this is what they're saying and what people on Hacker News were talking about, to your data centers, on-prem with the models, you know, enterprise search, all this stuff, the chat, all this stuff.

Which just seems kind of crazy to me because I don't know of any of the other AI labs that are kind of targeting the enterprise like this.

And I don't know how they're going to like... I don't know, prevent their IP from getting lifted and shifted. But yeah, apparently enterprises it, it's coming to the enterprise.

So, for those who aren't aware of Le Chat, it's kind of a ChatGPT experience with, you know, the full foundational models from Mistral. Yeah. Crazy.

I don't know, any thoughts on enterprise or like the enterprise play in AI right now, Solomon?

Solomon: Yeah, I think there's a lot of money to be made. And not in a "quick, milk the hype before it dies down," but in a more foundational way than that.

Also, I think a lot of the rules of selling to enterprises will change. Like, for example, how fast the sell cycle has to be, you know, how slow rather. And like the kinds of arguments you have to make, et cetera.

Just because like you can touch budget line items that are pretty massive and you can change them in a really massive way. And the main blockers is proving that the tech works.

But I mean, there are some propositions that if you walk in the room and say, "I can divide your cost of this by a hundred," you know, like, the usual rules of procurement will not apply if you can prove that it works.

You know, 'cause the CO is going to be like, "Okay, screw your procurement. We're doing this."

You know, I mean that seems like something that's going to happen. So that's pretty fundamental.

John: Yeah, it's definitely something I've seen and you know, my time at the Linux Foundation working on, you know, the AI side of things, just the data privacy portion of it, is just so hard and so big, like, the governance around, like, where's this data going and do we really want, you know, OpenAI, or Anthropic, or any of these other sort of third party providers to just slurp up all the stuff to be able to provide, you know, like enterprise search or like a knowledge base essentially.

So this is exactly what I would want if I was, yeah, doing procurement.

Solomon: Assuming it works, you know, 'cause it's brutal out there.

John: Yeah, assuming it works.

Solomon: Yeah, so I think AI is super relevant to enterprise, I think they're going to be faster to adopt than other technology waves.

Yeah, I think arguments like your data's not going out there outside of your control will matter a lot.

I mean there are niches internationally, like, Mistral is a French company, they can play the European sovereignty card, right.

So there's way more European companies that don't want to send their data to OpenAI than American companies probably.

So that's a trend. That's the general fragmentation of the world along those lines and... Yeah, I don't know, lots of other thoughts but, you know, Mistral, French company, I'm French, I'm rooting for them.

I think it's healthy to have at least one, like, credible player outside of the US and China, basically. That would be nice.

John: Yeah, 100%, 100%. My other read this week was Concourse actually, you know, on topic here. Concourse has joined Cloud Foundry. Or the Cloud Foundry Foundation I should say. It's always been a Cloud Foundry thing.

But, yeah, I didn't know about the Cloud Foundry Foundation until I was at the last KubeCon and I walked by the Cloud Foundry Foundation booth and I just was like, "Oh, my gosh, it's still alive? That's so great."

I really thought that it had died in the pivotal acquisition, but very, very happy to see that some of these technologies live on. I'm sure there's big bank enterprises or something, you know, speaking of enterprises that still utilize these technologies.

Solomon: Oh, yeah.

John: Very slow to move.

Solomon: Yeah, I mean, at the beginning of this, when we were talking about Dagger's pitch and its approach to the software factory to building platforms, in a way our vision for platforms is kind of the complete opposite vision from a monolithic path, right. Which Cloud Foundry is representative of.

John: Oh, yeah.

Solomon: And also Red Hat OpenShift, right?

John: Yep.

Solomon: But I cannot be more convinced that this model scales better and will ultimately win. But you can't deny that today.

I mean Cloud Foundry actually don't know, but OpenShift, I know, is a billion dollar plus ARR business, which is ridiculous to my engineering mind, but makes sense to my enterprise go-to market minds, like, wow that's a successful platform.

You know, it's more than legacy. It's the norm today. It's how you do it as an enterprise.

Not necessarily OpenShift, you know, you could have be doing using something else, but the world actually thinks in paths right now, the tech world.

John: Yeah.

Solomon: And they're experiencing all these pains because of it. Like we just sort of, we got used to it.

So, you know, it's going to take a first cycle of proving that you can be 10 times better with a modular approach, you know, ship 10 times faster, you know, or do things 10 times cheaper, and then it's going to start percolating to these enterprises.

John: Yeah. You know, not to retread a bunch of old ground, but yeah, I think Cloud Foundry was one of those technologies that sort of sat in such a weird place and it was pretty hard, I think, you know, from my days at Pivotal when I remember that acquisition.

They were trying to do PKS, which was Pivotal Kubernetes Service. Heroku was eating a bunch of our lunch with PCS, Pivotal Cloud Services.

And I think inevitably probably Red Hat with OpenShift also was just, you know, making it so hard for us to actually make a full commitment to the paths and shipping that as like the actual product.

Probably depends on which industry an analyst you ask too, 'cause that was kind of a hard perspective to have just, you know, working mostly on, like, observability in the platform and working on pipelines and stuff.

But anyways, I'll handed off to Brian 'cause I could just chat on and on about Cloud Foundry.

Brian: Yeah, yeah. And I don't know if you mentioned, but you used to work on Cloud Foundry.

John: Yeah, I worked on, oh gosh, the observability stack, which was actually Grafana before they went to an AGPL license.

And then we kind of scrambled to figure out what that was going to be and the logging pipelines, and something that we call the platform automation, which was all Concourse based.

Solomon: Oh, Bosch?

John: Not Bosch. It was Bosch based, but it kind of stood up a bunch of that stuff to then, you know, make sure that Cloud Foundry was in theory orchestrated and healthy.

You know, we were trying to do stuff I think even before like Docker Swarm was trying to do different, you know, automation things, but it was all like, yeah, Bosch based, which was just really gnarly.

Solomon: Bosch was ahead of his time. Yeah, I agree.

Brian: Cool, well, I have a pick. And this is kind of relevant towards the end of our conversation, which is it's a blog post from Simon Willison who's pretty prolific I guess now in this AI space. I think he's known for Django, is like his original claim to fame.

But he's been doing a ton of stuff with dataset and yeah, basically he's just on the internet and he's got some good takes. But the blog post is titled, "Not all AI-assisted programming is Vibe Coding, but Vibe Coding rocks."

And I enjoyed this read 'cause it kind of really puts up some good talking points, especially with this world of like, should we vibe code, should we not vibe code, should we trust these tools that are just hand stuff over the agents, and then debugging, and like deploying stuff without our hands on it.

But his take is basically pretty simple, which is vibe coding's great for weekend projects, it's great for prototyping, it's great for isolated things that you need to ship really quickly.

Any sort of large projects infrastructure, like that still has a lot of parameters and a lot of tokens that are going to... It's going to be more expensive to leverage so you're going to get less value out of it.

But this has actually been leaning into what I've been doing. I built a... John gave me some guides on building some stuff in Ruby, which is basically getting some GitHub analytic infrastructure.

And the way I did it, and it's kind of explained in his article, which is kind of like I vibed with it, I did test driven development with my agent.

So I had just basically say, "Hey, in my rules, like, we're always doing TDD, so write a test, make sure it's sound, make sure you follow all effective Ruby parameters," which is like an entire book on how to, like, short methods and, like, everything's readable.

Which is super helpful for me 'cause it's, like, I haven't written Ruby in like four or five years. So when approaching the code that they're writing, it's all readable, it's testable.

I can validate it works every time. Oh, and the other thing is it runs the test for me every time it adds new things, so then it also tries to get the test to paths.

So what I'm getting at is, like, I think we're in a world where, like, if you know how to fly the airplane, it's okay to start doing autopilot if as long as you have like the test to keep you, you know, within balances and like within these levels.

Solomon: Yeah, totally.

Brian: And I really enjoyed this article to kind of like encourage me to sort of test out and try some of this stuff.

Solomon: What editor are you using for all that?

Brian: So I'm using a combination of VS code and Continue.dev. So continue is, you could do a self host or local models through Ollama and stuff like that, but they have an agent mode as well, so very similar to all these other tools, but I've been doing it that way.

And they've got essentially all these blocks where I can set up rules inside my blocks that I can actually index the entire Ruby standard lib.

And if doesn't exist and the modelers hasn't been trained so you got some languages that are a little outside the band of popular languages.

Continue gives me extra fuel to make sure my agent doesn't, you know, hallucinate.

Solomon: Yeah, it's interesting because from my point of view I'm seeing that, like, the emergence of this new ad hoc workflow for developing and it's very one-off and interactive, right.

And to me, I see these rule files accumulate in like 20 different formats with 20 different tools and editors.

And I see the next layer of artisanal scripts, you know, it's the same thing as the make file and the shell script that we had before.

Or you know, the Jupyter Notebook session, you're just like playing, messing around and you get to what you want, and then you do it once, you do it twice, and by the third time you're like, "I should probably automate this," right.

And then, so there's a sort of a pipeline, right, where tasks are done one off interactively in your loop, in your dev loop interactively, like really rapidly one off.

And then eventually if you do it often enough, you kind of want to turn it into something repeatable that becomes part of your tool chain and then it's sort of run continuously in the next loop.

Basically that's what we help people do with Dagger. And so, I see all these manually edited, carefully tuned prompts and like, "Oh, wait, let me do it this way, that way."

And then I see the next wave of Dagger functions, you know. Because eventually you say, "Okay, let me make this a function," you know, I need a function that just, you know, you give it a directory with source code and Ruby, and then it spits out a new version of directory with the tests or something and then boom, I have it, it's mine, I can use it anytime I want.

And then you can have another agent iterate. And now, that's just a tool to the agents. You know, so the convergence of tools and agents into a common model, I don't know if that makes sense, is super important.

Brian: Yeah.

Solomon: 'Cause right now we're being taught to keep them very separate. Like, okay, this is the agent and it's got rules and this thing and then these are the tools and you can use MCP or whatever.

But in fact, why can't an agent be wrapped in a tool for another agent to call it and just kind of keep building like that.

So our approach is, it's all functions. Just write a function, define the inputs and outputs, and if there's an LLM doing stuff inside the function, great.

It's a function that's really smart. You know, but it's still just a function and then you just kind of keep building up.

And I'm just really interested to see how that's going to play out, you know. 'Cause right now the tooling's just not there yet.

Brian: Yeah, and I think there's... I was talking to someone at Google DeepMind who's building some of the degenerative like art and images stuff inside of Gemini.

And was kind of explaining some, like, how they were approaching that without, you know, sharing the secret sauce.

But then I think Perplexity has a good example of like, I haven't read any of the stuff that, how they're building it, but they do have a sort of tool chain, an accessible tool.

So like you've got the models that have the context since like, you know, 2023 or whatever the last six months maybe might have that data, but then the tool will go fill in the data.

So like specifically, I'm in my warrior's gear today, but on Monday, was it Monday when Steph Curry got injured? Maybe it was Tuesday.

I went into Perplexity, asked a question of like, "Okay, based on his injury," I actually we didn't know what it was. You know, "How long is he going to be out?"

And what's nice is that Perplexity can leverage the tools that they have accessible to them to know what I'm talking about, and then use its knowledge base to then like, give me an answer of like, "Oh, yeah, one to four weeks."

I'm like, "Oh, okay, this is... Season's done." But at least I knew that as a fan.

John: Yeah.

Solomon: Yeah. It's just, we're going to get used to this stuff really quickly, right?

John: Yeah. I was just going to say, I feel like I'm really left behind. I'm still on just, you know, copy pasting stuff out of Claude.

Mostly just 'cause like, I'm stuck in Vim land and it's like I'm never going to be able, and I know that a lot of these things have a, you know, Vim modes and then these days, but I'm like so deep down the rabbit hole.

Solomon: I do that a lot.

John: Oh, you do that a lot, yeah?

Solomon: Well, yeah, I mean I use ChatGPT but like I use Zed as my main editor and they just launched the full blown agent editing.

And I actually haven't built up the habit to using it, but I've been using the contextual, like I'll just select one piece of text, very specific context and then they have this control enter and in that context you can say, refactor or you know, add error checking or whatever.

And I use that. But you know, the code base wide, "Hey, refactor me everything."

John: Right.

Solomon: I haven't really like, I don't vibe code beyond that, but they'll have ChatGPT like, "Code me a function that does something," that I just, I'm too lazy to do, and then I'll just copy paste it so.

I feel like that's what most people still do. So I don't think you're left behind. It's just Brian who's ahead of us.

John: Yeah, exactly.

Brian: Yeah, and that's because I get to write tech debt every weekend, so yeah. No one's, no one's reviewing my code.

John: There you go.

Solomon: If you guys want to write some like autonomous agents on the backend, to work on your code base, I can hook you up with some Dagger 'cause it's pretty fun.

Brian: Yeah, it sounds like, John, you've just got to build out the ecosystem. Support all these NeoVim users.

John: I'm on it.

Brian: Excellent. Well, with that, folks, be sure to stay ready.