Ep. #45, When Agents Outrun Human Review with Dotta of Paperclip

Brian Douglas: Welcome to another installment of Open Source Ready. Hey, John, how you doing?

John McBride: Hey, I'm doing good. I have been Jevving it up. What about you, Brian?

Brian: Oh, I am Jevving. Jevving like a feving.

John: Like a fiend.

Brian: It's funny because I just hit 40. So I started using insoles. And that joke is from Dr. Scholl's. Are you gelling?

John: Oh, yeah. I remember those from when I was a kid.

Brian: Yeah, I'm dating myself. My entire upbringing was either Nickelodeon or some sort of random commercial. And then memes. Memes took over that spot in my brain.

But with that being said, we do have a guest. We have got Dotta here from Paperclip. Dotta, how are you doing?

Dotta B: 10 out of 10. Thanks for asking.

Brian: Very cool. Well, 10 out of 10, man. I just saw your repo, Paperclip. It's like 84,000 stars.

I remember back when it went from like 5 to 10 pretty quickly. You guys kind of took over the open source internet for a moment.

But yeah, we'd love to find out, one, who are you? But two, what are you working on?

Dotta: So I'm Dotta. I'm a software engineer by trade. I've worked in a lot of things, from programmer education. I did a tour in crypto.

And I've been working most recently just on Paperclip. So Paperclip really was just born out of the AI industry moving too fast. There's too many things every day. There's too many things to try.

And really, as soon as you settle into one workflow, there's something else that you're supposed to be trying, right? And I was trying to build almost like a life raft of, I need a stable point where I can integrate all these things together.

Beyond that, I think that you're doing too much. You're literally managing 20 or 30 or 40 tasks all at the same time. Because as soon as you start one task, you're not going to wait for three hours for it to finish. You're going to start something else.

And I just was finding the existing tools are not really organized to handle that scale of work. It's a new way of working.

We all have a team of like 100 agents at our fingertips. And so how do you manage that? It's not just through a list of chat sessions.

It's not just through Slack or something. There's something new. So that's what we're trying to do with Paperclip.

John: I remember coming across Paperclip after I had gone on kind of a deep dive in Gastown and probably put myself in AI psychosis and beads and all that stuff. But what I really appreciated was just the nice approach that you're talking about. I think it's worth double clicking into.

What is the general approach for a team of agents or a company of agents? How does that differ from some of the other things that people are constantly speed running these days?

Dotta: If you just look at the UI, Paperclip looks like a task manager, which you will immediately understand. And it's easy to underestimate it and be like, oh, this is just Linear with agents. And I think, fair enough, that's how it looks. But when you look under the hood, we're trying to answer these four questions.

How do you understand what your agents are doing? How do you control what they have access to? How do you evaluate that the work that they do is actually good? And then how do you improve their performance?

And doing all of that within a multiplayer infrastructure. Most of these agent tools are not truly multiplayer. Let's say that I set up an agent that can read my Gmail and you're my boss.

Can you create a task for my agent that can read my email? Maybe, maybe not. It depends on the company that you're on.

And so really the organizing principle of Paperclip is, the agents will create more work than we can ever review. And that it's not just coding, that all jobs are going to go the way that programming jobs have gone this past year, right?

I'm sure you noticed, right? In October or November of last year was really when I feel like coding was solved. Maybe not engineering as a discipline, definitely not, but this idea that you're going to type out each character, that was solved.

And so I, as a programmer, really had this identity crisis, right? And a lot of us did, where it's like, the computer can now do this job, what I thought was my job, better than I can. And so what's left of work?

And I read this paper by Christian Catalini, which was this sandwich architecture. Humans declare their intent, then something does the work, and then the humans review the work. And in the middle, what you have now is this growing body of the agents do the work. So humans declare their intent, agents do the work, which is growing in capacity every day, and then a human verifies it.

And so Paperclip was designed to be like, oh, that's what's left of work. As long as humans are involved, I'm going to declare my intent and I'm going to verify the output. And so I need a tool that can capture that layer. So I think that's part of what we're setting out to do.

So it's multiplayer. It's built in with a lot of permissions that you need for real world work. It's got this structure to be able to declare your intent, connections to all of the agent things so the agents can do the work, and then verify it at the bottom. So I think that's part of why it resonated with people.

Brian: Yeah, so I get the control plane thing. And I originally saw the earliest conversations, maybe it was you guys, maybe it was somebody else, about these AI organizations where each agent has its own role. Are you guys building the agents, or are you just connecting the agents? Basically, what is the product, actually?

Dotta: So Paperclip itself is more of a horizontal control plane and less of a vertical agent. So there is no Paperclip agent. It's designed to plug into Codex, OpenCode, Claude, Hermes, OpenClaw. Basically, bring your own agent.

Again, this idea was any agent, any harness, anytime. Maybe we'll create our own agent, but the idea is how do you capture what's out there under one roof and get them to communicate with one another.

And so with Paperclip, part of the product is the front-end UI to invent this new way of work. And then there's also the back-end, which is basically an agentic OS. How do you deal with sandboxing for all of these different types of agents? How do you deal with the permission systems and the role-based authentication?

So the product that we have now, well, for one, it's open source. So there's nothing that you can actually pay for for Paperclip, and it's almost entirely open source. We're starting Paperclip Cloud, which is basically a hosted version of what's already open source. So you can host it yourself or let us host it.

The organizational question that you're referring to is around this idea around human org charts. So Paperclip supports this idea that you can have an agentic org chart also mixed with a human org chart. It's less about the humans reporting to an agent, though I guess you could do that, and more of just parallel org charts.

I essentially think that, okay, there's always this big debate. Are human org charts useful for agents or not, right? Because everyone's in this reevaluation period where we're saying agents don't need a human's org chart.

And it's like, well, how sure are you? Part of the reason why humans have org charts are almost fundamental to nature. And more than that, I think the agents themselves already have human org charts based in the weights, right? If I take an agent and I say, you are a security agent and you report to the CTO, there's a huge amount of behaviors that are actually baked into the weights already, so the agents know how to behave.

I also think that you find that the different agents have different personalities, or the different models, excuse me. If you use Claude, you know that the personality is totally different than Grok in terms of what it thinks is appropriate or what its judgment is. And you also have this issue of credential access. You don't want the agent that has access to your health data to also be the one that's connected to your Twitter account, right?

And so these org charts solve a lot of problems very naturally. It's not just that it's a human org chart. It's just that the human org chart was invented to solve things other than just our own limited intelligence and time.

So I think another piece is just humans need a human face. The context of these agents, the labs are all aiming to basically make a wish fulfilling God, right? That you have one chat where you say what your wish is and they deal with all of this structure underneath you for you automatically. When they can do that, you do have to ask how much are humans still involved in the work at all, right?

And so I'll admit that if we reach the singularity, maybe you don't need Paperclip. Paperclip is probably a transitionary piece of software until the singularity is reached.

But until then, you need this kind of tool, which is the new productivity suite for the AI era. You need a way to specify what agents have access to what. You need a way to talk to the agents that you know can do the work you want.

And then they have limited context. They don't have infinite context. So you need a way to partition that within different personalities, but still operate within one organization.

John: Yeah, this was something I think about honestly with building product out in the open. Open source is the whole idea of betting against the model, which I think is something that Sam Altman said. And if we get to God GPT or not, I think we can assume that models will continue to get better, smaller, faster, on cheaper inference, and probably just be more of these things running out in the wild, doing more stuff.

How do you see your company, Paperclip the product, the open source projects, or just at all, adjusting to that? How do you not bet against the model? Is it really leading into what we have today? Or is it some future cast site that you have?

Dotta: I think that we're betting on model diversity, that you're going to see more different types of models. We've seen that this week with Jev. I'm not trying to make the claim that it's a general purpose model exactly, but you understand what I mean, which is we're seeing new innovation in the way that models are created and used.

I think we will see, especially in the next 12 months, a greater move towards local models, right? My M5 Mac can run Qwen pretty nicely, 20 tokens a second. And you're just going to see that get faster and faster. And what that means is you have a proliferation rather than a concentration of models.

You're also seeing essentially every SaaS service, every startup try to create their own agent. And really there is this idea of specialization that, again, until the singularity is reached, the researchers, even at the frontier labs, are somewhat limited in their area of expertise. They're trying to get better already, but many of them are scientists rather than domain experts.

You don't need your coding model to know when the Treaty of Versailles was, right? But a God model has to.

And so there's this future where you have more agents, more integrations. It gets spewed everywhere. There are more vertical agents that understand their specific domain. There are more models. And so that's why something like Paperclip becomes even more necessary. Because you might as a company be perfectly comfortable just saying, we're happy with Claude managed agents.

We're happy using the Claude models for our whole company, and we're going to use the Claude family and we're all going to use Claude. But as they come up with more project management and task management tools, and as every computer job is now an agentic job, so all of your computer employees are managing agents, are you really going to commit your entire company to Anthropic? All of your project management, everything?

Companies are already starting to become a bit wary of the concentration of power that these model labs have, especially when a lot of times they're training their models on your secret sauce.

And so this layer has to be neutral. It probably has to be open source.

John: Yeah, that's such a good take, especially when you put something in the commons to be able to let other people using it. We recently went in on a bunch of DGX Sparks because we arrived to the same conclusion where as a company, we want to hedge some bets on the closed labs all having all the inference, having all the data, doing a lot of the stuff that now is so critical to day-to-day operations and running stuff in a fully agentic and AI-focused company.

It gives me a lot of faith in open source as a thing going forward. Anyways, that's my soapbox.

Brian: Yeah, so if I could restate what you were saying about how you guys are approaching this problem, and I guess maybe potentially the future cloud product. Humans are still in the loop today. So you get a span of agents or you get a team of agents and Paperclip solves that problem for you. People have managed to see what's happening.

Do you see Paperclip also navigating us in the space where zero humans in the loop are potentially possible?

Dotta: Yeah, absolutely. So the original motto that we had for Paperclip was "the open source software for zero human companies." And I think that helped make kind of a splash in the early days. And we've moved away from it. And I think we're going to move back into it.

So I'll explain. Really, the original thing was open source software for zero human companies. And the expectation that a lot of folks had was that Paperclip was itself an agent, right?

And so you'd have people expect, like, I wish it would do more. I say that I want to run a BrainRot t-shirts shop on Shopify. And I just expect you to keep working until that's done.

And one of the things that we found pretty early on was we had this focus: if we were going to go with a vertical agent, okay, here's an agent that can make an e-commerce store for you and market it. Or were we going for more of something that was compatible with a lot of different pieces?

And so we basically chosen to solve a lot of these, I don't want to say just infrastructure, because that makes it sound like a backend tool, because there's a front end piece as well. We're aiming to be like, Slack is a really important tool for everyone in your company to come and chat, and Paperclip will be that for agents.

We weren't trying to build a coding tool. I already don't read most of the code that I write. It's the front-end tool like Slack, but for every other job that you're doing with agents.

And so I'll say that the Zero Human Company framing came with some expectations. At the same time, it's also quite provocative to people who have jobs and like them. They're like, I don't want to be put out of a job.

And similarly, a lot of people who are using agents are doing it with their team. Paperclip is a human company. We have seven people already and we need to hire more.

And so calling it tools for zero human companies is kind of wrong, because actually what's happened is that everyone's ambitions have just leveled up. It's not that you actually ran out of work and these agents are replacing you. It's just that you're doing 10 times more, or you can.

Brian: Yeah. Like six months ago, I appeared on Jason Lengstorf's four web devs and the idea thing. It's a YouTube channel. And we were talking about the transition from, and this is off air, but we're talking about, okay, you run this thing. It's basically a hackathon. You have four hours, people build things.

I've also talked to MLH about the same problem where, what has AI done to hackathons? What has AI done with people with ambition?

And it has 100x people, where most people would be like, I'm going to do this little small, make the widget move in four hours. Now it's like, oh, I've got a full-on deployed scalable infrastructure as a demo for the hackathon. So ambitions to 100x, maybe that stuff never at a work or maybe it's not scalable or maybe just a lot of cleanup. But when it comes down to my title at the company we're running is called CEO. So I do a lot of sales, I build a lot of side quests, and tech debt.

And all those things give me a lot of hubris to say, oh, you know what? I heard a thing. I did a thing. I hand the thing over. I heard a thing. I did a thing. I hand it over. And then now the team's just like, hey, dude, which one of these things should we be juggling?

So yeah, we're in a very interesting in-between times as you're navigating these mottos between what you had before to what you have now to what you go back to. And I think everyone's probably in a very similar position that even Paperclip is, versus engineer on a team who was told to use tokens and don't stop.

Dotta: Yeah, because they will create whatever you ask. And sometimes it's good and a lot of times it's bad. And I think that's something that we've run into too as a company is adjusting our expectations of quality, because we're craftsmen. Most of us are a bit further along in our career.

I've been coding for 20 years. And one of my co-founders, Scott, was the head of product design at Pinterest. And so he has this incredibly high bar for, when I publish work, this is where it needs to be. All of us have it, in Scott especially.

At the same time, if you take everything to 100%, you actually are throwing the brakes on the speed at which you can publish, especially relative to where the expectations are right now, because there's something new every day.

On the other end, though, if you just simply never look at your PRs, just slop it from Claude Code, hit merge and ship it to production super fast, you get an unmaintainable rat's nest that basically doesn't work. And so really what we're working through, which is a lot of people, is how do you build your internal software factory and then subsequently everything factory.

But we'll start a software such that you're able to build the skills that you need, the agents that you need to do code review, you have the guidelines articulated well enough such that you can get as close as possible to quality work. By the time the agents are done with it, you really just have to click through it.

John: Yeah, interesting. Do you see any potential knock-on effects to that shift in prioritizing the craft of it? I've been thinking about this tweet all week from Thorsten Ball, friend of the pod, who said this long thing about how there used to be all these shoemakers and cobbling was a profession. And yes, there's a handful of people still making Italian leather shoes, but for the most part, people are not using those. Everybody's just getting off the shelf stuff.

And the reason I've been thinking about it is I'm like, yes, but there's been a lot of bad knock-on effects to that. Not to get too crazy here, but child slavery and children in the factories making these shoes in some country. I don't know where these shoes are coming from. And it's a weird through line on a bunch of that. Do you see any weird through lines to the craft dissolving?

Dotta: Yes. There's this idea that the market cheapens down to people's average taste, which is quite bad on average. And so craftsmen who have nice shoes care a lot about the quality of the details, but most people just want to get the cheapest pair of shoes that works for them, right? And so we've seen this through all kinds of durable goods, especially post-COVID, I think, that just the quality has gone downhill.

You will certainly see that in software because of the tight connectedness of social media. There's this effect where almost the attention substitutes the substance. And it's better to win the day on Twitter than to actually like a product of substance sometimes.

And so it's a problem. I think a more durable problem that I'm hearing from friends is that non-engineers are starting to use Claude Code in particular to build internal applications. I'm not gatekeeping programming at all.

I love it. I think it's an incredible thing that the code becomes almost like a new literacy, that everyone has the power to code things on their own. And it's not gatekept by programmers. In complete support of that.

But a fallout effect I'm hearing is that basically the IT guy makes a new CRM, the company switches over to it, and it basically doesn't work, right? And someone who maybe doesn't necessarily have the training.

Some of these things that the craft of programming has taught us around unit tests and user testing and good abstractions, it turns out maybe you still do need those to ship quality software.

Will you still need that next year, or will Claude take care of that for you? I'm not sure. But for now, I think we are seeing, I don't know, maybe a pushback to some of the over-enthusiasm. I don't know, what are you guys seeing?

John: Yeah, the time in the saddle, I think, still is the number one differentiating factor with your 20 years of experience versus somebody just ripping a bunch of Claude Code stuff or something. I don't know. I see a wide, wide spectrum of people, even still today, saying, yeah, I tried Fable and Opus 5, and it was bad.

And I don't think this bring a lot of value in general, or I have to go back and rewrite a lot of this code. And I'm like, well, I just don't look at it.

Dotta: Yeah, exactly. And you move faster that way.

John: Yeah, yeah, yeah.

Brian: Another analogy outside of Italian shoes, but when I was 18, I got my first car. I did everything. What I could save money on, I was in college. I learned how to change brakes. I changed oil. Anything I could save some dollars on, I did.

And I got a bunch of reps working on a car. Do I work on my car today? No, I do not. Because I have the value.

I understand how things work. If that tire has a little bit of gash in there, I know how far I can get with it before I need to actually pay for a new tire. There's a lot of stuff I learned through at-bats.

And I think in a world that you didn't build your car yourself. You didn't even build the car or work on the car with your dad in the garage because everyone's got a new car or whatever. Not everyone, but you know what I'm getting at.

So now we're at a point with engineering, I think for those who want to know, they can. So just like if you want records, if you want the classic car, if you want to stick shift, there's an avenue for that.

But there is a world where people are going to be on the rails and have your 100 agents constantly slopping it up. At the point that you want to go figure it out, you will, but you also might outsource that to your...

Maybe you could get your oil changed to Walmart, but you also get your agents fine-tuned next door. I don't know if it's that extreme, but IBM wasn't always doing what IBM's doing. Nintendo was playing cards. There might be another company that emerges, like Chipotle. It's not a place to get your agents slopped. But you also get your slop bowl.

John: People were getting those free tokens from the Chipotle bot. Do you remember that?

Brian: Oh yeah, from the chat app.

Dotta: Exactly. There are a lot of new jobs coming up every day. And a lot of agents introduce a lot of problems around tool access, memory, sandboxing, just integrations in general, how you compact their context, what jobs they can do. It's just an incredibly exciting time to be an entrepreneur.

And I think this idea that anyone can create a program is obviously a net good. Controlling computers have been too far out of reach for too long.

John: Yeah, that's a good take. You had mentioned a bunch of stuff in Paperclip around authentication, authorization. Where are we at with agent AuthZ, the general ability for somebody to give fine-grained access to an agent? It still seems like a really hard problem.

Dotta: It's getting better. So there's a couple of pieces. The first thing people start out is they run agents on their Mac, and then they realize, this is bad because it has access to everything that I ever wanted. And so I guess I better buy a Mac Mini.

And so you buy a Mac Mini, but, I'll say this a little tongue in cheek, buying a Mac Mini is almost the dumbest form of sandbox. It's helpful, but the longer that you use that machine and the more access you grant to that Mac Mini, I don't see how it's any different than your laptop eventually.

Right? And so that's why you need a richer piece of software that says this agent has access to these secrets and these services and no more. And they're in a sandbox and there's no way to break out of it.

So yeah, not trying to talk my own book, but I'm just saying I see those problems, which is part of what we're doing in Paperclip. But there are other people working on it too, at different layers. For example, one way to think about it would be permissioned proxy tools.

So you have Composio is one, Arcade, Executor.sh, Vercel Connect is one, AWS has a product for that. And then Paperclip. Paperclip's open source, though of course we don't have as many connectors as some of those other platforms yet, first party.

But essentially what you want to be able to do, even when you look at, for example, GitHub access tokens, I don't know how technical your audience is, but with GitHub, when you get access to PRs, getting access to read your PRs and merge PRs is the same permission. That's ridiculous. And so if you have something like an MCP server, you can actually separate those out and say, well, you're allowed to read my private PRs, but you can't write to them.

And in Paperclip, we have the ability to, per connection, per agent, assign approve, deny, or ask, right? So you're allowed to do this, you're not allowed to do this, or you have to ask me if you want to.

And the way that you would do this in general, we do this in Paperclip, is you have the control plane, which is secured, but no agents run on the same box as the control plane. You actually launch the agents in a sandbox, and the sandbox basically grants those permissions back.

Now, when you put agents in a sandbox, you lose everything that made your agents powerful. Now you're like, okay, fine. I've constrained access to my MCP server, but where are the files that this thing just made?

I asked you to make a video. Where is that video? And all of the structure around identity, memory, keys, you lose all that.

And so you do need some sort of control plane. To get that architecture that you want, that control, just to turn off one MCP tool requires basically a lot more infrastructure.

But that's one of the things we say is trust is an infrastructure problem. If you don't trust your agents to do work or have access to your accounts, it's because you don't have the right tools available to you.

Another way that people do it would be HTTP proxies. So again, tell me if I'm getting too technical here, but often when your agents make a connection out to the internet, they will often pass a token in the header. And one of the things that people will do is they will have a proxy which sits between them and Instagram or whatever.

And when they go to post a message, instead of giving the agent the complete keys to Instagram, they actually have to go through a service that checks and verifies the permissions, and then it passes it on. And so not every single API is supported through a credentials proxy like that. But that's another way that we're seeing people do it.

John: Yeah, the ye olde gateway architecture, which we're all too familiar with at the Paper Compute Company.

Dotta: How do you do it at Paper Compute?

John: Well, it's mostly on Envoy. We're mostly focused on inference. So you stick a gateway between the client and the actual inference provider. And then you can apply policy, do compliance, logging, tracing, etc., etc.

The actual permissions bit is interesting because you're mostly concerned with what model maybe somebody picked, a client maybe specifically has access to. So once it goes through the whole job policy and blah, blah, blah, authentication, you can then authorize that specific person for a certain model or apply a certain policy to it.

So there's all different kinds of things you can start to do on an AI gateway that unlocks a lot of capabilities, especially at the bigger organizational level where maybe you have teams and teams and teams of teams and all of those cascading things. But it's an interesting space to be in because it's ever-evolving and ever-emerging, as I'm sure you're aware.

Dotta: Yeah.

Brian: So I had one question before we go to reads. The open source project, how is the project maintained? Because obviously you're now this post-world of agents.

It started in this world. I'm always fascinated to understand how do you take all the noise. How do you manage the noise? Do you have an agent? Or do you have a Paperclip that manages the repo?

Dotta: It's funny you should mention this because I got a DM not half an hour before this podcast from one of our investors who's been helping us with managing these PRs. I would say that it's been extremely painful.

I don't know the exact number, but I think we've had something like, let's call it 5,000 PRs from the community. And it's bad because there's duplication. They will submit a PR using a really old or cheap model. So the code is bad.

We have people constantly trying to hack us. And not only will they introduce vulnerabilities, but we use Greptile, which I love. Not paid at all. It's critical to our infrastructure. We're leaning hard on Greptile.

But one of the things that'll happen is they'll submit a PR, Greptile and all our checks will run it and it'll pass. And as soon as it passes, they submit another commit that introduces the hack essentially. And so you actually have to set it up to, okay, if someone submits another commit, we have to rerun all the checks again. And it's not cheap. So you're actually paying all of your inference for random PR that you have no knowledge of the quality of what's going in.

And so it's basically this asymmetric warfare essentially. But then at the same time, you still have some people that do a really good job. They're being really careful. They found a real bug that existed or they added a feature that you wanted and it's architected good.

And it just gets lost in the noise. And so a lot of it has to happen a bit socially.

Kind of route things up. We've created a bunch of agents that try to filter things for us, try to reproduce bugs. We're starting to lean towards this idea that issues might be better than bugs. Because what happens is, if we get a PR, then we check it out. We check out the code before the PR, you run it and try to reproduce it. Then you apply the PR and see if it fixes it.

And then you fix the PR to fix it. It's just, why am I even using your code at all? You just tell me the problem and I'll just fix it with my own tokens. And so I don't know. I don't think we're going to turn off PRs because it is helpful because you essentially have other people paying their tokens to make your product better. That's part of the point of Paperclip is that it's a community place to come around and deal with this.

What is good is when a new agent harness comes out, somebody submits a PR to Paperclip for it, right? And that's nice. And so we want to maintain that. So I don't think we'll turn off PRs, but I will tell you that it's painful and we're learning. We're definitely not at the level of the folks like OpenClaw where they'll merge 5,000 or close 5,000 PRs in a day.

I will say that I think the products are a bit different. We have a front end and it's really important that the UX is good. We care a lot about what the front end is. Not that OpenClaw doesn't, but there's more backend there than their UI essentially.

And all that to say, they're just really good at agentic development and they also have an unlimited token budget whereas we're operating a bit more like tighter constraints.

Brian: Yeah, actually, I'm close to the OpenClaw team and I'm very familiar with their take on that. So hopefully...

Dotta: Tell me their secrets. I need their help.

Brian: Yeah, I'll definitely want to introduce you to them and you guys should definitely talk. Thanks for the insight. I probably have so many questions.

John and I both also worked at a company that managed open source data and helped support some of the stuff pre-AI.

To be honest, I'm kind of glad we stopped before AI was a thing because it's a different beast.

John: Yeah. If those GitHub charts are to be believed, it's like 30, 40x or something insane. Straight up hockey stick of data, PRs, issues, just stuff happening on the platform. So my hat's off to GitHub. I'm kind of astounded it's up at all. So downtime may be deserved.

Brian: Excellent. Well, Dotta, are you ready to read? That's my question to you.

Dotta: I'm so ready. I'm a good reader.

Brian: So, John, you have quite a few reads. I basically have the same read, but you go first.

John: Well, I think probably the biggest one that we'll talk a little bit about, mentioned already, is Jev. Very exciting from TypeSafe AI. It's a generalized classifier model, basically. Give it a bunch of options in JSON, and then it can provide back to you the probabilities of what it proposes to pick, essentially.

Kind of undersells it, but it's really cool with some of the stuff people have been building. The demo that I linked to in X here is Jev playing the trolley problem and picking to save the masses more often than not. So Jev is moral, maybe? Or just has some training data in there from philosophers.

Brian: It's been labeled correctly.

John: Yeah, exactly. What have you been doing with Jev?

Brian: So actually, I played Tetris using AI, and I used Jev to basically identify... I guess a lot of these demos came out pretty quickly. It was a lot of these game demos and the trolley demo.

So Jev is definitely great at playing Tetris. But I identified when Jev hits its latency limit or its window, and then starts handing over to the model. So I was just trying to figure out what the hypothesis of when to use Jev. Mostly always, but there's a point where you get to the later stages of the game.

So if you run a perfect game of Tetris, it could run you about two hours. So Gemma 4B is a great model for Tetris, but Jev could also play Tetris for about a good 50 to 60 minutes. And then once it gets below 70 confidence, which is essentially either panics and it's got itself in a corner or whatever, because it's not random, but it's a bit random with Tetris. It would just hand over the model whenever it draws below 70%. So it was just fun to see that.

I've also done a bunch of labeling, legitimate labeling data within the paper platform as well. Actually, the one thing I've actually done is I labeled my email. So obviously, Gmail has your priority inbox. It's always off. It's wrong. There's stuff that gets buried. I had a Carter email in there that got buried for a week.

So I started doing Jev labeling. And then everything that's below 70%, I created a Tinder app that I actually run on my phone. It's all local, but I swipe left and right if I should follow up or swipe right if I should archive it. And it started to train Jev to add new labels.

So that was actually a fun little maybe 30-minute project that I just slopified. But yeah. Dotta, are you Jev-ing?

Dotta: Yeah, yeah. So I think my main thought is around just Paperclip. I think we'll see competitors for Jev within the next probably 60 days. I won't be surprised to see them.

Brian: So today, this morning, I could mention a couple of them.

Dotta: Oh, I'd love to see them, yeah.

John: And almost right away, there were a bunch of these open source ones that just appeared.

Dotta: And I don't know how to even eval Jev exactly, or what they've done there, what the open source ones look like, but they're so cool. So I do think it is a good platform to build on.

Obviously, we've had classifiers forever, but it's like we almost rediscovered, oh, hey, these can be used for general purpose classifiers without retraining.

Some of the things, of course, in Paperclip is just tools and skill discovery. That's low-hanging fruit just to do that a lot faster.

One of the ways that Paperclip is different from just sessions in Claude Code is rather than being an infinite chat, you create tasks. And so the agents work on a task that has a status. Is it to-do? Is it done? Is it in progress? Is it blocked? Does it need to be reviewed?

And so that's a clear case where Jev is helpful because the LLMs will make a guess, but with Paperclip, you're not always using Fable or Astra. You're using cheaper models too. And so it's nice. Sometimes those cheaper models make a mistake.

So it's nice to have this cheaper classifier to say, is this task done? Does this need to be reviewed? If so, by who? So yeah, Jev looks really promising for that.

John: Yeah. I bet that the Jevs or the Jev-likes will get more integrated into harnesses. Another use case that I think would be amazing is deeper integrations with MCP, actually, where I've found with any number of maybe past five or ten MCP servers and all those tools, just a lot of context bloat. But some way you could select without over-bloating the actual LLM driver with all that tool context. Maybe Jev could do it. Maybe we could be Jevving with MCP. Who knows?

Dotta: I think that seems right.

Brian: I'll mention the two that I had, my reads. One's Kev. So Jared Palmer, who's head of engineering over at Devin, Cognition. Created Kev over the weekend. It's about two days old. Got a couple thousand stars. But it's exactly what you think it is. It's Jev-like. And it's open source written in Python.

Yeah, I haven't tried it yet, but it's bookmarked. And it's built on top of... With Qwen in mind, basically. So if you use local models, you know Qwen's kind of... It's a bit of a powerhouse. It could do a lot for you. And also runs quantized on Macs if you need it to.

So the other one that just popped up on my feed this morning, but I guess it's been out for a couple of days, it's called laya-mlx. And this one's coming out of somewhere in China. I actually probably should have taken note of that because someone called it out that the Chinese labs are on board. It's a new paradigm, I guess. It's been in existence as far as data labeling.

John: Yeah. But the way I'm thinking about it is the transformer was around for a long time. And we had neural networks and all these things.

This is kind of a ChatGPT moment for classification or classifiers with a really nice API behind it.

That maybe goes understated for TypeSafe is that they did a really slick rollout. They had people on a waiting list for two days and then got everybody access. And just deep integrations almost immediately on API.

So it's kind of a ChatGPT moment for this, even though BERT classifiers and a bunch of these were around. Or even, you could get an LLM into code mode or JSON mode on an API and basically get it to do the same thing. But because it's not a large language model necessarily, you get a lot of the benefits of not getting distracted by a bunch of the other things in this context. Yeah, Jev, it's great.

Brian: Cool. John, you got another read?

John: Yeah, a couple other reads. So another thing I think really exciting this week is Google is working on agent sandbox infrastructure or something, AX, with this thing underneath it called Substrate. And this seems to be what a bunch of these people deep in infrastructure in Kubernetes and a bunch of these things at Google have been working on. I definitely note that the agent substrate project has a bunch of these legends from the Kubernetes community working on it, like Tim Hockin and Ben the Elder.

Shout out to those people. But it's very interesting. I wonder how commoditized some of these things end up being as we maybe start building more on top of infrastructure.

It seems like the kind of default answer here is that we're going to build this AX thing on top of Kubernetes with some good APIs and definitions, then you can do all the sandboxing stuff that you need and eventually get it onto other platforms or other hypervisors or clouds come and adopt it.

I'm not exactly sure what it is. It's one of those things that you look at and you're like, I see some YAML. Not exactly sure what I'm doing here.

Dotta: Yeah, we were looking at it this morning for Paperclip because these are problems that we have. You kick off a task, you want it to run. Let me tell you a couple challenges, which would be, one, the files.

Your agent writes files. Where do they go? What's the persistence model? If you have two agents of the same identity and they edit the same file, what happens?

Can multiple teams of agents work on the same file at the same time? Number two would be computer use. I want to be able to have a computer where either multiple copies of the same agent or multiple agents can work on the same machine. I want to have my React app right there in the browser and it's hot reloading as my agents are working on it in between terms.

Number three, let's say that my agent actually has to monitor a task. He'll see this with ChatGPT. You say, create a PR, and it monitors a PR and says, I'm going to go to sleep now and I'll check back in 30 minutes. So the turn ends, but you still actually have a shell process running under the hood and you actually need to resume that to finish what you're doing.

So how does that work with AX? Because they say, you fire off a task and then it's sandbox and your agent does it. How do you deal with those three? How do you deal with the files? How do you deal with computer use where it's shared? How do you deal with timers and monitors? I don't know.

John: Yeah, it's a good question. We got to give credit where credit is due, project's two days out of announcement. But I'd imagine the answer today is a bunch of spicy Kubernetes.

Dotta: Yeah, there you go.

John: And see where you can go from there. But yeah, it looks interesting. It just makes me wonder and think how some of these other big sandbox providers that do a lot of this will get encroached on. Stuff is moving so fast.

It's crazy how much value, not necessarily getting destroyed, but speaking of open source, just gets commoditized or a thing that just becomes a part of the commons.

And I think about this with containers. I'd be kind of shocked if anybody was paying for a container platform, a true bespoke container platform that wasn't Docker or Podman or heck, just Linux under the hood. Maybe agents will get so commoditized, or at least the underlying paradigms like a container, that underneath all of it you're just running some open source thing or something that's become ubiquitous. Who knows?

Do you have any spicy takes, Brian?

Brian: No spicy takes. I think it's very interesting. The person who tweeted at Rackle, I worked with in passing at GitHub. They kind of did a short tour there and then went back to Google and they're working on the right thing at the right time.

So I'm glad to see that Yama is some cool stuff. And obviously, there's a whole team behind her doing this. But yeah, no spicy takes, very intrigued by this world.

I think we could probably say that, maybe Dotta, you could also agree, agents are not going away. This new paradigm shift, you mentioned you were in Web3 for a moment. There was a lot of debate on whether Web3 was important or whether it's worth it. I think ship has sailed on the agent side.

Dotta: Yeah.

John: So yeah, one more read from me. This is a great little post from US Graphics, which is this tiny company that made the Berkeley Mono font, which you wouldn't think that 70 plus dollars for a font would be worth it. But one of the best licenses I've ever bought. I love Berkeley Mono. It's a great font.

But they had just this fun font, I guess icons package, which is on GitHub. It's the BeOS R5 icons. And it's a bunch of these early 90s fonts from a collection of older operating systems like GNOME 2 and KDE Classic. It brought me so much joy looking at all these fonts because I'm a 90s computer kid at heart.

Brian: Very cool. Yeah, these look like Slack emojis. That's a meme waiting to happen.

John: Waiting to happen, yeah. Especially the burning disk one. Which I guess you needed when you had a disk drive that you were burning a disk to, right?

Brian: Yeah. Man, I actually have a collection of Lord of the Rings and Dragon Ball Z DVDs here that I'm trying to get digitized. And I had to buy a CD burner so I can get all that stuff off there. So it's a little weekend project that I've got the first half of the series in. But it's monotonous.

John: It's crazy the I.O. on a lot of hardware these days. It's just created a lot of lost media. I wish a laptop still had at least a Blu-ray player. Most don't.

Brian: No, I actually researched it because I have a couple of PCs that I could have just shoved one in, like this insert from the back. But I ended up getting a USB-C one that I could just plug and play and make it work when I need to.

But yeah, getting all the stuff off DVD and I'll probably end up setting up a little server at the home so I could show the kids all the cool things that they never knew existed, like Lord of the Rings. I literally have never watched that as a family. It's the extended cut too as well.

John: Oh, nice. Man, get on that. Worth it.

Well, Dotta, thanks for joining us. Before we go, where can people find you? Where can people find Paperclip and the open source project?

Dotta: Yeah, you can find our website at paperclip.ing. You can find me on Twitter at Dotta, D-O-T-T-A.

John: All right, very nice. Well, listeners, remember to stay ready.