Ep. #37, Is AI Killing Open Source Software? with Stormy Peters

Brian Douglas: Welcome back to another installment of Open Source Ready. John, how you doing? You got your tea?

John McBride: Hey, I got my tea. I'm doing pretty good. Brian, I have a quick question. Are you a fan of Stanley Kubrick by chance?

Brian: Stanley Kubrick, the director?

John: Yeah. 2001: A Space Odyssey, Eyes Wide Shut.

Brian: Oh, yeah, yeah, I've seen both of those. Yeah, so I guess so. Yeah.

John: Okay.

Brian: Historical movies. I watched them in college for a film class, actually.

John: Oh interesting. Yeah, my wife and I have been going to these free lectures they've been giving on Stanley Kubrick, and we were at one last night and it was kind of crazy.

Brian: Oh, amazing. You live next to Yale. I live next to UC Berkeley. I should probably take more advantage of being so close to a prestigious university.

John: Yeah, it's excellent.

Brian: Speaking of prestigious we've got a guest. Stormy Peters. Hello. How are you doing? Welcome.

Stormy Peters: Hey, Brian. Hey, John. Thanks for having me. I remember reading 2001 when I was like, 12 or 13, and my mom told me it was the most boring movie in the world and I would hate the book. And I loved it.

John: Oh, wow.

Brian: Oh, amazing. I didn't even know there was a book. I got to catch up. I've been actually trying to. Usually Fridays I'll try to go to a coffee shop and catch up with some nonfiction. Again, trying to do some palate cleansing of all this sort of heavy AI back and forth, like reading prompts. Need to basically get ready for the weekend.

John: Like a human cleanse.

Brian: Yeah, yeah. So the coffee shop I go to is right across from the library, my local library. So I'm going to go give it a look, see if they got some cool science fiction there. But we're not here to talk about science fiction. We're talking about real life AI. So it's no longer science fiction.

Stormy, I actually caught up with you down at the Linux Fest SCaLE 23x in LA. I got to sit in your talk around Open Source and AI. I think you were at All Things Open as well, maybe early in the year? Or there was another event I saw you at.

Stormy: MCP Summit last week.

Brian: Oh, I saw you last week at MCP Summit. So yeah, you've been on a tear doing not the same talk, but you've been doing multiple talks throughout the years. But I'd love to catch up because I used to work for you at GitHub and you're not at GitHub anymore, you're at Amazon.

So you can just introduce yourself, tell us what you do at Amazon, tell us what you're interested in and then we can talk about the content of the topic today.

Stormy: Yeah, so I'm at AWS now. I lead the Open Source Strategy and Marketing team. So about half of my team works internally into AWS, helping AWS teams do open source, whether it's open sourcing something, consuming it, and just helps them make sure that they're doing it in a way that works for Open Source and works for AWS, to be good citizens.

And the other half of my team does events and blogging and social media. So outbound AWS messaging and Open Source.

Brian: Yeah, yeah. And I mentioned GitHub. You got quite the background. You were at Microsoft. You were also at Mozilla as well. So a lot of really strong open source companies in your background, which makes sense with the role you have today.

But I'm curious right now. You've been giving a talk. Can you remind me the title of your last talk you did at MCP Summit?

Stormy: I didn't give one at MCP Summit, but the talk I've been giving is called Is AI killing Open Source Software?

Brian: That's what it was, yes.

Stormy: It actually reminds me of a talk I did really early in my career, which I was worried that giving maintainers paid jobs working on open source was going to kill open source. I also made it the theme of a talk and did a whole bunch of research. And so I like these big questions, lots of research, lots of conversations. They're fun.

John: Yeah. Do you feel like we're in the midst of that still where paid maintainers are, You know, we're still evaluating if that was a good thing, a bad thing? Do you feel like we can evaluate that?

Stormy: At the time I decided that paid maintainers didn't kill open source.

I was worried about like the intrinsic versus extrinsic motivations for working on something, and I thought maybe the pay would replace the intrinsic motivation. I concluded that if you got paid to do something and they stopped paying you, you might switch projects.

Like, you might not feel like that project just wasn't-- Like no one's paying you to work on it anymore. But it didn't kill motivations. I think we're facing a different problem now with maintainer burnout and just the amount of usage of things that are written by a few people is a very different problem.

John: This really reminds me of a conversation we had with Davanum "Dims" Srinivas in the Kubernetes ecosystem, who's at Nvidia now but has been leading up, you know, a lot of community stuff in the Kubernetes ecosystem. He was at VMware for a long, long time, was at AWS working on team for a while too.

But that conversation was really impactful for me because it was like, "we need maintainers, we need help." And there were a lot of people in that VMware exodus to Broadcom who just kind of went hands off on Kubernetes because it was like that was their job, was upstream open source for Kubernetes.

And then it was like, "well, I'm not getting paid to do that and it's a big complicated project. Don't know if I want to spend my nights and weekends working on it." Right?

Stormy: Well, and also you contribute at a different level. Like if you're doing it at a volunteer level, you're probably not going to work 40, 60 hours a week at it. You've got to also make a living and do other things.

John: Interesting. Yeah.

Brian: I studied finance at school and I remember the last semester we studied the whole, like a SVB bank, had sort of bank run and then like the enacted the sort of too big to fail in action. And this might not be the right overlap of like in the conversation, but when I think about Kubernetes, I think about some of these other projects that are way more mature that you kind of do need full time staff to keep the lights on, manage security vulnerabilities or CVEs when they come up--

But then I also think about the, a story that we were at GitHub, Daniel Stenberg from cURL had a bunch of issues about it's like maintenance and being like the solo maintainer. And he had done that full time, went at Mozilla, did it full time after Mozilla. And then as of recent, in the last 12 months he's actually turned off, maybe he's turned it back on, but turned off outside contributions.

We've also had Mitchell Hashimoto come on and talk through contributions for Ghostty. And like he's created what is it Vouch so that you have to be vouched to be able to make contributions. It's like this sort of CI tool that now it's like embedded into his projects to make sure he can like filter through noise.

And then another thing I saw today from GitHub is now they identify within the issues and PRs the role of the contributor. So do you have like roles that are like maintainer, contributor, outsider? And it shows up in the list so that we can start filtering through like, oh, these are maintainer contributions. And then they do have a bot role now as well. So just shipped today as of recording this podcast.

Stormy: I always thought that on a project you would know all the people that contribute to your project. Like back when open source first started you usually did. And then I was talking to some maintainers who have a whole bunch of projects that they maintain and lots of contributions on each of them.

And some of them keep spreadsheets of everyone and like notes on, you know, what that person has contributed and how the interaction went and all sorts of other things. So the more that GitHub could pick up on that, the better. Right?

Brian: Yeah, yeah. The city I grew up in, just outside of Tampa, we moved in 1988 and there was 10,000 people in that town. I think last time I checked it's about 300,000. So like it's a pretty dense city. It's not even like the major city Tampa.

But I mentioned that because like there's a lot of folks when I go back home that I just know, like I run into, I run into the grocery store, you just run the people all the time. And, and it used to be the case up until I left about 10 years ago. And there's like this huge run of folks moving to Florida where I'm from but I imagine with open source, same deal, it's like you start up, you have the core group folks all sort of like grew up together making contributions.

We kind of all know where all the bad code is. We kind of hang out in the good code sections. But now we have the advent of bots and Open Claws and drive by contributions with LLMs. And I was curious like your take on that, but also how are organizations able to maintain and manage that type of noise?

Stormy: I think they're all working it out right now.

I feel like we sprung this AI thing on maintainers and they were already working really hard, already approaching burnout in some cases. And then AI just amplified the problem and we didn't give them all two weeks off or a month off to go figure out what to do about it.

So we're kind of playing catch up as all of the tools try to help them with AI based tools, and the projects try to figure out how to adapt to all of the contributions and how to use AI. So we've seen projects do everything from say turn off contributions like we were mentioning earlier, to making people attest that they wrote them themselves, to saying that first contributions can only be a certain number of lines.

We see, just because bots tend to be very prolific and submit, you know, pull requests that have lots and lots of code in them, we've also seen projects like I think Valkey and OpenSearch use AI in their review process to try to help with the workload.

We have some best practices, but we don't have like the bible of best practices yet.

Brian: Yeah. John, you just added a review bot to one of our projects. I'm curious your take on how that's going. Like we only had like one PR, I think, so far.

John: Yeah, it's been useful. It's a lot of noise. You know, I think the to noise to signal with really any LLM can be challenging. It is nice just to get like a final, I don't know, kind of like sweep of like, okay, there's no like blaringly bad issues or things in here.

The discourse around, you know, review bots and like just the sheer volume of AI code that a person can go and actually review and read and like you just remove all of the, all of the trees in the forest, you know, to be able to look at all those if it's like many tens of thousands of lines a week, more than that probably, honestly.

So it's been useful at least. I don't know, I hope it gets more useful as these things get better.

Brian: Yeah, yeah. I remember actually talking to some of the earliest CodeRabbit folks and then a couple other tools that came out and very similar. Talked to the Copilot team as well about their review, earliest review Copilot experience. And I think we've come a long way.

What I like is the tools that actually create long standing context for the projects. And I think CodeRabbit does this, Devin does this pretty well. But we can talk about tools later. I actually wanted to, Stormy, get your take--

The signals of what a good project is and what a healthy project is has changed completely. I saw Garry Tan talk about 37,000 lines of code per day. Like is what he's doing for one of his projects. Like lines of code seem obviously, as John was saying 2000 lines per day is like the average. Obviously you can't count lines of code anymore. So what are some things that you see as good health metrics or good signals?

Stormy: Yeah.

It never was about lines of code. Right? Like even though we counted them, that's not super accurate metric of how good the code is.

To me it's still about interactivity, like responsiveness, like is the community talking? Do they respond to pull requests? Do they respond to issues? Even if they're using AI or a bot to respond to some of them? Like, is it an active project that people are participating in?

Brian: Yeah, I mean that's fair. And you also mentioned Valkey and OpenSearch and like their approaches. I'm curious, did they have any sort of like from your purview, like any sort of tips and tricks? Cause I mentioned a lot of projects. It's actually pretty amazing, you can go from zero to tech debt in like a weekend.

So there's some projects that you basically have, you have an idea and then you get a thousand stars out of nowhere. And then now you have like essentially tech debt and users. So I'd be curious like from the Amazon purview, like do you have any tips and tricks for folks on how to manage that sort of conversation and inbounds and contributions?

Stormy: I've shared some of the tips that different projects have come up with. I think, absolutely, like the tool you're using, John-- People should use tools as part of their review process.

I think most maintainers got into this because they liked writing code, not reviewing other people's code. and that's quickly become their job.

So I think using AI to review the tools, looking at patterns-- So I actually started this talk because I thought that people wouldn't contribute to open source software because it was so easy to generate the code. Like if I created a note taking app in like 30 seconds, then why in the world would I put it on GitHub for others to use? Because you could just create your own in 30 seconds.

And then the real concern that people had was the slop issue. I do think my first concern is valid and I do think that is happening. But people were much more worried about the slop. But when I asked people during the talk, most projects aren't seeing a lot of slop. I think like some really big ones are, especially those that offered bug bounty programs or offered money for contributions.

But when I asked people to raise their hand, most people weren't seeing like a huge influx of PRs. And there are actually even a couple projects that said that AI is making it easier to get new contributors because they can make a contribution more easily and ask an AI if it's good and how to contribute it and how to do it.

So I thought that was really interesting. Didn't quite answer your question, but your question made me think of that.

Brian: Yeah, yeah. I mean it just still feels like everything's in progress. Because I've actually, to be honest, I've actually shipped a lot more private projects that aren't open source mainly out of this like--

I had a couple of projects that got like, you know, a good 50 to 100 stars out the gate. Not intentionally like it sounds like a weird flex, but yeah, this is the world I live in, being a public persona. Haha.

John: The Beyonce of open source. There he is. Haha.

Brian: Yes. I'll have to retire that title. But so I've actually like, I just built a OpenClaw interaction. So I have a chief of staff that's my OpenClaw and it's got a SOUL.md. It's got all these tools that are like bespoke and specific to me. There's no like private information in there that would actually like grant being closed source.

But I'm also like, I don't know if I want people opening up issues and like starring the thing that's just like giving me value, that's like my little side thing. I've got a couple other things I've shipped in like the same vein of like, I just want like a quick little calculator to do like a random thing. And I tend to, I have an addiction to ship stuff to GitHub because I'm just so used to it, out of habit of being there so long.

But there's just a lot of-- I've got 460 repos, not because I'm a prolific coder, but more because I just have a lot of ideas I want to set up on the shelf and look at every now and then if I have like that type of idea later down the road. But I'm also, I do have a bit of fear of like, okay, I open source d something, somebody's like following me. I've got quite a few followers on GitHub.

So then if someone's like, oh, hey, can I contribute to this? And I'm like ah, not really like sure, like clone it, fork it, don't force me to archive it right away. But it is because I think the bar of entry is so much easier. The thing I'll also point out is I've been writing a lot more Go code, a lot more Python code.

I've never been full time Go or Python in my previous careers, but I can approach writing bespoke Python code that's performant in the ML and then handles traces and then could go running like a Jupyter notebook because my model knows how to do it and I've seen it happen before so I can prompt it.

So going back to the barrier of entry to make contributions and to share code, it has been lowered.

Stormy: Do you mark them differently if you had a lot of AI help? Because I talked to someone who, he had these, I forget the original language but he'd written a whole bunch of tools that he thought were really useful in a language and then he used AI to move them to a different language. And he was like, "I can't post this because I don't understand this well enough to like say it's mine."

So when you're really prolific like that and you post something that's not really for production, like do you work it?

Brian: Yeah, I should, honestly I really should for the stuff that I do end up going public on. What I end up doing, because like I also write a lot of blog posts and do a lot of like research on my own. This I transitioned from doing sales to writing code because this was my hobby and now it's my job.

So I end up writing like blog post type documentation inside of my code base for my own benefit. So then I've also, I've been getting the model to do that for me inside of like a docs folder. So a lot of that folks can kind of grok what's happening either through the readme or through a docs folder. And I mainly do it for me, but yeah, a lot of it I don't share yet.

Stormy: So I was listening to one of, I think it was your last episode. It was the MCP episode that you all had. And he said they didn't take docs contributions because the docs are generated automatically and docs is often a way--

There's a problem with new contributors in the AI world. So docs was often a way that new contributors got started. And that's gone away. Submitting pull requests was the way they got started and, especially the easy ones, that's kind of gone away because AI can do them. I thought that was interesting.

Brian: Yeah, I mean we are losing some of the breadcrumbs that kind of made open source such an entry point. Like I actually benefit from docs contributions in the React native project and the React project just like finding weird errors and being able to like unblock myself but then sharing like upstream what I found.

But now we live in a world where like that stuff gets discovered. At the time of this recording, like Mythos is not public yet. But like what we're seeing is like with the Linux project we're now seeing Mythos could find, Mythos being the Anthropic model for folks who are following along from home.

But yeah, basically they could find vulnerabilities, zero day vulnerabilities and yeah. So I don't know if we're going to also lose security researchers. I highly doubt it. But we do have a world that sort of shifting quite a bit.

Stormy: There's two other thoughts. So I asked at the talk at, not the talk you saw at Scale, but one at PlanetNix. I asked people what they thought new contributors could do and three different people, like as if they couldn't hear each other in the room, suggested the same thing. And it was that those new contributors review PRs because that's what they didn't want to do anymore.

And then someone joked that probably the AI would review it for them. So it's not really, It's the same problem. I wondered if new contributors could reproduce problems, like if that would be a valuable contribution?

Brian: I mean there was a company I worked with years ago who was doing open source documentation, but by basically archiving a bug and then your first contribution is basically fixing a bug that's already been fixed, but it saves state in different branches, so that you can go through--

You can learn the code base by solving this one bug. And this was pre AI, you had to do it manually. It was like a series of NPM scripts. It obviously didn't take off because I don't think there's a whole prolific folks who are doing this. But the one thing, I was actually just talking to a friend earlier before this call this morning, he does data science and he was asking because he's like getting into more coding because he can solve, he can expand his footprint of what he can do at work.

So he was like what could I do to like level up or like understand like the best practice of AI coding. And my response to him was like just grab a book. And I literally had this book like Grokking Algorithms, which is a book I'm holding up on camera and I've got another one which is AI Engineering.

But my pitch to him is like grab books specific and bespoke to like situations. So at least you know what prompts you should be typing because I think that's one of the challenges with junior engineers and like newcomers within projects. You don't know what you don't know.

And if you don't have the breadcrumbs of good first issues or documentation or reviewing PRs, you've got to go manufacture those by kind of learning about data intensive applications and applying that knowledge to an open source project.

Stormy: And if you don't have time to read the book, you can ask Google LLM to summarize it for you in a podcast.

Brian: Yeah. 100%. Right after you listen to this podcast though and give us five stars. Haha. Before we sort of shift over to Reads, John, do you have anything else?

John: I don't think so. I've been very honestly inspired by Amazon open source recently, especially with like the Valkey. I guess this is maybe just saying like, "woohoo, go Amazon." But you know, Valky stepping up and like creating the community piece of that I think was really cool.

Still seeing a lot of like Kubernetes contributions and stuff. Yeah, I mean it sounds like Amazon is bullish on open source.

Stormy: Yeah. Have you tried out Strands?

John: I've not. What is this?

Stormy: Strands is an open source tool from Amazon that helps you make agents.

John: Oh, cool.

Stormy: So it's fun.

John: Very nice, very nice. I'll have to give it a try.

Brian: Yeah, yeah, I've definitely got to try. I've got my AWS CLI all logged into our company account. So I've got some agents to run this weekend. Maybe I'll shift over my chief of staff to an EC2. Haha.

Excellent. Well, Stormy, I do want to transition us to Reads. So the question for you, are you ready to read?

Stormy: I'm ready.

Brian: Excellent. So John, I've got actually a read. I'll just jump in and this was actually a little fun read, actually topical for us because we've been sort of exploring some open weight models for the work we've been doing. But I do apologize for the show notes folks. the link does require a account to sign in, but I'll read it to you. And there's also a LinkedIn post that you could read a portion of it.

But Shopify Cuts Inference Costs by 75% using Qwen 3 for Merchant Data Extraction. So open weight models for the win. Tobie, the CEO of Shopify has been pretty prolific within the open source world, but also just within sort of being very LLM forward.

But what's interesting, they've moved off of GTP5 to go with the open weight model Qwen 3 which is a really good testament of like using open weights in open source and getting value from it at the same level of what you're already getting for like a large model.

In our past lives John, you had spent some time doing a bunch of RAG pipelines and embeddings and you had this popular blog posts on saving a bunch of money in embeddings and inference by going Mistral models. So I was curious like what's your take here?

John: Yeah, I think especially again like for things that are non ambiguous things that you kind of have a workflow around or you can like really drive the direction for a model via the system message or whatever, you know, RAG stuff that you're percolating into the context window. You can get some really, really good results with small parameter models even, you know this was what like two years ago now that we were doing this.

Even back then had like really good results for some of those AI workloads. That was a lot of like summarizing text and understanding, you know, sentiment analysis from basically freeform text fields, which again anything involving language like pretty great at. They're large language models after all.

So yeah I always think that once you nail a workflow you can kind of like start to do some of that context engineering. You could probably also start to think about like reducing the power of the model I guess or kind of like turning that knob a little bit down, even down to like a Haiku model or a Sonnet model, you can still get great results I think.

Stormy: I think it's awesome open weights, just like open source in general is helping solve these problems.

I was hoping that we would get to this point sooner than later just to save the environment and be able to use AI more effectively, but I think we see this with all technologies. Like I still remember the days of long distance phone calls costing money and now you just pay one fee a month and you can use your phone however you want.

John: Yeah, this actually gets into one of my Reads, which was the second link from Mason Wang who was the founder-- I didn't realize this after I had read this, but was the previous founder at Tilde Research. But basically it's how they went and trained a Llama 2 model which is you know like several generations before, you know what we have now with GPTs and everything but on basically $5 of compute and like with the power of like basically nothing to run on these Raspberry PI 0s, which are very low power compute boards.

And basically built a whole C library just to do Pytorch essentially on those pies, which is called Pytorch. And I just thought was so cool because there's so much software optimization still. I think like people don't realize just the layers and layers and layers of GPU kernels and C code and C++ code out there that just is ripe and ready to be like optimized for various different hardwares and different things.

We see this a lot in the Apple ecosystem with Apple Silicon like every day it seems like there's some news about how there's the next optimization in you know, for an M5 can run, you know, huge parameter models, a little bit of unified RAM. You know, you don't need like an H100 or something.

Yeah, huge plus one, Stormy. I think it's very exciting where we're headed with that.

Brian: Yep I've got three Raspberry PIs down here that I'm actually going to be noodling with this weekend building a little home server. But I'm inspired by this as well. I get a little nervous with the RAM prices going up and the chip cost that I was like let me just grab a couple of these and at least I'll have something to do with them.

So this is the weekend where I bring back the value of purchasing these PIs ahead of actually having an idea.

Stormy: Are you going to share what you create on GitHub?

Brian: I will have to. Haha. I might archive it right after though. "Looks cool, might delete later."

Well Stormy, thanks so much for the conversation. Folks, definitely check out what Amazon's doing in the open source world. Check out Stormy at a future event, a talk. And listeners, stay ready.