What if the AI Harness Was Your Computer?

Giving AI Data Ownership Back to Users

Researchers have warned for years: Frontier models are potentially running out of data to train on. However, the apparent lack of data hasn’t stopped AI labs from endlessly launching newer, shinier models with endlessly improved capabilities, which, in turn, makes switching costs painful for users who have their data locked up in a previous-gen version of a competitor’s model.

After significant experience working in general software development and then in localized inference, veteran developer Terence Pae began working on turning his own Mac into a personal AI harness, freeing him from vendor lock-in from increasingly commoditized models and giving him significantly more control and ownership over his data and permissions. Now his open-source project Osaurus has passed 140,000 downloads and 6,000 GitHub stars, all with zero paid acquisition.

What if your MacBook was your AI harness? Image courtesy Osaurus

Why Vendor Lock-In Has Made AI Development Painful

Pae explains that the project came from his own concerns as a developer who was heavily dependent on third-party harnesses. “I'm such a heavy user [of AI], and it’s easy to not really think about all the data I might be sending out and the potential implications there.”

“As AI evolves and grows more powerful, it’s constantly taking in all this information from users to make models better. But at the same time, as an individual user, I’m wondering: Where is my privacy? Where is my data going and where is it being stored?”

The founder also notes that vendor lock-in benefits the frontier labs while making switching between models painful. “These AI companies are building moats, and they’re not using their data to do that. They’re actually using your data. The thesis with Osaurus is that as a user, you should be owning everything that is related to your data, your code, your context, your tools, and everything else. All you really need is the inference, whether that’s running locally on your hardware or in the Cloud, so you’re not dependent on a specific provider.”

Lessons Learned from Decentralized Infrastructure

The founder notes that deciding to go with a building-in-public approach to open source was the key to growing his project, and that his work in decentralized infrastructure led him to focus on local-first AI development. “There are many ways to build AI harnesses. For example, you can write code that runs on the Cloud, which depends on having a certain infrastructure. And you need a database to host your user's data. But Osaurus is, itself, a client and a server that runs locally, and spins up a SQLite database locally. Everything is self-contained on your device.”

Pae draws on his experience with peer-to-peer systems, where every node acts as both client and server. "Each node can connect with the others and create a network where no individual node has to rely on a centralized server. And if an AI is meant to be owned by the user and designed for the user, it also needs to act like a node that doesn't depend on a centralized service. That software should still work even if the company behind it gets hit by a meteor."

Pae's design approach assumes the future of AI won't always have a human in the loop. "Right now, we're relying heavily on a chat interface with agents and having a human in the loop constantly interacting with them. But I believe the next evolution is going to be more about agent-to-agent communication. For agents to be truly autonomous, they'll have to rely on other agents."

The founder’s experience working with cryptography informed his design. “Basically every agent will have an identity with a public key and a private key, so that agents themselves can verify each other. Being able to use a signature to verify each other has many benefits. But the key benefit is just making sure that this agent is talking to the right agent.”

“If my agent is reaching out to somebody else's agent, it needs to be able to verify that that particular agent is owned by that person. Designing in those specific protocols helps us to get to where we need to be in terms of agent-to-agent collaboration. Those are the basic building blocks I'm setting in place, but the future vision is about giving agents the autonomy, the guardrails, and the ability to act on your behalf without compromising or being compromised by other systems.”

I believe the next evolution is going to be more about agent-to-agent communication. For agents to be truly autonomous, they would have to rely on other agents.” - Terence Pae, Founder/Osaurus

Is On-Device Harness + Compute a Viable Alternative?

Pae acknowledges the growing challenges of cost management in AI operations, including organizations burning through massive token budgets utilizing third-party vendors. “I think [on-device] could absolutely be an alternative. In my opinion, the current model of scaling and building all these data centers to power all the demand in the world doesn’t seem very feasible.”

“I think the rate of agentic use is only going to go up from here. As a software engineer, my AI costs used to be about $15 monthly. Now, between cloud subscriptions and API usage, I'm paying $3,000 to $5,000 a month. That's just me as an individual engineer, and it's exactly the cost curve that makes local inference worth taking seriously.”

The founder suggests that while token economics seems increasingly likely to affect SaaS-style pricing, AI computing at the edge may also be competitive soon. “Local AI has improved almost exponentially in terms of intelligence per watt. For example, the recent Gemma 4 models run under 24GB of RAM and match what frontier models could do only a generation ago. That capability now fits on a laptop.”

Pae also points out that not every use case demands a trillion-parameter, general-purpose model. “You may just need a good-enough, smart-enough model to help automate things. And it's the harness's job to really bring out the potential of the model to not only increase capabilities, but also reduce some of the dependencies as well. In the future, we’re all going to be less dependent on the Cloud and more dependent on local infrastructure, in my opinion.”

Getting On-Device AI into Enterprise: Models, Tools, Education

While the promise of having full control over your data for daily AI tasks, without interruptions from cloud service outages, seems like an easy sell to individual developers, the founder concedes that risk-averse enterprises may need more nudging to actually adopt local-first AI.

“I think there are three issues preventing enterprise adoption. First, the local models themselves: There’s constant innovation happening here. The models may not be fully ready today, but maybe soon, possibly in the next year. Second, there don’t seem to be enough tools and harnesses that can work with local models. Major frontier labs like Anthropic and OpenAI not only have models, but also have tooling and infrastructure to be efficient and creative with their offerings.”

“Third is a lack of education: People may not know that these capabilities are good enough to use today. We speak with a lot of enterprises, including family offices and healthcare clinics that deal with sensitive data. The most common theme is there’s just no trust in the Cloud.”

The founder shares that cloud providers and their 30-day data retention policies still seem to rub enterprise customers the wrong way in terms of risk exposure to court subpoenas and the like. “As a company, that’s a huge risk. As an enterprise, you lean towards trying to have no risk at all.”

In contrast, having a private AI instance means sensitive work can stay on local infrastructure and never touch the Cloud. Osaurus supports both local and cloud inference, but the user decides what leaves the device. “Unlocking private AI with a full-featured agentic harness is going to really unlock the productivity that these enterprises are looking for: Having a private AI infrastructure with a frontier-level model, with a frontier-level harness that they can actually use internally.”

The founder also asserts that not all enterprise customers are stodgy, middle-aged, middle managers who are terrified of using AI at all due to risk. “The people we talk to definitely want to use AI, and want to simplify their workflows. They're really curious about it. A lot of the people that we chatted with use AI personally, but they don't use it for work. So they know AI’s capabilities.”

Pae recalls starting initial conversations with enterprise customers with his own preconceived notions: Would enterprise users seek very specific things, similar to familiar tools like Microsoft Excel? “Actually, most of the feedback we got was that [enterprise customers] want a ‘general AI,’ a general harness that is able to do what ChatGPT does. The people we talk to are looking for something more broad in terms of applications.”

Three issues are preventing enterprise adoption: the local models themselves, not enough tools and harnesses that work with local models, and a lack of education."

Building AI Products for Startup Founders

Pae is direct in his advice to startup founders looking to start companies around AI-adjacent products: Don’t build on top of someone else’s walled garden. “I would say treat inference almost as a building block: You can have a query, and then it returns a specific output in a function. Otherwise, you don’t want to build software in a way that locks you into a single provider.”

“You don’t want to build products in a way that’s super-dependent on specific functionality [from one vendor]. You want to build [components] in a way that is easily replaceable. Inference should be treated that way as well. Building an agent? You should design it in a way that you can easily replace your inference provider, and in my opinion, also have the ability to swap in local models as that may make things a bit more defensible.”

“Vendors like Anthropic and OpenAI may have some really cool features, but not every AI vendor will have the capabilities to build something similar. If you offer functions that can work with many different providers, you can also build in a way that targets specific audiences, such as the rapidly-growing open-weight market in China.”

“Being model-agnostic has worked out for us because there are different communities using these different open-weight models. We’re not really bothered if Anthropic suddenly launches a feature similar to ours, because they aren’t likely to also support MiniMax or other models.”

“In a larger sense, I’ve been thinking that software isn’t exactly ‘defensible’ anymore, since you can potentially create anything with enough attention and time. If we treat software like a true commodity, I think that's not going to be scalable for the long term. What is truly defensible is the community that you build around your software and also the transparency you can provide. The community that forms around you gives you that ability to push forward.”