How to Make Agents Durable for Concurrent Systems

How Can Agents Be the Future if They’re not Reliable?

Some reports suggest that as many as 25% of enterprises have adopted agentic workflows. Isn’t this the era of AI agents? Why isn’t the number higher? Aside from predictable enterprise-level concerns (49% of enterprises are reportedly holding off due to security concerns, for instance), agents haven’t proven to be reliable workhorses at scale, particularly for long-horizon tasks. Not yet, anyway.

Agents still can’t “remember” their learnings from previous successful runs, still hallucinate when their context window becomes full, and still have a number of other issues. Wasn’t the perfect agentic future supposed to feature technical teams commanding fleets of infallible, well-trained agents to do their bidding?

Open-source creator Mike Hostetler believes such a future is possible if systems combine strong guardrails with fault-tolerant infrastructure designed for large-scale deployments. Here, he explains why he built his project, Jido, not using Python, the de facto programming language for AI, but using Elixir, which was inherently designed to support maintainability and massive concurrency.

Resolving Nondeterministic LLMs and Deterministic Algorithms

Hostetler, a veteran developer and founder, got his start working on the jQuery ecosystem, from which he built a career, a startup, and a lifelong curiosity about technology. He explains that his introduction to Elixir came from a tour of duty working in distributed systems, which inspired him to think through the challenge of being able to run multiple agents with real-world reliability. “I envisioned a future of ‘10,000 agents per human.’”

The creator notes that his original vision was running 10,000 agents on a single Raspberry Pi. In other words, in a resource-constrained environment with no direct access to LLMs. Models were getting better, but capital expenditure from foundation labs continued to skyrocket.

Hostetler suggests his goal was to make the best of both worlds: Capitalize on the improving performance of state-of-the-art models, combined with performant infrastructure that wouldn’t break the bank. “The intersection of these two ideas is: Agents needed to be easy to build, easy to deploy, easy to orchestrate and coordinate. And then, 20% of your agents could use an LLM as the thinking step. But it needed to be just as easy to build 80% of your agents on what I call classical AI algorithms.”

The builder clarifies that ‘classic algorithms’ also include logic that predates LLMs: “Everything from a finite state machine to a behavior tree to hierarchical task networks. Video game AI as well as LLM AI. And those could work in concert and then be easily and inexpensively deployable for a user, which gets into the real pragmatic piece.”

Hostetler clarifies that despite Jido’s explicit “AI optional” clause, his approach to AI is less skeptical and more pragmatic. “I love the nondeterminism of LLMs. I love pushing those boundaries. Just by happenstance, having put my own open-source project out there, I got to know Geoff Huntley and tried the Ralph Wiggum loop before it was publicly available. Loved it, and had Jido running loops.”

“But I quickly observed that [with successive loops], you end up compounding quality issues and you run into these challenges.” The builder’s observations led him to start experimenting with combinations of agentic loops and classical, deterministic AI algorithms, based on a self-taught course of study.

“I've used my computer science degree more in the last two years than I have in my entire career, which is just so much fun. So I’m not ‘anti-AI’ at all. What I want to put forward is something that lets you use both modes of tooling in the same context, instead of having a ‘developer mentality’ that requires a hard switch between ‘Here’s an agent that has an LLM as its brain’ versus an agent that has a finite state machine as its brain, and you have to build a bridge between the two. Jido makes both into first-class citizens. I knew as an engineer, as an architect, I would want both.”

How Can Agents Master Long-Horizon Jobs?

Hostetler admits to reading, and speculating about, the news headlines about agents seemingly getting closer to the holy grail of succeeding on complicated, long-horizon tasks in a single, zero-shot run. “Yes, I read the headlines, and I see the METR reports. I would say I have hypotheses about the topic, but I acknowledge the limits of my hypotheses.”

“We see new model drops like the [recalled-and-re-released] Fable 5, which can reportedly do a long-running task for eight hours, uninterrupted.” The creator notes that it can be hard to separate reality from hype, but notes that foundation labs may have different incentives than the rest of us.

“I would think there is a lot of pressure to embed the capability behind the walls of the model; to make Fable better via academic RL, or to have a lot of infrastructure running between the API call and the model itself. You see them pulling in things like their web API tools, and doing everything to make Anthropic a walled garden for any corporate customer.”

Hostetler contrasts the lofty ambitions of foundation labs against his day-to-day work, which requires him to service ERP systems maintained by tech leaders across the country who need reliable performance, day in and day out.

The creator suggests starting from a strong harness, but also looking into reliability. “A really concrete step is to start with a very high-quality model harness if I'm going to build my own agent. Yes, admittedly, Elixir’s philosophy is ‘let it crash,’ but you don't really want that to happen in practice. Your focus should be: How do you recover from the crash? How do you deliver high-quality software from that?”

“Elixir’s philosophy is ‘let it crash,’ but you don't really want that to happen in practice. Your focus should be: How do you recover from the crash? How do you deliver high-quality software from that?” -Mike Hostetler, Creator/Jido

Squaring the Circle: Autonomous Agents vs. Reliable Uptime

The builder recognizes the gap between LLM-powered agents that often fall down for a variety of reasons and Elixir-based systems like Discord and Pinterest that are expected to provide continuous uptime for millions of users. “Without getting too technical, this really was a key design decision of Jido itself.”

Hostetler explains that the project was designed to accommodate both builders of independent, solo agents as well as those building for coordinated, multi-agent use cases. “And also, there’s all the infrastructure concerns for when you go to deploy: Where does it live? How is it persisted? It's persisted with what I consider to be two schools of thought.”

“The first [persistence story] is that I persist my BEAM system back to a durable store, like a Postgres database which is centralized. No issue there. And I would say that more than half our users use this. The other key story is what I call the fully distributed, fault-tolerant approach for scale that you get into with the really big deployments, such as WhatsApp or Facebook Messenger. Multi-region, multi-clustered...the big stuff.”

“Jido was geared for [large-scale deployments]. We could get into the specifics of how Jido was created to live within the BEAM itself, but it is possible. And there are nuances here, too. People come to the project and they sometimes feel like it's overbuilt. Case in point, when you send a message into a Jido agent, we use a dedicated message envelope called a Jido signal, which, when you're in a single-node use case is really and truly overkill. And yet it's the standard built on the CloudEvents spec (which is independent of, and exists outside of Elixir).”

“We do this because you can't have it both ways: You can't have your typical simple JSON payload alongside a robust, enterprise-grade, transport-agnostic CloudEvents envelope and do the same thing. You've got to pick a lane. And so, Jido picked the ‘robust and scalable’ lane. I do work with some teams that are now asking me about the multi-cluster use case. I’m working toward a demo of a multi-region cluster independent of a durable store with resilient agents.”

“To clarify, everything I just discussed was outside the agentic loop. Within the agentic loop of determinism versus nondeterminism, there are guardrails within which Jido has to operate. It can't just go do everything because it was meant for the Raspberry Pi use case. Because Elixir and BEAM can natively cluster, you could easily have a deploy to support a use case for something like, let’s say, Internet of Things (IoT).”

“For example, you’re going to have a central cluster that then works with Jido agents that are deployed in an IoT setting that talk back to the central cluster, and you'd have two agents communicating with one another. You could have that modality in a larger deployment. As an example, one of the big successes in the Elixir world is the robot control project Beam Bots, which adopted Jido natively, so we’re seeing use cases for non-clustered low-level IoT all the way up to your typical Claude Code agent harness.”

How Startup Founders Should Think About Agentic Products

When asked how he would advise startup founders thinking of launching agentic products, Hostetler offers a handful of suggestions. “For your first level of planning, I would say: Focus on the default. How you define what an ‘agent’ is could be colored by your Claude Code glasses! Make sure that as you're defining an agent in terms of what you bring into your infrastructure, and that you have a tighter, concrete definition of what that is.”

“Second, the other mistake that I've seen (and have made personally) is that everybody wants to let the LLM ‘take the wheel.’ At the point when you're designing infrastructure to build a company on, you have to draw clear boundaries between nondeterminism and determinism, and create gates to bring it back and forth. And this is outside of the agentic loop.”

The creator suggests that customer expectations for agentic projects have grown over time. “After a few years, we’ve seen the patterns that ‘work,’ whether it's natural language or dynamic UI, we know the patterns to create user value and what users expect. The innovation has shifted from, ‘We can do just about anything,’ to ‘The user expectations are pretty much there, so how do you deliver on that?’”

Hostetler describes an interesting project a friend built in Elixir that effectively replicated Claude Code’s functionality, but for posting on LinkedIn, using a dedicated agent with a specific ID. “Persistence wasn't dynamically allocated, but the agent would persist, and then it would hibernate and rehydrate when you wanted to edit a piece of content.”

The project let users edit posts normally via the LinkedIn GUI, but also supported giving instructions to a coordinator agent. “The key is, he made that persistent throughout the entire experience. So, no matter which modality you were interacting with, there was a kind of predefined team structure for how the agents would interact with your content. And it just struck me as this really innovative mental leap, that any agentic products I see aren't thinking big enough.”

The creator suggests that the main challenge he’d offer to startup founders and product owners is to think bigger. “I don't see a lot of product owners thinking about [larger ideas] because they're really stuck in the mindset of, ‘Claude Code is my agent. It can do everything. I have to always use Claude Code.’ I say: ‘No, you really need to break out of that mental model.’”

“It just struck me...that any agentic products I see aren't thinking big enough. I don't see a lot of product owners thinking about [larger ideas] because they're really stuck in the mindset of, ‘Claude Code is my agent. I say: ‘No, you really need to break out of that mental model.’”

How to Think About Enterprise Adoption

When asked how he would advise ambitious startup founders looking to sell AI products to enterprises, Hostetler draws upon his own day-to-day experience. “I would say that we're early. I would acknowledge the timeline displacement, Ray Kurzweil’s riff on the William Gibson quote about the future being here, just not evenly distributed.”

“What's happening in enterprises is that there are engineering teams who are using the latest tools, and they're on that curve somewhere. But the operations teams are just starting.” The creator notes that he works with operations teams that are just starting to encounter the same challenges that engineering teams faced 18 months ago: Worrying whether AI will replace them, realizing AI likely won’t replace them, then trying to figure out how to fit into this brave new world.

“We're early in that phase [where people are trying to find] pragmatic use cases. The issue isn't so much the tools themselves. What's interesting to me is the layer above them, which I wrote about in my blog on orchestration. To me, orchestration is the cornerstone, because there's nothing actually orchestrating the tools.”

“I think that corporations are set up in silos. Sales isn't talking to marketing, who isn't talking to ops. Ops isn't talking to product. So, the pragmatic things that we need are places to deploy AI experiments, because the thing we've emphasized internally isn't so much AI as an end state, but rather, AI as iteration.”

The creator recounts a story of a team he works with using Claude Code to build a dashboard. In a completely unintended consequence, Claude Code used the Google Workspace connect to deploy the dashboard to Google Sheets via Apps Script to create a public URL via, of all things, a Google Sheets backdoor. “I burst out laughing. There's an entire product right there, in that space between a non-technical user and a model’s instincts to propose unexpected solutions.”

Hostetler also points out a strong business opportunity in governance. “During my tour of duty in corporate, I spent two years as a VP of enterprise technology. I managed all the IT and compliance stuff for a publicly-traded company.” The creator notes that “AI governance” could easily go beyond basic compliance frameworks like SOC 2.

“Every LLM call in your organization will eventually be proxied and logged. But nobody's doing this at the moment. Think about just capturing it as one problem. Analyzing and fingerprinting.” The creator predicts that sooner than we think, organizations may need to be “AI compliant” with whatever new version of the SOC2 standard that arises.

“Another [enterprise use case] I’d look at is insurance. Cybersecurity insurance with AI, mark my words, in the next two years, that will be a ‘thing.’ And this won’t just be your CrowdStrike-style, on-device stuff. It's all the things that get exfiltrated from what you build. Nobody's talking about that. The strategy there is very immature. Admittedly, this isn’t a very sexy space, but I think there's a lot of money to be made there.”

The Future of Elixir and Durable Systems

The creator is realistic about how his project and the greater Elixir ecosystem figure into the future of software development. “A non-Elixir shop is going to have a hard time adopting Elixir. So I can see Jido becoming an agentic sidecar that eventually makes it easy for TypeScript devs to spin up an agent with Elixir-level durability.”

“This might be for a market segment that could be ‘one step below’ the Temporals of the world. To clarify, I’ve used Temporal before, and it’s fantastic, though expensive. But thinking pragmatically, there are a lot more TypeScript devs in the world. Python will be its own thing. TypeScript devs are going to need an SDK to easily call Jido agents, and then Jido agents will operate async, and it'll just be self-contained with a dashboard to spin it up.”

The creator acknowledges that popularly-used languages and frameworks like TypeScript and Node.js are excellent for what they are, and that they solve specific problems. “But you need to be able to spin things up inexpensively that companies built on TypeScript can solve either by running in Docker via their infrastructure, or run out of a cloud where they can spin up platforms of agents, with one agent per customer (or dozens of agents per customer).”

“And to be able to do that practically is just going to be overwhelmingly expensive, even with Cloud costs. And you need to be able to stitch it all together, potentially using non-deterministic agents with deterministic agents to deliver a strong product experience. It’s a very early picture, but that's where I see the market now.”