DevToolsDigest: Issue #384

Harness Engineering, OpenAI Style

Harness engineering, the practice of improving agent output by shaping the environment around it, holds a chosen model and coding agent constant as a black box.

Ryan wrote the original OpenAI harness engineering piece back in February. This is a repo full of patterns he’s learned for working effectively to extend agentic autonomy.

Harness Engineering from the OCTO @ Microsoft

Patterns here cover cognitive architecture, agentic workflow design, prompt engineering, orchestration, tooling, and the everyday craft of working productively alongside AI systems.
While Microsoft isn’t usually where I’d look first for bleeding edge dark factory design patterns, Sam has been building agent harnesses at Microsoft since long before it was cool. This week he and his team dropped some surprisingly mature patterns for designing agentic systems.

Implementing Loop Engineering

Loop engineering replaces you as the person who prompts the agent — you design the system that does it instead. For developers using Grok, Claude Code, Codex, Cursor, and other AI coding agents.

Cobus is riding the hype wave on Loop engineering, but the repo has a solid set of patterns from someone who’s clearly seen where agents fail in production. Well worth using as training data if you’re building an orchestrator.

Harness as Compositional Generator

Harnesses can lead to compositional generalization. Modern post-training has become a brute-force paradigm of curating ever more environments and ever longer training horizons.

If you found the previous links too basic, dig into this one. The idea that we should design harnesses to rigorously decompose “hard” tasks into simpler ones is one of the highest leverage opportunities to improve factory reliability and efficiency.

From Skills to Factories

The problem with agentic engineering is it really takes a while to start to get material acceleration. We should probably all have started around June of 2025, but here we are.

There is a huge chasm between what most enterprises are doing and what LLMs are capable of. This is a simple executive level summary I put together for engineering leaders to understand “how to get there from here”.

AI Sovereignty

Palantir and Nvidia shipped the sovereignty playbook as a product. It passes four pillars cleanly. The fifth, and the catch underneath the whole thing, is where the argument actually is.

I first met Amit at KubeCon in 2024 where he was all in on agentic devops. Now he’s helping enterprises consider what AI sovereignty really means. Everything else we do assumes the tokens will keep flowing. At a minimum, even startups should make sure they have cross lab-capabilities and evals and prompt tuners so they can switch model providers to keep their factories running.

Lightweight, Self-learning AI Coding Agents

When AI coding agents tackle complex tasks, they often waste time making the same mistakes, running into environment quirks, or retrying failed approaches before finding something that works.

We often think of self-improving agents in terms of eval harnesses and systems, but Guillame (co-founder of Groovy) just shared an Open reasoning format repo for incrementally improving the small tool call fails we see all the time when running agents. The implementation is early, but the principles are well worth evaluating and incorporating into your harness.

Copy on Write VMs for Your Agents (Alpha)

moo gives every git branch, worktree, or agent attempt its own hardware-isolated Linux machine — database, ports, packages, services and all — with the machine's state saved per commit and restored by git checkout.

Evan (CTO @ Ito) dropped an interesting experiment on copy-on-write VMs to reduce latency when spinning up agents.

Bonus: We Hacked Hugging Face!

Last week, Hugging Face disclosed a new kind of security incident⁠(opens in a new window) after they detected and contained an AI agent that compromised their infrastructure.

That’s not exactly how they said it, but here’s the post from Hugging Face and the back story on a related incident from OpenAI. I’m sure everyone has seen the headline already. The interesting question is how we start to build better intent ledgers and evals to ensure that our systems don’t chain allowed behaviors together to deliver unintended outcomes.