1. DevToolsDigest
  2. issue #397

DevToolsDigest: Issue #397

Ankit Jain is a founder and CEO of Aviator, a developer productivity platform used by modern engineering teams to ship AI-generated code at scale, without the review bottlenecks, broken builds, or brittle deployments. He also leads The Hangar, a community of senior engineers and engineering leaders focused on developer experience, and Xoogler, the ex-Google alumni network. Connect with Ankit →

Subscribe to the DevToolsDigest

All you need is 3 minutes a week to stay ahead of the devtools space. Get the most relevant industry updates, insightful discussions, and top-notch resources delivered straight to your inbox.

    5 MIN

    When AI Can Code, What Is Left for Programmers to Do?

    This article describes human-AI collaboration at specification and code repair stages, and lays out the skills that humans need to accomplish those tasks in the current generation of AI.

    When an agent can hand you a 3,000-line PR, reading every line stops working. Shah and Oram argue the reviewers should focus on validating specs, schemas, and automated drift audits instead of line-by-line code.

    Hide Slop PR Descriptions Behind <details><summary>

    Today’s coding agents are great at writing PR descriptions. The problem is that they are too good. An agent-written PR description is accurate and comprehensive. It’s not necessarily slop, but it reads like slop!

    Simon Willison's small fix for agent-written PR descriptions: put a short human-written summary on top and collapse the full agent text in a details block labeled with the model that wrote it. The reviewer gets the author's judgment first and the detail only if they want it.

    No AI Slop: A Manifesto

    The goal of these resolutions is to embrace our important human role in harnessing AI safely, responsibly, in a way that creates value for everyone, and that will be relevant for years to come.

    Written by speakers and attendees of the AIE Code Summit (full disclosure: I was one of them, and a signer). It starts from a simple asymmetry: a thousand-line PR now costs almost nothing to produce but just as much to review, so its principles put responsibility back on the sender.

    Can Review Agents Take Humans Out of Code Review?

    Human review has well-documented limits: larger changes get less careful attention, and human oversight means there are still risks of bugs reaching production. But measuring an AI reviewer turns out to be tricky.

    Lizzie Matusov breaks down a benchmark scored against real, confirmed bugs: the best AI reviewer caught 72%, humans about half, and the two together 82%. They catch different bugs, so the takeaway is AI plus human rather than AI instead of human, and keep PRs small, because bigger diffs mean more missed bugs.

    Predicting the Token Cost of Agentic Code Review

    Organizations increasingly delegate code review to multi-agent large language model pipelines billed per token. Low code quality raises human maintenance effort, but its effect on agent review cost has not been measured.

    Lo Gullstrand Heander and co-authors found that each point of CodeHealth cut the token cost of agentic review by 3.5–6.1%, because agents keep re-reviewing messy code. Technical debt now shows up on the inference bill. We recently recorded a podcast with Lo on this; it's coming out soon.

    Attempting to Reverse Engineer Code Review

    “AI slop” is the new “I didn’t like it.” Everyone keeps complaining about code review. Code review is subjective work presented with the illusion of objectivity; everything unsaid can play out in it.

    Cate Huston turned 20 months of her team's review comments into automated checks and ran them on recent PRs. Most of the issues they caught had never been raised in review, by humans or bots.

    Cognitive Tours

    Cognitive Tours is a tool designed to mitigate and repair cognitive and intent debt. Cognitive Tours is a web-based prototype for creating and following curated paths connecting user needs, design decisions, code, tests, and open questions.

    Margaret-Anne Storey, who spoke at QCon this week, names two costs of AI-assisted development: cognitive debt (a team losing its shared understanding of the system) and intent debt (losing the record of why the code exists). Her prototype, "Cognitive Tours", makes that understanding something a team writes down on purpose.

    How the Claude Code Team Thinks About Verification

    If verification is critical to producing quality code at a high velocity, what types of tests should you use and how should you apply them across your pipeline? This article shares the process used by the Claude Code team.

    With Claude writing ~90% of Claude Code, the team's job has shifted from reading diffs to defining what "done" looks like and checking the evidence. This piece covers the tests and gates that make that work, including the rule that a change can never edit its own checks.