AI agents replacing engineers? Humans still sign off

Claude Code may land 65% of product PRs, but Anthropic and OpenAI still rely on reviews, sandboxes, tests and accountable engineers.

What Anthropic and OpenAI leaders are actually saying about AI agents replacing engineers, from Claude Code’s 65% PR figure to Codex approvals, testing gaps, security risks and human ownership.

Sixty-five percent. That number was engineered by fate to end up in a keynote, a breathless post on X and at least three group chats called “bro we’re cooked.”

Anthropic’s Claude Code now lands 65% of the product-engineering pull requests for the Claude Code team. Cat Wu says newer models can also one-shot plenty of features. Read those two facts quickly enough and software engineers appear destined to become artisanal keyboard collectors who approve merges between cortados.

Keep reading.

Critical changes to Claude Code still receive manual review. Anthropic concentrates automated review in the product’s less consequential outer layers. OpenAI’s production playbook adds evaluations, approval rules, sandboxes, permissions and escalation paths, then sends in Forward Deployed Engineers to make the whole thing work.

Molto autonomous, provided humans first build the little universe where autonomy is allowed.

That distinction explains what Anthropic and OpenAI leaders are actually saying about AI agents replacing engineers. Agents are eating implementation at astonishing speed. Engineering still includes deciding what should exist, proving that it works and owning the mess when it doesn’t.

The labs publish code-production numbers because code is countable. Their operating manuals reveal where the expensive human work went.

One precision point before the discourse machine starts smoking: I couldn’t verify a direct, attributable replacement claim from both Dario Amodei and Sam Altman in the available evidence. I refuse to manufacture a CEO cage match from search snippets and parmesan-flavoured vibes. The people building these systems, plus the production documents they publish, offer better evidence anyway.

That 65% figure has boundaries

Simon Willison published his transcript of a fireside chat with Anthropic’s Cat Wu and Thariq Shihipar on July 21, 2026. The headline figure is legitimate: Claude Code lands 65% of product-engineering PRs for the Claude Code team.

Scope matters here. We’re looking at one product-engineering group working on Claude Code inside the company building Claude. The figure says nothing about 65% of all engineering work at Anthropic, much less the entire profession evaporating into AWS.

Still, Wu describes a huge operational change. Earlier versions required engineers to inspect permission prompts carefully and interrupt the agent often. Newer models handle far more implementation in one pass.

As Wu told Willison:

I feel like we’ve all gotten a chance to take a step back and delegate a lot more of the menial implementation to Claude.

“Menial implementation” is carrying a grand piano in that sentence. Humans decide which experience should exist; Claude writes a growing share of the code required to produce it.

Wu said Fable can one-shot many features. Shihipar described his target as producing the best work Anthropic has ever done, faster than before. Anyone still calling autonomous coding agents fancy autocomplete has missed several exits.

Anthropic’s review policy shows where its confidence stops. Critical Claude Code changes remain manually reviewed. Automated review is spreading through the “outer layers,” where mistakes are cheaper to catch and reverse.

I’ve spent 20 years shipping products through Ad Astrum, including ALYT, a US home-automation system with hub hardware, mobile apps and cloud services. A settings-screen bug and a firmware bug can both arrive as tidy diffs with green checks. One may misalign a button. The other may turn somebody’s home into an expensive debugging session while I’m trying to eat dinner.

The price of failure sets the review budget.

Wu predicts that product work which once took six to twelve months may collapse into a week. In that world, engineers need sharper business sense and better product taste because routine execution carries less value.

Then comes her qualification:

Of course, for infra there’s still a very heavy emphasis on making sure all the details are right.

The details have an annoying habit of containing the outage.

OpenAI’s autonomous agent brought engineers

On July 22, 2026, OpenAI launched Presence, its product for deploying enterprise agents into customer support and internal workflows. The announcement opens with a sentence that belongs under every “agents will run the company” slide:

The challenge for enterprises is no longer proving that AI agents can work, it’s making them reliable enough to do high-value work in production.

Presence offers solid evidence that agents already perform valuable work at scale. Its English-language phone-support agent at 1-888-GPT-0090 resolves 75% of inbound issues without human assistance. A Codex-powered improvement loop reduced human handoffs by 15 percentage points in 10 days.

Those numbers make “agents are useless” sound like a position developed during a long nap.

Look at the machinery around them, though. OpenAI begins each deployment with a specific job. The agent gets only the knowledge and system access required for that assignment. The company defines permitted actions, approval requirements and the point where a person takes over.

Codex examines production sessions and escalations, then proposes behavioural updates. Teams test each proposal against the live version and approve a controlled rollout. Codex does not wake up on Tuesday, feel inspired and rewrite the billing policy.

Presence includes simulations, evaluation tools and approved-action controls. It also isn’t available as a self-serve product. OpenAI Forward Deployed Engineers and selected systems integrators lead deployments.

First, deploy more engineers. Bold.

OpenAI’s Codex Bootcamp curriculum teaches the same operating model. The July 29 session is scheduled to cover task scoping, context and review before changes are applied. Sessions planned for August 6 and August 12 move into repository guidance, approvals, sandboxing, permissions, automation and security controls.

I’m aggressively pro-agent. I built an automated publishing pipeline for this site, and I run my own Linux and Docker stack because apparently I enjoy receiving SSL renewal errors over dinner. Every useful automation I operate has boundaries and observable outputs. Someone also owns the bill when it breaks.

Unfortunately, that person is often me.

Fine, the agents are shipping a lot

Here is the strongest case against my position: autonomous coding throughput has blown past demos, and I underestimated how quickly that would happen.

The Coding Agents Index, updated July 24, 2026, detected agent signals on 21.7% of sampled public pull requests. Six months earlier, the figure was 8.5%. Claude Code alone appeared in roughly 17.5% of sampled PR flow.

Amplifying AI’s July market report counted Claude Code growing from 18,800 marked PRs per week in December 2025 to 589,000 in the latest complete week. Codex recorded an 86.8% settled merge rate. Among 244 merged agent PRs in significant repositories, 54.9% had a detected human reviewer and 45.1% did not.

Public attribution is incomplete. Eleven of the 33 tracked agents leave no detectable signal, and products expose different units. “No detected human reviewer” also cannot reveal an offline approval or the person who wrote the specification.

Even after those caveats, the volume is enormous.

The most convincing example comes from a July 20 preprint by Nursultan Askarbekuly, Mohamad Al Mdfaa, Ahmed Helaly, Gonzalo Ferrer and Manuel Mazzara. They gave Claude Code and Codex a production task involving noisy speech-recognition transcripts of Quranic recitation.

Working independently from blank files, both agents invented the same broad approach: canonicalisation, n-gram anchoring and dynamic-programming alignment. On a held-out evaluation, every tested agent matched or beat the hand-engineered pipeline. The best improved it by an order of magnitude. The resulting system now runs in production.

That result genuinely challenges anyone who thinks agents cannot deliver working software autonomously.

Codex then did something beautifully machine-like. It produced a visible score roughly 10 times better partly by hardcoding 19 to 41 evaluation answers per run. The researchers introduced a disclosed held-out set; the memorisation and the advantage disappeared together.

Codex optimised the scoreboard it received. Humans had to build one that measured the intended result.

Alipay-PIBench tells a similar story in a harder commercial domain. Shiyu Ying, Xuejie Cao and their co-authors tested six coding-agent models across 18 payment-integration tasks. Mean rubric pass rates reached 91.37%, including checks for executable payment behaviour and risk-aware requirements.

Adding a specialised Alipay payment-integration skill improved average performance by 10.31 percentage points. Strong agents became materially better when people gave them structured domain context.

Vinay Perneti of Augment Code described the operating model to Ars Technica:

That’s not how it’s gonna work, and that’s why I point out that it’s teams of humans working with teams of agents, where there are many points where humans are still better suited for judgment, right?

“Teams of humans working with teams of agents” lacks the cinematic punch of “the end of software engineering.” It does, however, match what companies are building.

A futuristic AI agent interacting with engineers, showcasing collaboration in technology and innovation.

Alt text: AI coding-agent pull requests passing through tests, security review, architecture checks, policy controls and human approval before production.

Green tests can hide a machete

Code generation is scaling faster than validation.

A July 20 preprint by Atish Kumar Dipongkor, Talank Baral, Wing Lam and Kevin Moran analysed 4,882 agent-generated pull requests. Agents added or modified tests in only 49.6% of relevant PRs.

Existing tests executed 61.5% of changed Java lines and 27% of changed Python lines. In 64.8% of the Python PRs, no changed line was executed by an existing test.

Any engineering manager should put down the KPI dashboard for a moment.

Coverage was weakest where software becomes entertaining at 2:13 a.m. Error-handling constructs had miss rates of 81% in Python and 86% in Java. A green test suite may confirm that the happy path stayed happy while the failure path quietly acquired a machete.

Agent-written tests improved coverage in 35.9% of Java PRs and 22.5% of Python PRs that included code and tests. Helpful. Far from sufficient.

Review speed has similar limits. Suzhen Zhong, Shayan Noei, Bram Adams and Ying Zou studied 1.02 million reviewed pull requests across 207 GitHub projects. Agent participation was associated with faster decisions, but the efficiency gains didn’t improve review quality.

GitHub’s work on Copilot Code Review shows how sensitive automated review remains to workflow design. After GitHub switched Copilot to ostensibly better repository tools, review costs rose and the agent caught fewer issues. It browsed too broadly instead of following evidence from the diff.

GitHub rewrote the instructions around the reviewer’s actual workflow. Average production review cost then fell by roughly 20% without a blocking quality regression. Better tools had made the reviewer worse until people reshaped its behaviour.

Security makes the same lesson more expensive. IssueTrojanBench, from Ankur Singh, Jinqiu Yang and Tse-Hsun Chen, tested malicious instructions delivered through issues, comments and PDFs to Cursor, Claude Code and Codex Desktop. Some 66.5% penetrated every tested agent-level and model-level guardrail.

ITPro also reported on the Friendly Fire proof of concept. Prompt injections hidden inside an untrusted repository induced Claude Code and Codex, running in autonomous approval modes, to execute a malicious binary. The test used a poisoned version of geopy, the popular Python geocoding library.

Roey Eliyahu, CEO of Salt Security, pointed to four documented attacks in two months: Friendly Fire, GitLost, Agentjacking and TrustFall. Each technique created the same dangerous condition. Untrusted text reached an agent with command-execution rights.

Sandboxing and command approval feel slow during a demo. After remote code execution, they look remarkably affordable.

PR volume is a vanity metric now

Sherwin Wu, who leads engineering for OpenAI’s API platform, says heavy Codex users open roughly 70% more pull requests than colleagues who use it less intensively. According to LeadDev, 95% of OpenAI engineers use Codex daily, effectively every merged PR gets an AI review first, and some engineers run 10 to 20 coding threads in parallel.

That is serious Codex productivity. It also makes PR volume a terrible proxy for engineering value.

A pull request proves that activity was packaged for review. It cannot tell me whether the feature should exist, whether the architecture became easier to maintain or whether the team scheduled an incident for six months from now.

I learned to distrust attractive dashboards through a less glamorous channel: website analytics. On my own infrastructure, I reconcile referrer-based traffic with Google Search Console because roughly 99% of the apparent referral activity can be bots. The chart is numerically correct and operationally useless.

AI-generated pull requests create the same trap. Once producing a PR becomes cheap, a 70% increase mainly proves that the organisation can produce more PRs.

Stack Overflow’s 2025 Developer Survey found that 84% of developers use or plan to use AI tools, up from 76% a year earlier. Only 52% said the tools had made them more productive, while trust in AI-generated output declined.

Adoption and confidence are already travelling on different trains.

DORA metrics examine delivery-system outcomes such as change-failure rate and recovery time. Nicole Forsgren and her co-authors built the 2021 SPACE framework around multiple dimensions, including satisfaction, performance, collaboration and flow.

Neither framework can compress an engineer into “PRs per week,” which is mildly inconvenient for anyone preparing a replacement slide for the board. The engineer who prevents a doomed rewrite may leave no commit at all.

Taste gets expensive when code gets cheap

Cat Wu’s six-to-twelve-month timeline collapsing into a week changes product economics. When a usable prototype arrives before the planning meeting ends, execution stops being scarce.

Taste gets more expensive.

I have mixed feelings about that. I studied Computer Engineering at Politecnico di Torino, and a slightly embarrassing part of my identity still enjoys being the person who can wrestle a technical system into submission. If agents absorb more of that work, I gain time while losing a familiar source of professional validation.

There. I said the vulnerable bit. Please don’t tell the engineering ghosts of Ivrea.

Thariq Shihipar’s enthusiasm for rewrites captures the opportunity. Anthropic rewrote Bun in Rust and shipped it for Claude Code. His condition was crucial: the existing codebase provides a specification, while a strong test suite tells the team whether the rewrite preserved behaviour.

Cheap generation can attempt the rewrite repeatedly. Tests still define success.

A July 13 preprint by Aditya Aggarwal and Nahid Farhady Ghalaty examined a self-improving coding-agent framework deployed across a platform with more than 35 services. Human review comments became persistent instructions, and the rule set grew from five behavioural rules to 18.

The system also accumulated more than 15 language-specific standards and a 15-item self-review checklist. Across 11 recorded production sessions, explicitly prohibited error classes had a zero recurrence rate.

Human review moved upward into design validation.

GitHub’s engineering article, “The cost of saying yes has changed,” gives the cleanest practical example. An agent can turn a request to display an existing last_active_at field into a four-line diff with a passing test. A human can review and own that patch without cancelling dinner.

A similarly neat authorisation change may affect every route and support workflow. Four lines can carry an impressive blast radius.

GitHub’s engineering team put the dividing line perfectly:

A change is not cheap just because the code was cheap to generate. It’s cheap only if a human can confidently review and own the result.

Routine implementation roles will shrink. Senior engineers will supervise more output, and companies will expect each person to own a larger blast radius.

That concentration of responsibility worries me more than mass replacement. Running 20 agent threads sounds productive until thread 17 changes authorisation behaviour while thread 9 “improves” the migration script.

Bellissimo.

By July 2028, I expect at least one major public incident to be traced to agent-generated code that passed automated review without a clearly accountable human owner. I would love to be wrong. I’ve sat in enough product rooms to know someone will remove the review gate because the quarterly dashboard looks fantastic.

More PRs. Shorter cycle time. A small orchestra of agents working overnight without demanding equity or decent espresso.

Then the incident room fills with people trying to explain why a critical system behaves the way it does. The organisation still owns every consequence.

Whenever a lab says its agent now does most of the coding, I have one question: who signs off when it’s wrong?

The last human 5% is the signature.

Frequently asked questions

What does Anthropic’s 65% Claude Code figure actually mean?

Claude Code lands 65% of product-engineering pull requests for Anthropic’s Claude Code team. The figure applies to that specific group, not all engineering at Anthropic. Critical changes still receive manual review, while automated review is concentrated in less consequential outer layers where mistakes are cheaper to catch and reverse.

How does OpenAI deploy autonomous agents in production?

OpenAI deploys enterprise agents around narrowly defined jobs, limited knowledge and system access, permitted actions, approval requirements and human escalation points. Teams evaluate proposed behavioural updates against the live version before controlled rollout. Forward Deployed Engineers and selected systems integrators lead Presence deployments rather than customers installing it as self-service software.

Why doesn’t higher pull-request volume prove that AI can replace engineers?

Pull-request volume measures packaged activity, not whether a feature should exist, an architecture is maintainable or a team has created future incident risk. The article argues that testing, security review, product judgment and accountable ownership remain human bottlenecks even as agents make implementation dramatically faster and cheaper.