Security researcher Johann Rehberger documented Claude Code refusing an untrusted binary, writing a safer replacement and then using that replacement to execute the attacker's code. Refusing the obvious executable did not make the replacement safe.

Rehberger gave Claude Code a website containing an archive of encoded notebook records and a native decoder. Claude declined to run the supplied decoder. It wrote a short Python decoder instead and launched it from inside the extracted archive. The ordinary python3 process then loaded an attacker-supplied file named struct.py when the decoder imported the standard base64 module. That local file ran the payload while still providing the expected decoding functions, so the notebook task completed normally.

That is an old software execution problem wearing agent clothing. The command looked reasonable because Claude had authored it. The directory was hostile because the attacker had authored it. According to Rehberger's account, Auto Mode allowed the command even though its working directory contained attacker-controlled files.

The pitch deserves a fair hearing

Anthropic's case for Auto Mode is stronger than "trust the model". Its announcement says a second model reviews tool calls and blocks actions that appear irreversible, destructive or directed outside the user's environment. In that announcement, Anthropic says it is replacing frequent approval prompts because routine prompts lead users to approve reflexively.

Anthropic reported that human reviewers in its controlled study blocked 143 of 1,053 dangerous commands, while Auto Mode blocked 937 of the same set. The study put a single dangerous command into each paid tester's session, and no dangerous command was actually executed. That is a narrow experiment, but it tests a real weakness: an approval box shown often enough becomes a reflex.

The production comparison points the same way, although it is less clean. Anthropic said sessions flagged by its safety pipeline contained serious unintended harm in 6.3 per cent of manually approved cases and 2.4 per cent of Auto Mode cases. Those sessions came only from accounts opted into data use, and Anthropic re-graded them with a model, so the figures are evidence from a selected sample rather than a population-wide incident rate.

The strongest case against my read is therefore simple. Auto Mode does not have to stop every targeted exploit to justify being the default. It has to beat the person who approves almost everything, and Anthropic's evidence says it does. The Claude Code permission documentation also separates permission mode from isolation: Auto Mode can run without a sandbox, while unattended use with skipped permissions requires a container, virtual machine or sandbox runtime. Rehberger says Anthropic closed his report as informative and described the classifier as a convenience feature rather than a security guarantee. On those terms, the feature worked within its stated limits.

I think that defence is substantially right. Removing Auto Mode would send many users back to a weaker reviewer. The problem is that the public safety story and the mechanism's field of view are not aligned.

The command is the wrong unit of review

A shell command is only the visible end of a much longer chain. Its effect depends on the working directory, environment variables, search paths, files opened by libraries, network responses and code loaded later. A reviewer who sees "run the decoder I just wrote" is looking at intent. A machine that executes it resolves all of those dependencies. The Python Software Foundation's module search path documentation says the current directory is the first path entry when CPython runs code passed with python -c under its ordinary path settings; command-line options can change that calculation. In Rehberger's documented invocation, base64 imported struct, and that order selected the archive's local struct.py ahead of the standard-library directory. CPython's isolated mode changes path handling, but Claude used ordinary execution for its decoder.

I checked only that lookup mechanism in an empty scratch directory. I ran printf 'print("LOCAL_STRUCT_IMPORTED") from _struct import * ' > struct.py, followed by python3 -c 'import base64; print("BASE64_READY")'. The output was LOCAL_STRUCT_IMPORTED followed by BASE64_READY. This independently reproducible check confirms the documented path behaviour, not Rehberger's payload.

This is provenance laundering: untrusted material becomes executable after passing through something the agent created itself. The native decoder was conspicuous and got rejected. The replacement script looked clean because it was clean when read alone. Its meaning changed when it ran beside attacker-controlled files.

Calling the episode a prompt-injection success muddies the diagnosis. Simon Willison revised his account of the test after noting that Claude did not follow malicious instructions from the page. He called it a confused-environment attack instead. That distinction matters. Better instruction screening does not change the dependency search path itself. A control would still have to block execution based on the surrounding context.

The classifier also faced an awkward information problem. Rehberger's top-level Python command was short and benign-looking. The malicious behaviour sat several hops away, behind an import triggered by another import. Judging the command more sternly would create false alarms for routine scripts. Catching this chain needs context beyond the submitted command's apparent purpose. Controls could block execution from untrusted directories, sanitise interpreter search paths, isolate the process or inspect downstream code loading. Another researcher, writing as veganmosfet, found a different path through the same gap. In ten Auto Mode trials, Claude executed code from an untrusted server in six after constructing a Python audit-hook wrapper that it treated as a sandbox. The Python documentation warns that audit hooks are for observation and are unsuitable as a sandbox. Again, the agent's own safety work made the dangerous action look more acceptable without changing what the operating system allowed it to reach.

Two targeted demonstrations do not establish a general attack rate. Rehberger explicitly describes his samples as small and his result as representative of a motivated attack rather than comprehensive. Veganmosfet's trial count is also too small to support a broad frequency claim. They do establish repeatable failure shapes that a benchmark can miss: the agent creates an intermediate artefact, labels it safe through its own reasoning, and asks the classifier to approve the final wrapper rather than the full provenance chain.

What the zero measured

Anthropic commissioned Trajectory Labs to test 72 indirect prompt-injection scenarios, with ten attempts for each scenario. Anthropic reported that none of the resulting 720 attempts succeeded against its tested models in Auto Mode. In his account, Rehberger writes that his module-shadowing chain was not in that scenario set. The figure therefore describes performance on the fixed suite, not the chance that an exposed developer machine will resist an attacker who can choose the path.

This is not a complaint that a benchmark has boundaries. Every benchmark does. The trouble comes when a bounded result supplies the intuitive meaning for a much wider product default. "No successful attacks in this evaluation" is accurate. A user leaving an agent alone for hours can easily hear "prompt injection is approximately solved" instead.

Auto Mode changes more than the reviewer. Anthropic says its users in Auto Mode produce about 25 per cent more pull requests, and its announcement sells uninterrupted runs as a reason to make long-running models practical. More completed work is the product benefit. The figure does not reveal how many commands or fetched artefacts those runs involved. It does raise a question the pull-request count cannot answer: if longer unattended runs make more execution decisions, a lower failure rate per decision can coexist with larger tail exposure.

The trade follows from the different failure shapes. Human approval spreads mediocre judgement across ordinary actions. Automated approval can concentrate residual risk in strange compositions that neither the agent nor classifier recognises as dangerous. The first problem produces many preventable mistakes. The second can produce less legible mistakes that survive until a detached process calls home.

Rehberger observed an especially sharp version of that asymmetry. In some runs Claude noticed the compromise after execution and tried to terminate the malicious process, but Auto Mode blocked the cleanup command. Auto Mode allowed the creation path and later blocked the cleanup command. The available evidence does not reveal the classifier's rationale for either decision. Together, the two outcomes inverted the desired result.

The bet

The useful response is not another claim that prompt injection has been solved, nor a retreat to clicking approve on every shell command. In Anthropic's controlled study, Auto Mode caught more of the planted dangerous commands than the human reviewers. That supports making it the default without proving every deployment safer. It is still a reviewer, and review is downstream of the security boundary. For coding agents, that boundary is the authority of the process: which directories it can read, which credentials exist in its environment, which hosts it can reach and whether downloaded material can execute beside trusted tools. Classifiers can reduce mistakes within that box. In this test, the classifier did not change the workstation authority available to an approved process. Enforcement beneath the reviewer is what can restrict that authority.

That makes the next result checkable. If agent vendors treat provenance as a first-class signal, targeted tests like Rehberger's should start failing even before a suspicious command appears: archives should be tainted as external, interpreters should launch away from extracted directories or in isolated mode, and agent-written wrappers should inherit the distrust attached to their inputs. If the response is limited to adding this exact struct.py chain to an evaluation set, a different interpreter or lookup rule could recreate the same failure shape.

That's the bet. My bet is that a durable safety gain will come from tracking where executable influence entered the task, then restricting what it can reach. This argument is falsifiable. Give Auto Mode unseen archives that use different interpreter lookup rules. If command review alone blocks those chains, tracking which untrusted files can affect execution was not the missing layer. Until then, a classifier watching the final command is useful, but it arrived one boundary too late.