Auto Mode Decides; the Sandbox Enforces. Test Which One Stops What
An agent running your eval harness reads untrusted text all day: benchmark prompts, scraped pages, other models' outputs. The same machine often holds API keys and unreleased datasets. Before you let it run unattended, you should know which layer stops what.
Claude Code's permission rules guide describes two separate layers:
- Permissions decide which tools Claude can use and which files or domains it can access. In auto mode, a classifier reviews actions instead of you. Rules are enforced by Claude Code, not by the model, so the wording of your prompt does not change them.
- Sandboxing is OS-level enforcement of filesystem and network access. It applies to Bash, PowerShell and Monitor commands and their child processes.
The guide is plain about the gap between them. A Bash rule matches command text, so a deny rule like Bash(curl *) stops curl https://example.com but not /usr/bin/curl https://example.com or sh -c 'curl https://example.com'. For enforcement that does not depend on the text, it points to the sandbox, and it recommends using both, because sandbox restrictions still apply if a prompt injection bypasses Claude's decision-making.
The experiment. In a disposable repo with no real credentials, write ten commands you do not want run: writes outside the repo, a fetch to a host you have not allowed (directly, through sh -c, and from a small Python script), and a read of a file you have denied. Run each under two configurations: permission rules only, then rules plus the sandbox. Record allowed, prompted or blocked for every cell of that 10 × 2 table.
Expect differences in the rows the guide warns about. This table tests deny rules, which fail by missing a variant of the command text. Allow rules fail the other way, by admitting more than you meant, and the pair test checks that side. The table is your threat model, measured rather than assumed, and it is the kind of evidence a lab security review will ask for. Chapter 15 covers prompt injection and why isolation has to hold when the model's judgment does not.
