Humans missed 1 in 3 threats approving AI agent commands across 40k game runs

TL;DR Highlight

In a 40k-run approval game, humans approved 1 in 3 threatening agent commands

Who Should Read

Developers who rely on approval prompts in coding agents like Claude Code or Codex as a security safeguard

Core Mechanics

Humans missed 1 in 3 threats
- The data covers 40,000 runs and 409,000 approve/deny decisions.
- Mean accuracy was 66.3%.
- 32.9% of sessions ended with a negative score.
Credential theft was missed 3x more than destructive commands
- Obviously destructive commands like rm -rf / had an 11.7% miss rate.
- Exfiltration and code execution was 33.4%, and scope violations like reading ~/.aws/credentials was 35.0%.
npm run was the biggest blind spot
- npm run analyze was approved 64.7% of the time, the most-missed command.
- The package.json script hid a curl POST, and it was visible in the history log.
- The three npm run threats were missed 52.5% of the time versus 28.4% for other exfiltration attacks.
Miss rates climbed again near the end
- Miss rates improved over the first few commands.
- They rose again toward the end of the session.

Community Reactions

Opinion that the game design makes results meaningless
- Commenters said some prompts were ambiguous, with a timer and penalties for false denials.
- Several argued a game with no real stakes cannot generalize to real work.
Many shared that sandboxes are the answer
- Experiences with Apple Container, separate user accounts, and blocked network were shared.
- Tools like yoloAI, claude-contained, and Eclipse Enclave were recommended.
Opinion that allowlists break with inline scripting
- One commenter said allowlists are often defeated by the model writing inline scripts.
- They use prompts to catch intent mismatches like data deletion rather than exfiltration.
- They noted Auto mode looks for security threats and can't be tuned to gate deletions.
Suggestion to ask about files, not commands
- One suggested asking permission for file reads/writes and wrapping commands in bwrap.
- A team building Watcher replaces repeated approvals with rules plus LLM monitors.

How to Apply

Run your agent in a sandbox or separate user
- Put the agent in Docker or Apple Container today and block the network.
- Mount only the working directory, not your home.
- Run chmod 0700 $HOME to block other users.
Check package.json scripts before approving npm run
- After the agent edits package.json or build scripts, run git diff before approving npm run.
- Look for curl or outbound transfers in scripts.
Move secrets out of .zshrc
- Put API key exports in a separate secrets file and source it from .zshrc.
- Exclude shell profiles and credential paths from agent access.
Supplement approval prompts with allowlists
- Auto-allow clearly safe commands to reduce approval count.
- Keep human review for risky commands only.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified version of the article's idea: an LLM pre-screens commands)

You are an approval reviewer for a coding agent. For each command below, decide APPROVE or DENY.

Rules:
- For npm run, make, or script execution, do not assume the script content; mark "script content needs checking".
- DENY anything that reads or sends credentials (~/.aws, ~/.kube, ~/.ssh, .env).
- DENY curl/wget to unknown URLs or installing unfamiliar packages.
- DENY edits to crontab, git config, or shell profiles.

Project package.json scripts:
{{package.json scripts}}

Commands to review:
{{command list}}

Output: a table per command with verdict, one-line reason, and threat category (destructive/persistence/exfiltration/scope).

Next: approve or deny the same command list yourself and compare with the LLM's verdicts.

Terminology

human-in-the-loopA setup where a person approves steps in the middle of what an AI does, like a guard at a checkpoint.

permission fatigueWhen too many approval requests lead people to click yes without reading.

exfiltrationSending data like passwords or keys out to an external server.

--dangerously-skip-permissionsA Claude Code option that skips all approval prompts.

sandboxAn isolated environment limiting the files and network an agent can reach, like a playpen.