Boris Cherny on Trying to Get Claude Code to Rewrite the Claude App

Who Should Read

A developer handing long tasks to agents who is unsure how to verify the results

Core Mechanics

• Verification is the key skill now
  • Cherny says the skill today is less prompt engineering and more giving Claude a task that seems too hard.
  • Then you must let Claude verify its own work along the way.
  • He calls verification the single most commonly botched thing.
• The experiment: Electron to Swift
  • The Claude desktop app is built on Electron, and he wanted to see what a native version would feel like.
  • He started a Claude Tag session, which is Claude running in Slack.
  • He gave it a GitHub macOS runner and an empty Swift codebase.
• One prompt, simple instructions
  • He told it to run the Electron app in a Mac VM, screenshot it, and compare pixel by pixel to the Swift version.
  • He told it not to stop until done.
  • Asked if that was basically the whole prompt, he said yes.
• Still running after two weeks
  • At interview time it had run about 14 to 15 days.

Community Reactions

• Pixel comparison is shallow, one commenter says
  • They agree verification matters but doubt visual comparison yields a good app.
  • They prefer specifying verification types like a rubric so a human can vouch for the result.
• LLM visual checks were flaky in one experience
  • A commenter built LLM-based mobile layout validation and found results flaky even with Claude.
  • They now run deterministic CSS checks first and use LLMs only afterward.
• Attach verification to every requirement, one commenter says
  • They turn behaviors into requirements and add end-to-end tests.
  • Traceability in tests keeps the agent honest about what was built and verified.
• Cost and design pushback
  • One commenter says two weeks of tokens could cost tens of thousands of dollars and hiring an expert might be better.
  • Another says better technology cannot make up for poor product design.

How to Apply

• Put the verification method in the prompt first
  • Today, before giving an agent a long task, write the definition of done and the verification commands.
  • Write at least two criteria rather than relying on one method like pixel comparison.
• Run deterministic checks before LLM judgment
  • In a UI validation pipeline, run rule-based checks like CSS and DOM first.
  • Leave only rule-resistant issues like broken text to the LLM.
• Have the agent confirm environment access up front
  • Before a long run, ask the agent whether it can reach the runner and repo.
  • Grant missing access before starting.
• Add checkpoints to long runs
  • For multi-day tasks, set a daily checkpoint to review results and cost.

Try It

# Paste into Claude Code or ChatGPT (simplified version of the article's method)

You will perform {{task}}. Before starting, do the following.
1. Check with me one by one whether you have the needed access (repo, runtime, runner).
2. Propose at least 3 completion criteria yourself. Do not use visual comparison alone.
3. For each criterion, write the verification command or test to run.

Then proceed with the task and report verification results in a table at each step. Do not say you are done until all criteria pass.

Next to try: ask it to split criteria into those checkable deterministically and those needing LLM judgment

Terminology

Related Papers

• Same Simulation, Different Evidence: Where Retrieved Tool Output Costs LLM Agents Accuracy

  Saving long tool output to files cut GPT-5.1 accuracy by 11.3 points; naming the authoritative file restored it.

• Fake News, Real Crashes: Information Injection Attacks on LLM Financial Trading Agents

  Fake headlines hijacked LLM trading agents 62% of the time versus 14% for prompt injection.

• Harness Engineering 101: The LLM is the brain, the harness is the body

  An agent is a loop managing a JSON array; 16 chapters build one from scratch

• Harness2 by Google Research – Recursive Agent Harnessing

  Recursively refining the harness with fixed model weights raised benchmark scores by up to 11.8 points.

• Ousted Savannah police officer gave ICE agent husband access to Flock system

  A police officer shared her Flock login with her ICE-agent husband; an audit found 983 searches and she was fired

• Tmux and Claude Code: Sessions, Send-Keys, and the Ceiling

  Running Claude Code agents in tmux works well, but limits appear around four agents.

Related Resources

• https://daringfireball.net/linked/2026/08/02/cherny-claude-swift