OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use

Who Should Read

An agent developer building browser or desktop automation who is struggling with slow, costly screenshot-and-click loops.

Core Mechanics

• Programs replaced click lists
  • The model emits one Python program per call with if, for and assert.
  • Intermediate results pick the next action, so loops and checks need no extra model calls.
• GUI+CLI access alone wasn't enough
  • With the same GUI and shell access, code execution added 6.1 points to mean score over batched GUI+CLI.
  • Steps dropped 14.8%.
• Screenshots only on request
  • The model calls screenshot() instead of the harness capturing after every step.
  • Dropping redundant images raised cache hit rates.
• A todo ledger and intent track state
  • The model records statuses like changed_unverified and claimed_verified.
  • Intent is for user understanding, so scores dipped slightly.
• Real devices exposed big model gaps
  • On the same harness, Qwen3.8-Max used 19.4x GPT-6 Astra's reasoning tokens.

Evidence

• All eight models improved
  • On the 82-task OSWorld-v2 offline set, mean partial score rose 19.6 points (45.3 to 64.9) vs GUI+batch.
  • GPT-6 Astra went 67.1 to 75.7, and GLM-5.3-Flash 29.6 to 64.9.
• Steps fell 53.0% on average
  • Versus GUI+batch, GPT-6 Astra dropped from 102.1 to 33.5 steps, a 67.2% cut.
  • Output tokens fell 27.2% on average, while GPT-6 Astra stayed about the same.
• On-demand screenshots raised cache hits
  • Versus code execution alone, Qwen3.8-Max went 75.6 to 86.1% and DeepSeek-V4.1-Flash 61.9 to 76.3%.
  • Task scores rose for both.
• Four real macOS tasks compared
  • GPT-6 Astra completed 4/4 with 12.9K output tokens per task on average.
  • Qwen3.8-Max completed 3/4 with 73.4K tokens, and reasoning tokens were 2.6K vs 50.2K.

How to Apply

• Add code-execution rules to your agent prompt
  • Today, make your agent emit one Python code block per step, with bash() and assert examples.
  • Attach screenshots only when the model requests them.
• Force verification in code
  • After file creation or form filling, have the model assert file counts and contents.
  • Make it print completion counts to catch omissions.
• Separate unverified work with a todo ledger
  • On long tasks, record changed_unverified and claimed_verified separately.
  • Check for remaining unverified items before declaring completion.
• Measure cost and time curves when picking a model
  • Run candidate models on the same harness and record cost and steps per task.
  • Compare whether the best-scoring model changes by budget.

Try It

# Paste into a Claude or ChatGPT system prompt / first message (simplified version of the paper's method: code execution + verification)

You are an agent working on {{environment, e.g. Linux desktop}}. Each step, write one short <intent> sentence, then exactly one Python <code>...</code> block.
- Combine multiple actions into one program with for/if/assert instead of a list of clicks.
- Prefer bash() over the GUI when files or the shell can do the job.
- Request a screenshot via screenshot() only when code output can't answer your question.
- Verify results with assert before finishing, and print counts.
- Track progress with todo.update(id, kind, content, status) using pending/changed_unverified/claimed_verified.

Task: {{task description}}

Next step: run the same task without this prompt and compare step count and tokens.

Terminology

Related Papers

• Same Simulation, Different Evidence: Where Retrieved Tool Output Costs LLM Agents Accuracy

  Saving long tool output to files cut GPT-5.1 accuracy by 11.3 points; naming the authoritative file restored it.

• Fake News, Real Crashes: Information Injection Attacks on LLM Financial Trading Agents

  Fake headlines hijacked LLM trading agents 62% of the time versus 14% for prompt injection.

• Harness Engineering 101: The LLM is the brain, the harness is the body

  An agent is a loop managing a JSON array; 16 chapters build one from scratch

• Harness2 by Google Research – Recursive Agent Harnessing

  Recursively refining the harness with fixed model weights raised benchmark scores by up to 11.8 points.

• Ousted Savannah police officer gave ICE agent husband access to Flock system

  A police officer shared her Flock login with her ICE-agent husband; an audit found 983 searches and she was fired

• Tmux and Claude Code: Sessions, Send-Keys, and the Ceiling

  Running Claude Code agents in tmux works well, but limits appear around four agents.