Harness2 by Google Research – Recursive Agent Harnessing | AI Paper Digest

TL;DR Highlight

Recursively refining the harness with fixed model weights raised benchmark scores by up to 11.8 points.

Who Should Read

A backend developer who hand-tunes prompts, tools and skills for agents and wonders whether that tuning can be automated.

Core Mechanics

A harness is everything around the model
- It covers prompts, tools, memory, skills and orchestration.
- Optimizing that system is called harnessing.
Eight components are editable
- They are system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins.
- They control context construction, the tool interface, delegation and behavior inside the tool loop.
Each task runs Propose–Probe–Compose
- It proposes edits and probes their effects on the same task.
- It composes a refined harness from behavioral contrasts between executions, without ground-truth feedback.
Lessons persist across tasks
- Adopted harnesses and evidence-supported lessons go into a persistent improvement harness.
- That record guides refinement on future tasks.
A single step already improved scores

Community Reactions

How to Apply

Run the local demo without API keys
- On Linux with Python 3.12 and uv, run harness2-demo --output runs/demo --k 2.
- It exercises both recursion modes with deterministic editors and real Python executions.
Break your harness into the eight components
- List which of system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins you use today.
- Decide which ones to open to automatic edits first.
Diff failed and successful runs to draft edits
- Run the same task at least twice and extract the behavioral differences.
- Draft prompt or tool-guidance edits from those differences and keep only adopted ones as lesson notes.
Start benchmarks small and block the network
- Begin with --max-tasks 1 and --dry-run.
- Restrict executor network access before evaluation to prevent benchmark answer leaks.

Try It

copy & run

# harness2 local demo (shell). No model credentials needed

uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

# Model-backed benchmarks cost money, so start small
harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel --only-domains antitrust-competition --max-tasks 1 --dry-run

# Next: drop --dry-run, then run harness2-judge and harness2-report with the same options

Terminology

harnessThe system around a model that lets it act as an agent. Think of the chassis and controls around an engine.

Propose–Probe–ComposePropose edits, test them on the same task, then merge based on behavioral differences. It is like merging config changes based on A/B results.

open worldA setting where the agent meets tasks unseen when the harness was designed. It is like production inputs your tests never covered.

task-level recursion budget (--k)How many refinement steps run per task. It is similar to a retry cap.

Harness2 by Google Research – Recursive Agent Harnessing