TREX: An AI code reviewer that runs your code

Jun 17, 2026•dakshgupta•View Original

TL;DR Highlight

Greptile's AI reviewer runs PR code in sandboxes and attaches evidence, not just reads diffs.

Who Should Read

Backend or full-stack developers using AI code review or coding agents who keep missing bugs that only appear at runtime

Core Mechanics

Static review has a ceiling
- It can reason about what code says but not what it does.
- Examples include logic errors needing a specific state sequence, UI regressions after page load, and race conditions needing real requests.
Generating tests is not finding bugs
- The first version was a standalone agent that generated and ran tests, and bugs did not surface.
- The tests were irrelevant to what the user was doing, created noise, and missed edge cases.
Both separate and merged agents failed
- Separate agents explored the same code twice without sharing findings and wasted compute.
- One merged agent was overloaded by spinning up services, taking screenshots, and running tests.
An orchestrator spawns one subagent per issue
- The main reviewer reads the diff, picks issues worth investigating, and runs a TREX agent per issue in parallel.
- Each subagent inherits the main agent's findings and has its own context window.
Bullet summaries can't be verified
- An early agent sometimes claimed to have tested things it hadn't.
- Each finding now comes with screenshots, logs, API traces, execution scripts, and video.
Each review gets a disposable sandbox
- An isolated compute instance starts fresh per review and is thrown away afterward.
- Base images and per-repo snapshots speed setup, while each review fetches exact PR commits and rotates credentials.
A model-agnostic harness avoids lock-in
- The main agent and subagents can use different providers.
- The open-source evaluation harness performs on par with native provider harnesses.

Community Reactions

Execution environments boost agent quality, per experience
- A commenter said sandboxed execution, browser/API driving, and vision-based output checks plug a big hole in the agent feedback loop.
- They described agents running the full stack in Docker containers and injecting audio and video into a real video call for end-to-end testing.
A question about external dependencies
- One commenter asked how this works with many external dependencies like S3 buckets, secret managers, and third-party APIs.
A security concern about running code
- One commenter wondered how long until someone exploits the system.

How to Apply

Have an agent execute one PR today
- Give a recent PR's diff and repro steps to a coding agent with the prompt below.
- Make it separate conclusions from reading versus evidence from actually running.
Require evidence in review comments
- Add fields for the command run, logs, and screenshot paths to your agent review template.
- Reject any "tested" claim that has no evidence.
Split work into per-issue subagents
- Have the main agent list suspicious spots from the diff.
- Hand each spot to a subagent with its own context, in parallel, with a summary of the main agent's findings.
Run reviews in disposable sandboxes
- Spin up a fresh container per PR and check out the exact commit.
- Discard it afterward and cache only dependency installs to avoid stale state.

Try It

copy & run

# Paste into Claude Code or a coding agent that can run code locally/in a sandbox — a simplified version of the TREX approach

You are a code reviewer. Don't just read the code; verify it by running it.

[Change]
{{diff or PR link}}

[Steps]
1. Read the diff and pick up to 3 suspicious spots that could only show up at runtime (state sequences, post-load UI, race conditions, real requests).
2. For each spot, decide how to reproduce it and actually run it. Set up the environment yourself (including auth and feature flags).
3. Report each spot in this format.
- Suspicion:
- Command/script run (verbatim):
- Raw log/output:
- Result: bug confirmed / works correctly / could not run (reason)

[Rules]
- Never write "ran" for something you did not run.
- On failure, say whether it was setup, assertion, or environment.
- Record working behavior as "tested" with evidence too.

Terminology

Static code reviewFinding bugs by reading code without running it. It's like judging a dish from the recipe alone.

OrchestratorA main agent that splits work, hands it to sub-agents, and gathers results. Like a lead assigning an owner per issue.

ArtifactEvidence of what an agent tested, such as screenshots and logs. Like showing your work on a math test.

SandboxA disposable, isolated environment whose effects don't leak outside. A scratch workroom you throw away afterward.

Model-agnostic harnessA run-and-eval framework not tied to one LLM provider. Like a standard outlet that accepts any plug.

Related Resources

https://www.greptile.com/blog/trex-code-execution

TREX: An AI code reviewer that runs your code

Jun 17, 2026•dakshgupta•View Original

TL;DR Highlight

Greptile's AI reviewer runs PR code in sandboxes and attaches evidence, not just reads diffs.

Who Should Read

Backend or full-stack developers using AI code review or coding agents who keep missing bugs that only appear at runtime

Core Mechanics

Static review has a ceiling
- It can reason about what code says but not what it does.
- Examples include logic errors needing a specific state sequence, UI regressions after page load, and race conditions needing real requests.
Generating tests is not finding bugs
- The first version was a standalone agent that generated and ran tests, and bugs did not surface.
- The tests were irrelevant to what the user was doing, created noise, and missed edge cases.
Both separate and merged agents failed
- Separate agents explored the same code twice without sharing findings and wasted compute.
- One merged agent was overloaded by spinning up services, taking screenshots, and running tests.
An orchestrator spawns one subagent per issue
- The main reviewer reads the diff, picks issues worth investigating, and runs a TREX agent per issue in parallel.
- Each subagent inherits the main agent's findings and has its own context window.
Bullet summaries can't be verified
- An early agent sometimes claimed to have tested things it hadn't.
- Each finding now comes with screenshots, logs, API traces, execution scripts, and video.
Each review gets a disposable sandbox
- An isolated compute instance starts fresh per review and is thrown away afterward.
- Base images and per-repo snapshots speed setup, while each review fetches exact PR commits and rotates credentials.
A model-agnostic harness avoids lock-in
- The main agent and subagents can use different providers.
- The open-source evaluation harness performs on par with native provider harnesses.

Community Reactions

Execution environments boost agent quality, per experience
- A commenter said sandboxed execution, browser/API driving, and vision-based output checks plug a big hole in the agent feedback loop.
- They described agents running the full stack in Docker containers and injecting audio and video into a real video call for end-to-end testing.
A question about external dependencies
- One commenter asked how this works with many external dependencies like S3 buckets, secret managers, and third-party APIs.
A security concern about running code
- One commenter wondered how long until someone exploits the system.

How to Apply

Have an agent execute one PR today
- Give a recent PR's diff and repro steps to a coding agent with the prompt below.
- Make it separate conclusions from reading versus evidence from actually running.
Require evidence in review comments
- Add fields for the command run, logs, and screenshot paths to your agent review template.
- Reject any "tested" claim that has no evidence.
Split work into per-issue subagents
- Have the main agent list suspicious spots from the diff.
- Hand each spot to a subagent with its own context, in parallel, with a summary of the main agent's findings.
Run reviews in disposable sandboxes
- Spin up a fresh container per PR and check out the exact commit.
- Discard it afterward and cache only dependency installs to avoid stale state.

Try It

copy & run

# Paste into Claude Code or a coding agent that can run code locally/in a sandbox — a simplified version of the TREX approach

You are a code reviewer. Don't just read the code; verify it by running it.

[Change]
{{diff or PR link}}

[Steps]
1. Read the diff and pick up to 3 suspicious spots that could only show up at runtime (state sequences, post-load UI, race conditions, real requests).
2. For each spot, decide how to reproduce it and actually run it. Set up the environment yourself (including auth and feature flags).
3. Report each spot in this format.
- Suspicion:
- Command/script run (verbatim):
- Raw log/output:
- Result: bug confirmed / works correctly / could not run (reason)

[Rules]
- Never write "ran" for something you did not run.
- On failure, say whether it was setup, assertion, or environment.
- Record working behavior as "tested" with evidence too.

Terminology

Static code reviewFinding bugs by reading code without running it. It's like judging a dish from the recipe alone.

OrchestratorA main agent that splits work, hands it to sub-agents, and gathers results. Like a lead assigning an owner per issue.

ArtifactEvidence of what an agent tested, such as screenshots and logs. Like showing your work on a math test.

SandboxA disposable, isolated environment whose effects don't leak outside. A scratch workroom you throw away afterward.

Model-agnostic harnessA run-and-eval framework not tied to one LLM provider. Like a standard outlet that accepts any plug.

TREX: An AI code reviewer that runs your code

TL;DR Highlight

Who Should Read

Core Mechanics

Community Reactions

How to Apply

Try It

Terminology

Related Papers

Related Resources

TREX: An AI code reviewer that runs your code

TL;DR Highlight

Who Should Read

Core Mechanics

Community Reactions

How to Apply

Try It

Terminology

Related Papers

Related Resources