Cross-Model LLM Code Review: Should you use Claude to review Codex or vice versa? | AI Paper Digest

TL;DR Highlight

Claude reviewing Codex drafts lifted pass rate from 71.6% to 89.7%; the reverse hurt.

Who Should Read

Developers building pipelines where one coding agent writes and another reviews

Core Mechanics

Review value depends on direction
- Reviewing Codex GPT-5.5 drafts helped.
- Reviewing Claude Opus 4.7 drafts did not help.
Claude as reviewer lifted Codex drafts
- Claude Opus 4.7 review raised Codex GPT-5.5 drafts from 71.6% to 89.7%.
- That nearly matches Claude solo at 91.4%.
Codex reviewing Claude backfires
- It broke 13 passing solutions and fixed only 3 failing ones.
- Pass rate fell from 91.4% to 82.8%.
Claude self-review changed nothing
- It fixed 3 and broke 3, leaving 91.4% unchanged.
- It only added cost.
Good reviews patch, bad reviews rewrite
- Good reviews fixed one small flaw in a sound draft.
- Bad reviews discarded a working draft and rewrote it into a failing one.

Evidence

Effect on Codex drafts (116 LiveCodeBench hard+medium tasks)
- Claude review: 71.6%→89.7% (p_BH=.001), 26 fixes, 5 regressions.
- Codex self-review: 71.6%→84.5% (p_BH=.022), 21 fixes, 6 regressions.
Effect on Claude drafts
- Codex review: 91.4%→82.8% (p_BH=.046), 3 fixes, 13 regressions.
- Claude self-review: stayed at 91.4% (p_BH=1.0).
Cost and latency per task vs solo
- Codex solo $0.190 and 38.5s; Claude review $0.443 and 112.4s; Codex self-review $0.312 and 67.9s.
- Claude solo $0.226 and 86.2s; its self-review was $0.389 and 135.8s with no accuracy gain.
Cost per net extra pass (authors' calculation)
- Claude review (OA) about $1.40; Codex self-review (OO) about $0.95.
- Reproducing all six conditions costs about $225.

How to Apply

Audit your writer-reviewer pairing today
- List which model writes and which reviews in your pipeline.
- Check whether a stronger model reviews a weaker model's draft.
- If a weaker model reviews a stronger one's draft, compare against skipping review.
Add a 'return unchanged if correct' rule to the reviewer prompt
Compare pass rate before and after review on the same tests
- Count fixes and regressions separately on your existing test set.
- Drop the review step if net gain is zero or negative.
Pick the reviewer by budget
- With a weaker writer and under about 70s latency, use self-review.
- When accuracy matters most, add a stronger reviewer.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified version of the paper's reviewer prompt, plus an 'decide whether to intervene first' rule)

You are an expert code reviewer. Review the following solution to a programming problem.

PROBLEM:
{{problem statement}}

SUBMITTED SOLUTION:
{{writer's code}}

Your task:
1. First decide whether to intervene at all. If the draft looks correct, return it unchanged.
2. Otherwise identify bugs, incorrect logic, missing edge cases, or inefficiencies.
3. Fix only the specific flaw. Keep the writer's data structures and interface; do not rewrite from scratch.
4. Do NOT run or test the code; reason purely from code inspection.

Output only the final solution inside <solution> tags.

Next: feed the same {{writer's code}} to another model, compare results, and count fixes and regressions with your tests.

Terminology

LiveCodeBenchA benchmark that keeps adding recent contest problems. Release dates let you filter out problems from before a model's training cutoff.

Static reviewReviewing code by reading it without running it, like a pre-CI human skim.

McNemar testA paired test comparing pass/fail on the same tasks across two conditions. It counts only tasks where the outcomes differ.

BH correctionA correction for running many comparisons, reducing false positives from chance.

RegressionA solution that passed before review but fails after it.

Related Papers

Original Abstract (Expand)

Developers increasingly use two coding agents together: one writes a draft, and the other reviews it. However, it is not clear whether the pairing is worth its cost and time, or whether the order of the pairing matters. We run a controlled experiment on 116 recent hard and medium lcb tasks with Claude and Codex across six conditions to approximate a software practitioner's workflow: both solo baselines, both cross-model orderings, and both same-model orderings. The reviewer sees the problem and the writer's draft but cannot execute tests, which approximates a code review step. Claude review raises Codex drafts from 71.6% to 89.7% ($p_{BH}=.001$); Codex self review raises them to 84.5% ($p_{BH}=.022$). The reverse direction does not pay off: Codex reviewing Claude drafts drops the pass rate from 91.4% to 82.8% ($p_{BH}=.046$), and Claude self review leaves the 91.4% baseline unchanged. Our evaluation indicates that the useful pairing is asymmetric: use Claude to review Codex, not the other way around.