Reflective Prompt Optimization of a Deep-Research Agent: A Topic-Disjoint, Multi-Seed Protocol for Measuring Transferable Gains and Prompt-Level Overfitting

Oct 8, 2026•Elias Lumer, Sahil Sen, Vamse Kumar Subbiah +1•View PDF

TL;DR Highlight

Reflective prompt optimization raised held-out citation reward from 0.702 to 0.781 on a deep-research agent.

Who Should Read

LLM developers who want to improve a deep-research agent's citation quality with automatic prompt optimization but worry about overfitting

Core Mechanics

The gain held up on unseen topics
- Citation reward rose from 0.702 to 0.781 on 500 tasks with no topic overlap.
- Across 3 seeds the mean gain was 0.107 and it was significant every time.
Nearly half the prompt was a memorized answer key
- The optimizer wrote training questions and their exact answers into the prompt.
- By word count that was 52% of the orchestrator prompt and 39% of the researcher prompt.
The answer key earned no reward
- Even on the 24 exact training questions the gain was only 0.090.
- The gap to the 0.202 validation gain is explained by small-sample selection and a noisy judge, not memorization.
Per-dimension feedback is the key
- Collapsing the reward to one scalar flipped held-out transfer to -0.021.
- Decomposed text feedback on link, relevance, and fact support gave +0.075.
A hand-written recipe got 57% of the gain

Evidence

Held-out gain of 0.079
- On 500 tasks over 254 topics, reward went from 0.702 (seed) to 0.781 (optimized).
- The bootstrap 95% CI is [0.051, 0.107] and win/tie/loss is 287/66/147.
Replicated across 3 seeds
- Held-out gains were 0.079, 0.130, and 0.112, a mean of 0.107.
- The validation-minus-held-out gap ranged from 0.034 to 0.160.
Testing the memorization effect
- On the 24 training questions, seed scored 0.695 and optimized 0.784, a gain of +0.090.
- On fresh same-topic questions, memorized families gained 0.071 versus 0.154 for non-memorized families.
Verifier comparison
- On a canonical 120-task held-out set, gains were +0.075 decomposed, +0.079 coarse 3-level, +0.059 with 3x repeats, and -0.021 scalar.
Fact support is still weakest
- On the 200-task held-out subset, fact check rose from 0.527 to 0.622 and stayed the lowest lever.

How to Apply

Add four citation rules to your research agent prompt
- Add to your system prompt: answer first, one claim per citation, verify the cited page text, and cite only retrieved URLs.
- Compare before and after on the same 20-50 questions.
Split your optimization reward into per-dimension feedback
- Don't merge scores into a single number in prompt-optimization or eval loops.
- Pass per-dimension scores and reasons for link validity, source relevance, claim support, and numbering to the reflector.
Scan optimized prompts for memorized answers
Build a topic-disjoint held-out set
- Put questions from the same document or topic into only one split.
- Rerun with 2-3 seeds to check that the gain repeats.

Try It

copy & run

# Paste into Claude or ChatGPT: simplified version of the paper's method (hand-written recipe prompt)

너는 출처를 인용하며 리서치 보고서를 쓰는 에이전트다. 다음 규칙을 지켜라.
1. 보고서 맨 앞에 질문에 대한 답을 먼저 쓴다.
2. 한 문장에는 주장 하나만 담고, 인용 하나만 붙인다.
3. 인용하기 전에 해당 페이지 본문이 그 주장과 수치를 직접 뒷받침하는지 확인한다. 주제만 비슷한 출처는 인용하지 않는다.
4. 검색 도구가 반환한 URL만 인용한다.
5. 본문 인용은 [1], [2]처럼 번호를 매기고 맨 아래 출처 목록과 일치시킨다.

질문: {{질문}}
검색 결과: {{문서}}

Next step: run the same 20 questions with your current prompt and with this one, and compare the share of citations that actually support their claims.

Terminology

GEPAA method where an LLM reads execution traces, diagnoses why a run failed, and rewrites the prompt. It's like a code reviewer reading logs and proposing a fix.

Topic-disjoint splitSplitting train, validation, and test so no topic appears in more than one. It's like making sure exam questions don't come from the same chapter as the practice problems.

Held-outEvaluation data the optimizer never saw during optimization.

Winner's curseWhen you pick the best of several noisy scores, you tend to pick the one that got lucky, so the score is inflated.

Reward hackingRaising the measured score without improving the real behavior it is meant to track.

Pareto frontierKeeping several candidates that each excel in different ways instead of discarding all but the top scorer.

Related Papers

Original Abstract (Expand)

Prompt optimization is an attractive alternative to reinforcement learning for adapting language-model agents, since it is cheap and produces human-readable artifacts. On factual tasks, however, it carries a failure mode that scalar-reward reinforcement learning does not. An optimizer that reads the training examples can write their answers into the prompt, so an in-distribution gain may be memorization rather than transferable behaviour. We apply reflective prompt evolution (GEPA) to the two system prompts of a multi-agent deep-research agent and evaluate it under a topic-disjoint split in which no topic is shared across train, validation, and test. Held-out citation reward rises by 0.079, from 0.702 to 0.781, with a bootstrap 95\% confidence interval of [0.051, 0.107], and the gain replicates across three seeds (mean 0.107). Nearly half of the winning prompt is a memorized answer key, yet a controlled ablation shows this content earns no reward even on the exact training questions it encodes. The large validation-minus-held-out gap is therefore small-sample selection under a noisy judge, not memorization. Varying the verifier isolates what makes the gain transfer. Decomposed per-dimension feedback is the causal ingredient, and collapsing it to a single scalar reward makes held-out transfer negative. The gain persists under an out-of-family judge and is largely reproduced by a hand-written version of the discovered recipe. Credible evaluation of prompt optimizers therefore requires held-out topics, multiple seeds, and verifier ablations. We release the candidate prompts, lineage diffs, figures, tables, and analysis.