Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

TL;DR Highlight

Post-training a 4B open model on Postgres data targets cheaper agentic retrieval

Who Should Read

Backend developers who want to cut frontier-model calls because multi-turn search agents are too slow and costly

Core Mechanics

Multi-turn search is slow and pricey
- The post says one multi-turn search request with gpt-5.6-sol takes over 10s end-to-end and costs about $0.03.
- It attributes this to calling the frontier model on every loop iteration.
Small open models are 100x cheaper but weaker out of the box
- The post says small open-weights models are 100x cheaper.
- Without extra training, their capability lags closed API models.
RL post-training closes the gap on specific tasks
- RL means the model attempts a task, gets scored, and improves.
- The post claims post-trained open models can match or beat frontier models on tasks like search.
Training needs a task, an environment, and a reward
- The task is answering a question, the environment is a search tool over your corpus, and the reward grades correctness.
- Castform synthesizes questions and answers from your existing documents to make the tasks.

Community Reactions

Three approaches to agentic search, one commenter says
- They list good retrieval itself, generator/evaluator harnesses, and models trained for retrieval, and say this post is the third.
- They also point to SID.ai and Glean's Waldo model as similar work.
Missing comparisons and metrics
- Commenters note there is no comparison with cheaper models like Luna or DSFlash, and no speed comparison.
- Another found it dubious that no common benchmark such as browsecomp plus or metric was reported.
Smaller models won on fact retrieval in one commenter's tests
- One commenter says small models beat larger siblings at fact retrieval from documents.
- They guess larger models overthink, and say they did not verify on a large dataset.
Privacy and corpus quality concerns
- One commenter can't upload sensitive data to a cloud and asks whether this works with rented GPUs and open-source stacks.
- Another asks how it handles outdated or wrong corpus content when the reward comes from the corpus.

How to Apply

Measure cost and latency of your agent search calls
- Today, pull 20 multi-turn search requests and log call count, total latency, and cost.
- Compare against the post's 10s and $0.03 per request to judge whether post-training is worth it.
Wrap hybrid search as one agent tool
- Merge BM25 and vector results with RRF and expose them as a single search tool.
- Use the same tool in training and inference.
Generate QA pairs and draft a reward function
- Use an LLM to generate a question and answer for each document chunk to build an eval set.
- Write a scorer that sums source retrieval, citation, and correctness.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified version of the post's method: synthetic QA and reward scoring design)

Using the document chunks below, build evaluation data for a search agent.

{{문서}}

For each chunk, produce this JSON:
- question: something a person who hasn't seen the document would ask, without reusing its wording
- ground_truth: a one-sentence answer grounded in the document
- source_id: the chunk number it came from

Then define a scoring rubric. Score retrieval (found the right source), citation (cited the right chunk), and correctness (reached the right answer) as 0 or 1 each, then sum them.

Next to try: ask it to grade 10 answers from your current agent with this rubric and summarize what the low-scoring questions have in common.

Terminology

RL post-trainingTraining where a model attempts tasks and improves from scores. Like practicing a game and getting better from the score.

Reward functionA grading rule that scores each attempt. It decides what the model becomes good at.

RolloutOne full run of a task by the model. Training runs thousands of these in parallel.

RRFA way to merge rankings from multiple searches into one. Like summing several judges' rankings.

Hybrid searchSearch that combines keyword-based BM25 and embedding vector search.