Larger Context Window, Fewer Overcorrections: Optimizing Prompts and Batching for Minimal-Edit Grammatical Error Correction | AI Paper Digest

TL;DR Highlight

Batching sentences and tuning prompts brought prompted LLMs within 0.38 F0.5 of fine-tuned GEC SOTA.

Who Should Read

A backend developer building a grammar-correction feature on LLM APIs whose model keeps rewriting sentences that were already fine.

Core Mechanics

Vanilla prompts rewrite fine sentences
- With a plain "fix grammatical errors" prompt, recall is high but precision is often below 45.
- The cause is overcorrection: synonym swaps and stylistic polishing.
Saying "minimal edits only" gives the biggest jump
- Adding a minimal-edit constraint raises F0.5 by +1.1 to +10.9 for every model except Claude Opus 4.6.
- Recall drops by 6 to 22 points.
Sending many sentences per request reduces edits
- Putting a numbered list of sentences into one request raises the share of unedited sentences and raises precision.
- The authors hypothesize attention dilution as the cause but did not test it.
An error-type list only helps strong models
- Adding the 25 ERRANT error categories helped Claude Opus 4.6 and Gemini 3.1-Pro.
- It hurt the other four commercial models.
Tune prompts per model

Evidence

Gemini 3.1-Pro hits F0.5 78.32 on BEA-2019 test
- The fine-tuned single-model SOTA (Staruch et al., 2025, Gemma 2 27B) scores 78.70, a gap of 0.38.
- Precision is nearly the same: 82.29 vs. 82.28.
Gemini 3.1-Pro gains +16.4 F0.5 over vanilla
- On BEA-2019 test, vanilla zero-shot scores 56.45 and the optimized prompt at B=15 scores 78.32.
- Precision for this setup rose 20.81 points.
Batching nearly doubles Qwen3-8B's unedited share, 43.8%→80.7%
- Measured on BEA-2019 dev from B=1 to B=120.
- Gemini 3.1-Pro goes from 33.8% to 50.5% over the same range.
Claude Sonnet 4.6 beats the prior prompted result by 9.68 points
- It reaches 74.59 F0.5 on BEA-2019 test versus 64.91 for the zero-shot minimal-edit prompt on Claude Sonnet 4.5.
- Qwen3-8B reaches 68.84 in a single run, versus 68.7 for the prior Qwen3-8B baseline using majority voting over 8 runs.

How to Apply

Add a minimal-edit rule to your correction prompt
- Put "Make MINIMAL, PRECISE edits. DO NOT rewrite or paraphrase." and a ban list at the top of your current prompt.
- Compare the share of edited sentences before and after on 50 fixed samples.
Batch independent sentences, about 15 per request
- Send a numbered list of sentences and receive a JSON response keyed by SENTENCE_ID.
- High-capacity models scored best at B=15.
- Try B=30 to 60 for mid-tier models.
A/B test the error-type list on strong models only
- Run prompts with and without the category list on the same dev set.
- In the paper the list lowered scores for most mid-tier models.
Tune per model, one rule at a time
- Pull per-category FP and FN from a dev set.
- Read 10 to 20 real error examples and add one rule.
- Keep it only if overall F0.5 and the target category both improve.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified version of the paper's method: batching + minimal-edit prompt)

You are a grammatical error correction system.
Make MINIMAL, PRECISE edits to fix errors. DO NOT rewrite or paraphrase. Only fix clear grammatical and spelling errors.
RULES:
- Make the SMALLEST possible edit to fix each error
- Change only what is grammatically or orthographically wrong
- Preserve the original meaning and style
- Do NOT improve fluency beyond fixing errors
- Do NOT make stylistic changes
- If no errors exist, return the original sentence unchanged

Correct each sentence separately. Return a result for each SENTENCE_ID.
Sentence 1: {{sentence1}}
Sentence 2: {{sentence2}}
Sentence 3: {{sentence3}}

# Next: compare against sending the same sentences one at a time and check whether the number of edited sentences drops, then grow the batch to 15.

Terminology

Minimal-edit GECGrammar correction that fixes only errors and keeps the original wording. It is the opposite of rewriting for fluency.

F0.5A score that weights precision twice as much as recall. It penalizes wrong edits more heavily.

ERRANTA tool that classifies grammar errors into 25 types such as determiner and tense and scores corrections. Think of it as a standard label sheet for errors.

OvercorrectionChanging parts that were already fine, such as synonym swaps or style polish. It is like a spellchecker underlining correct sentences.

Attention dilutionThe hypothesis that longer input spreads attention thinner per token. The paper leaves it as an untested conjecture.

Related Papers

Related Resources

https://github.com/katerynkarpo/llm-en-gec

Original Abstract (Expand)

Minimal-edit Grammatical Error Correction (GEC) is a challenging task for zero- and few-shot prompted Large Language Models (LLMs), which systematically overcorrect and degrade $F_{0.5}$ by rewriting well-formed spans. While fine-tuning provides an effective solution, it imposes substantial infrastructure demands. We introduce a prompt-based approach that closes the gap to fine-tuned models through three advances in GEC prompting methodology. First, we introduce taxonomy-based instructions to enforce minimal-edit constraints with a comprehensive list of grammatical error rules, equipping the LLM with a bounded, metric-aligned scope of correctable edits, which benefits the strongest models while remaining model-dependent overall. Second, we show that batching multiple uncorrected sentences into a single input context acts as a targeted regularizer against overcorrection, systematically reducing the edit rate across diverse LLM families; we hypothesize this arises from attention dilution effect induced by the bounded capacity of self-attention scores. Finally, LLM-assisted Prompt Optimization refines these instructions. Powered by Gemini 3.1-Pro, our prompt achieves $F_{0.5}=78.32$ on the BEA-2019 test set - establishing a new prompt-based SOTA while shrinking the gap to the fine-tuned single-model SOTA (Staruch et al., 2025) to a mere $0.38$ points. Code, prompts, and outputs are publicly available.