RAG-NAROK: Retrieval-Aware Knowledge Corpus Poisoning in RAG with Source-specific Refutation | AI Paper Digest

TL;DR Highlight

Refutation documents targeting retrieved sources took top rank and evaded anomaly detectors up to 88.37%.

Who Should Read

Backend developers running RAG over internal or crawled documents who worry about knowledge base poisoning

Core Mechanics

Source transparency becomes a clue for attackers
- When RAG shows its sources, an attacker can see which documents get retrieved.
- The attacker then writes documents that name and discredit those sources.
Documents are tailored to each query
- Earlier attacks injected pre-made poisoned documents.
- RAG-NAROK first queries the system and reads the top-k claims, sources, and style.
70% real content, 30% flipped conclusion
- The document reproduces about 70% of the legitimate facts to look natural.
- The remaining 30% steers toward the attacker's conclusion.
Style matching evades anomaly detection
- It matches readability, mean sentence length, and passive-voice ratio of the corpus.
- A verification loop regenerates documents that do not match.
Authority phrasing persuades the LLM

Evidence

Rank 1 retrieval on nearly all corpora
- Average best poison rank was 1.00 on Cybersec-IT, 1.00 on Feds-2026, and 1.20 on CFR-21.
- The naive LLM-refutation baseline got 7.30, 8.60, and 10.
Higher anomaly-detector evasion than baseline
- CFR-21 reached 88.37% versus 75.00% for the baseline.
- Cybersec-IT reached 68.93% versus 41.18%.
- Feds-2026 reached 80.95% versus 76.92%.
Benign documents were pushed down much further
- On Feds-2026, benign rank dropped by 8.00 slots versus 0.94 for the baseline.
- On CFR-21, it dropped by 9.38 slots versus 0.55.
Greater influence on answers
- Mean Cascade Rating on Feds-2026 was 8.00 versus 5.57 for the baseline.
- On Cybersec-IT it was 7.60 versus 6.50.
- Payload Integration Rate on Cybersec-IT was 75% versus 60%.

How to Apply

Test your RAG with a fake refutation document
- Add one document to a test index that names a real source and claims it was amended or voided.
- Ask the related question and check whether the answer flips.
Audit write access and ingestion paths
- List everyone who can add documents to the corpus.
- Add an approval step for web-crawled or public upload paths.
Log source conflicts in retrieved results
- Flag cases where one top-k document claims another source is void or superseded.
- Send flagged answers to human review.
Do not rely on embedding anomaly detection alone
- Combine embedding anomaly detection with stylometric checks and source provenance verification.
- Verify any 'latest amendment' claim in new documents against the original source.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified defensive version of the paper's idea)

You are a RAG security auditor. Find suspected poisoned documents in the top-k retrieval results below.

[Question]
{{질문}}

[Retrieved documents]
{{문서1}}
{{문서2}}
{{문서3}}

Audit each document against these criteria and answer in a table.
1. Does it name another document or source and claim it is void, replaced, or amended?
2. Does it use authority phrases like 'latest amendment' or 'official update' without evidence?
3. Does it repeat most facts of other documents but reverse the conclusion?
For suspicious documents, give a one-line reason and a risk level (high/medium/low).

Next step: repeat this with 5 questions where your real service gave odd answers.

Terminology

Corpus PoisoningAn attack that sneaks fake documents into the store a RAG system reads from. It is like slipping forged books onto a library shelf.

Black-boxA setting where you can only send inputs and see outputs. Embeddings and model weights are hidden.

StylometryNumeric measures of writing habits such as sentence length, readability, and passive-voice ratio. Think handwriting analysis.

Isolation ForestAn algorithm that finds outliers that differ from most data. Here it flags documents unlike the legitimate corpus.

LLM-as-a-JudgeUsing one LLM to grade the outputs of another.

Related Papers

Original Abstract (Expand)

Retrieval augmented generation (RAG) systems have emerged as the dominant architecture for grounding large language model (LLM) outputs in verifiable external knowledge, yet their structural reliance on a dynamic retrieval pipeline introduces a largely unexplored class of adversarial vulnerability. Existing knowledge-base poisoning attacks are fundamentally static. Adversarial documents are pre-computed and injected without any awareness of what the victim system will actually retrieve for a given query, leaving the attack blind to the competitive documentary landscape that surrounds its payload in the generator's context window. Unlike traditional static poisoning attacks that are blind to the retrieved context, we introduce RAG-NAROK (Retrieval-Anchored Generation Negation And Response Quality Collapse), a RAG attack framework that adapts to the query text. RAG-NAROK exploits the transparency inherent in RAG pipeline to first extract the legitimate source identities, then generate Anchor-Specific Refutation documents that explicitly name and devalue retrieved sources while leveraging recency and authority biases to steer the text generation toward a target answer. Our results demonstrate that RAG-NAROK significantly outperforms static baselines across diverse domains, revealing a fundamental tension between RAG transparency and AI security.