How AI text watermarking works
TL;DR Highlight
A secret key nudges word choices to hide a statistical mark in AI text, detectable only by the key-holder.
Who Should Read
An LLM API developer who heard Claude and Gemini outputs are now watermarked and wants to understand how it works and where it fails.
Core Mechanics
- The mark hides in word choices, not characters
- At each word the model rolls weighted dice over a shortlist of candidates.
- Many candidates are equally natural, and that slack is where a pattern can hide.
- A secret key splits candidates into green and red and nudges green
- The key-derived coloring can only be reproduced by the key-holder.
- The color depends on the few words just before, so the same word can be green or red in different contexts.
- Detection means recoloring with the key and counting
- It does not judge style, only whether green appears more often than chance.
- Without a mark or with the wrong key, green lands about half the time, like a coin flip.
- A gentle lean needs a long text, so short texts are hard to call.
- Editing only erases the mark where runs of wording break
- Typo fixes and light edits dilute the mark but do not delete it.
- Only a full rewrite sharing no runs of wording with the original removes it.
Community Reactions
- Opinion that running an open model sidesteps watermarks
- Commenters said you can pick a provider that does not watermark or run an open-source model yourself.
- Another noted the watermark depends on how the model is run, so open-weight providers cannot apply it.
- Concern about reduced creativity and code quality
- One commenter argued pushing words toward one group cuts expressive variety, worse for code with few choices.
- They guessed code might be excluded from watermarking.
- Experience removing marks by hand or with a local model
- One developer said they transcribe AI output by hand, replacing words as they go.
- Another said rewording through a local model clears the marker.
- Questions about key recovery and legal use
- A commenter asked how much output it would take to work out the key.
- Others asked whether the EU law forces providers like those on OpenRouter to watermark, and suggested the marks could feed legal ownership disputes.
How to Apply
- Audit your AI output post-processing pipeline
- Today, list every feature that passes Claude or Gemini output to users unchanged.
- Assume provider marks may remain in that text and reflect it in your terms and notices.
- Never present watermark results as a definitive verdict
- Show a detection result only as a probability of being processed.
- Do not label text as human-written just because no mark was found.
- Exclude short and code outputs from detection
- Set a minimum length threshold if you build detection logic.
- Skip code, quotes, and lists where few choices exist.
Try It
# Paste into Claude or ChatGPT as is (simplified version of the article's method: a toy to experience watermark principles)
You are a text watermark simulator. Complete one sentence step by step using these rules.
1. Sentence start: "{{sentence start}}"
2. At each step, show 4 next-word candidates with probabilities (sum 100%).
3. Randomly split them into 2 green and 2 red, raise each green by 10 percentage points, subtract the same total from red, and show the new odds.
4. Pick one word using the new odds and append it to the sentence.
5. Repeat for 10 steps, then compute the share of chosen words that were green.
Question: How far above 50% was the green share? Explain how confidence would differ for 10 words versus 100 words.
Next to try: ask it to replace half the words in the final sentence and recompute the green share.Terminology
Related Papers
The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.
90% of US working hours are spent on tasks not covered well by AI benchmarks
Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage
Postgres 19 delayed after AI reviewers find more bugs than humans can fix
LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted
Show HN: Gemini 4 Argon plays StarCraft
LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.
A Plausible Explanation Is Enough to Mislead Multimodal Judges
Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.
Neural networks are clean
Looking at neurons directly shows most have one main job, challenging superposition.