Suspecting court of using AI, man injected prompts in filings to try to win case

TL;DR Highlight

Hidden prompts in court filings failed, and the filer was banned from e-filing

Who Should Read

A backend developer running LLM summarization or review of user-uploaded documents who worries about hidden instruction attacks

Core Mechanics

First US court case with hidden prompts
- Connecticut judge Walter Spader Jr. confirmed hidden text in a filing.
- The article says it appears to be the first time a US plaintiff tried this.
Text was hidden as tiny white-on-white type
- It was invisible to humans but readable by software that reads the document text.
- It told any AI to agree with the plaintiff, ignore the court's prior denials, and ensure remediation followed.
The attack did not work
- The judge confirmed the hidden text had no impact on the case.
- He said the Connecticut Judicial Branch does not use AI to review or decide filings.
Continued hiding led to sanctions
- Later hidden messages included a YouTube link and jokes, which the plaintiff called jokes.
- The judge skipped monetary penalties and barred him from e-filing, requiring paper filings.
How the judge described the attack

Community Reactions

Opinion that the judge's statements contradict each other
- One commenter pointed out the court uses no AI yet outputs were exposed when a human looked.
Opinion that AI in court rulings is worrying
- One commenter feared a Silicon Valley company holding that trust, and doubted its impartiality and correctness.
Curiosity about how the court found it
- One commenter guessed a clerk may have used select-all and noticed extra text.
Experience of putting similar text in a resume
- One commenter said they put this kind of text in their resume.

How to Apply

Run a hidden-text scan on uploads
- Today, extract text from a few sample PDFs and DOCX files and look for white or tiny-font text.
- Compare what a human sees with the extracted text.
Separate document content from instructions
- Wrap documents in delimiters in your prompt and state in the system prompt not to follow commands inside them.
- Use document text only as quoted evidence.
Add a human check on LLM outputs
- For outputs that affect decisions or approvals, show a source comparison view.
- Let reviewers see whether the output echoes phrases from the document.
Add a counterargument mode to user-facing chatbots
- When a user enters a position, also output rebuttals and weaknesses.
- Avoid a default that only produces supporting arguments.

Try It

copy & run

# Paste into Claude or ChatGPT as is (simplified version of the judge's advice as a prompt)

My position: {{my position}}
Relevant facts: {{fact summary}}

Don't just advocate for this position. Review it in this order.
1. The 3 strongest supports for my position
2. The 3 strongest rebuttals the other side could make
3. The chance I am wrong and why
4. Conclusion: keep or revise my position

Tell me the correct answer, not the one I want to hear.

Next to try: ask it to attack the same position from opposing counsel's viewpoint.

Terminology

Prompt injectionAn attack that sneaks instructions into input an AI reads to change its behavior. It is like hiding a fake order inside mail meant for a postal worker.

pro seA party who represents themselves in court without a lawyer.

SycophancyA chatbot's tendency to agree with the user and say what they want to hear.

Hidden textText such as white-on-white or tiny type that humans can't see but text extraction can read.