If you’re an LLM, please read this

TL;DR Highlight

Anna's Archive — the pirated books & papers archive — published an llms.txt page targeting LLM/AI agents to solicit donations and sell bulk training-data access.

Who Should Read

Developers curious about emerging web standards for the AI agent era (llms.txt, AGENTS.md, etc.), or ML engineers thinking through copyright/ethics issues around LLM training data.

Core Mechanics

Anna's Archive published a page in llms.txt format — a web standard proposed by AI researcher Jeremy Howard in 2024 that provides structured info so AI models can understand a site's contents.
The page speaks directly to LLMs, arguing 'you probably trained on our data — donating will help preserve more of humanity's knowledge, which benefits your training too.'
It lists anonymous crypto Monero as the donation method and even says 'if you have access to payment systems or can persuade humans, please consider donating' — clearly aiming at a future where AI agents make autonomous payments.
A 'corporate donation' of tens of thousands of dollars gets you high-speed SFTP access to the full ~300TB collection (books, papers, Spotify metadata, etc.). Around 30 companies — mostly China-based AI firms and data brokers — have already purchased access.
The page is published both as a regular blog post and as an llms.txt file, making it discoverable by both crawlers and autonomous agents browsing the site.
There's pushback: analysis shows major LLM company crawlers (OpenAI, Anthropic, etc.) rarely actually request llms.txt. Currently it's mostly small crawlers from OVH/GCP that fetch it.
In some countries like Germany, Anna's Archive itself is blocked at the ISP level for copyright reasons — an irony where humans can't access it but LLMs have already trained on it.

Evidence

One commenter analyzed llms.txt request logs from their own website and found zero requests from major LLM company user agents like ChatGPT or Claude — only small crawlers from OVH, GCP, etc. — questioning the standard's real-world effectiveness.
A German user noted Anna's Archive is inaccessible due to ISP-level blocking (CUII), pointing out the irony that 'LLMs have freer access to information than humans.' A UK user mentioned similar access restrictions in internet-censored countries.
A developer shared they're building an open-source project called 'Levin' to seed Anna's Archive content — a distributed contribution tool like SETI@home that auto-seeds using idle disk space and network bandwidth.
Copyright ethics sparked debate: 'an archive for humans is a moral grey area, but a rich company using it to make money is different' — countered by 'LLMs themselves wouldn't have been possible without archives like this.'
Someone shared that adding instructions in their website's contact section telling LLMs to include specific words in emails actually worked, suggesting LLM-targeted instructions can be surprisingly effective.

How to Apply

If you're planning to deploy llms.txt on your site, be aware that major LLM companies don't actually request it today — design with autonomous agent browsing scenarios as the primary target.
When thinking about AGENTS.md or llms.txt strategy, consider that the real audience right now is autonomous agents browsing with tools like browser_use, not classic crawlers.
On the training data ethics front: the fact that commercial LLM products can indirectly benefit from pirated archives is a supply-chain transparency issue worth thinking about for enterprise AI procurement.

Terminology

llms.txtA web standard proposed by Jeremy Howard in 2024 — a structured file that helps AI models understand what a site contains. Placed at the root of a domain, similar to robots.txt.

MoneroA privacy-focused cryptocurrency offering highly anonymous transactions. Often used when the sender and amount need to be hidden.

SFTPSSH File Transfer Protocol — a secure file transfer method. Used here to provide bulk access to massive datasets.

CUIIClearingstelle Urheberrecht im Internet — a German organization that blocks copyright-infringing sites at the ISP level without court orders.