The first version of llms.txt I published was robots.txt with a different filename. It took a second commit, ten minutes later, to actually write something that resembled what the file is supposed to be.
What went in first
llms.txt is a new-ish convention — a plain-text file at the domain root meant to give AI agents and crawlers a readable summary of a site, in contrast to the old machine-only directive format of robots.txt. The first version committed to this site's web/ root was eight lines:
# llms.txt for Linkhub# See https://developers.google.com/search/docs/crawling-indexing/llms-txtUser-agent: *Allow: /Crawl-delay: 2Contact: https://www.linkhub.dk/contactSitemap: https://www.linkhub.dk/sitemap.xml
User-agent, Allow, and Crawl-delay are robots.txt directives. They tell a crawler which paths it may fetch and how fast. None of them mean anything as prose for an AI agent trying to understand what the site is or how to navigate it — which is the actual stated purpose of the convention. The commit had, in effect, copied the syntax of one machine-readable format into a file meant to hold a different kind of content entirely.
What replaced it
Ten minutes later, the whole file was rewritten:
# LinkhubThis llms.txt file helps AI agents understand and interact with Linkhub.## Site Information- **Homepage**: [Linkhub](https://www.linkhub.dk/)- **Contact**: [Contact page](https://www.linkhub.dk/contact)- **Sitemap**: [XML Sitemap](https://www.linkhub.dk/sitemap.xml)## Crawling- **User-agent**: *- **Allow**: /- **Crawl-delay**: 2 seconds
An H1 header, Markdown-style links, a minimum length requirement — all named directly in the commit message as the reason for the rewrite. The crawling directives are still there, but now as prose bullet points describing them rather than as directives a parser would act on. Whether that's meaningfully closer to what llms.txt is supposed to look like is arguable — the convention is new enough that there isn't a single settled spec everyone agrees on — but it's clearly a different intent than the first version, which read as an attempt to satisfy the filename without engaging with what the file is for.
Why the first version happened at all
A brand-new convention with no long history of examples is exactly the situation where pattern-matching against something similar and more familiar — robots.txt, in this case — is the path of least resistance. The filename is different, the general idea (a text file crawlers read) is adjacent, and the directive syntax is genuinely well understood. What's missing is the actual distinguishing feature: llms.txt isn't meant to be machine directives at all, it's meant to be readable context, and that distinction doesn't show up anywhere in a filename or in surface-level pattern matching against a more familiar sibling format.
The real cost of getting it wrong the first time
Nothing about the first version was invisible or dangerous — it's a public text file, wrong in a way anyone reading it could recognize immediately. The actual cost was narrower: for the ten minutes it was live, any crawler or agent that fetched it and took the content at face value would have learned nothing about the site, just a slightly redundant echo of what robots.txt already says at a different URL. For a convention this new, that's a low-stakes place to get it wrong. It's also exactly the kind of mistake that's easy to make confidently — the file looked complete, had a plausible structure, and even cited what looked like an authoritative source in a comment. Checking whether a new format actually does what its name implies, rather than trusting that a superficially similar existing format is close enough, is the only thing that caught it here.