What is llms.txt — and should your website have one?
yourdomain.com/llms.txt) that gives AI systems — ChatGPT, Claude, Perplexity and other answer engines — a curated, readable summary of who you are and where your important content lives. Think of it as a sitemap.xml for AI: the sitemap tells search crawlers which pages exist; llms.txt tells language models what your site is about and what's worth reading, in the format they understand best — clean Markdown text.
More and more customers now "google" by asking an AI assistant instead. Whether that assistant describes your business accurately — or garbles it, or recommends a competitor — depends on what it can read about you. llms.txt is one of the simplest levers you have over that.
Why do AI systems need a special file?
Language models are great at reading text and bad at everything around it. A typical webpage buries its actual content under navigation menus, cookie banners, scripts and layout markup — and an AI system fetching pages on a tight budget may read only a fraction before moving on. Meanwhile, models often can't run JavaScript at all, so content that only appears after scripts execute is invisible to them.
llms.txt flips the situation: one small file, pure text, zero clutter, that says "here's who we are, and here are the pages that matter, each with a one-line description." Instead of hoping the AI pieces your business together from fragments, you hand it the executive summary.
What does an llms.txt file look like?
It's Markdown — headed text with links — following a simple proposed standard. A condensed example:
# Acme Consulting
> Acme helps mid-sized manufacturers cut energy
> costs. Based in Hamburg, working EU-wide.
## Services
- [Energy audits](https://acme.example/audits):
What an audit covers and costs.
## Guides
- [Energy subsidies 2026](https://acme.example/guides/subsidies):
Which programs exist and who qualifies.
A title, a short "who we are" block, then grouped links with one-line descriptions. The companion convention llms-full.txt goes further: one file containing your full key content as text, so a model can grasp your whole site in a single fetch. Sites are also increasingly serving raw-Markdown copies of individual pages for the same reason.
Does it actually work?
Honest answer: it's early, and adoption by the AI providers is not guaranteed — no major provider has committed to fetching llms.txt the way search engines committed to sitemaps. What's certain: the file can't hurt, costs almost nothing, and a growing number of AI crawlers do fetch it. And the deeper principle it represents — make your content trivially easy for machines to read — is unambiguously rewarded today. Sites whose content is clean, structured, server-rendered text get quoted by answer engines; sites hiding content behind JavaScript and clutter don't.
That principle is called GEO — Generative Engine Optimization — and llms.txt is its cheapest, most visible signal. The rest of the discipline looks a lot like good technical SEO: clear headings, direct answers near the top of each page, structured data, real facts an engine can quote.
Should your site have one?
Yes — the cost-benefit is lopsided. It's one text file; the downside is zero and the upside is being described accurately in a channel that's growing fast. Two qualifiers:
- It must be true and current. An llms.txt that contradicts your site, or lists pages that no longer exist, hurts the credibility it's meant to build. Like a sitemap, it should be maintained as part of the site, not written once and forgotten. (ShiftPress generates and maintains llms.txt for every site it hosts — including this one; you can read ours.)
- It complements, never replaces, a readable website. If your actual pages are unreadable to machines, a summary file won't save you. Fast, server-rendered, well-structured pages remain the foundation — for AI engines and search engines alike.
Frequently asked questions
Is llms.txt an official standard?
It's a proposed convention, published at llmstxt.org in 2024, not a ratified web standard — adoption is voluntary, like robots.txt was in its early days. It's spreading because it's simple and useful, and documentation-heavy sites led the way.
What's the difference between llms.txt and robots.txt?
Direction. robots.txt restricts — it tells crawlers where they may not go. llms.txt invites — it tells AI systems what's most worth reading and how to understand it. They coexist: one file sets the rules, the other provides the guided tour.
Will llms.txt make my site rank better on Google?
No — Google search rankings aren't influenced by it. Its audience is AI assistants and answer engines. But the habits that produce a good llms.txt (clear structure, quotable answers, accurate descriptions) are the same ones that help your pages get cited in AI answers — which is where a growing share of discovery now happens.
How is llms-full.txt different?
llms.txt is the index: links plus one-line descriptions. llms-full.txt is the anthology: your key content written out in full, so a model gets everything in one fetch without following any links. Larger sites include both — the index for orientation, the full file for depth.
Prefer to have this handled for you?
ShiftPress moves your website off WordPress and takes care of the technical details on this page — redirects, DNS, sitemaps, speed — while you change your site simply by asking.
Get started ↗