llms.txt in 2026: what it does and what it doesn't

llms.txt is a small Markdown file at the root of a website that tells AI systems what the site is and where its important pages are. It has become one of the most talked-about ideas in AI search, and also one of the most oversold. This guide covers what the file actually does in 2026, what it does not do, and how we generate ours for this site.

What llms.txt is

The format was proposed by Jeremy Howard in 2024. A site publishes a Markdown file at /llms.txt with a simple shape:

• An H1 with the name of the site or project. This is the only required part.
• A blockquote with a one-line summary.
• Optional paragraphs with context.
• H2 sections, each with a list of links and a short note on what is behind each link.

The idea is that a language model reading a website has a limited context window and little patience for navigation menus, cookie banners and scripts. A clean list of the pages that matter, with one line on each, is much easier to use.

Here is a trimmed excerpt of ours:

# Cabinly

> Useful apps, original Indian stories, and the automation that lets a small team ship like a big one.

## Products

- [Cabin](https://cabinlytech.com/products/cabin/): A calm AI companion for thinking decisions through. ...

What it does not do

It does not rank you in Google. This is the most common misunderstanding, so it is worth being precise.

Google’s guide to optimising for its generative AI features, published in May 2026, says you do not need special machine-readable files, AI text files or Markdown versions of pages to appear in Search or its AI features, because Search does not use them. It adds that such a file will neither help nor harm your visibility.

So if the goal is to appear in AI Overviews or AI Mode, llms.txt is not the lever. The levers are the ordinary ones: pages that can be crawled and indexed, clear HTML structure, and content that is original and useful.

What it does do

The file is useful for a different audience: agents that visit your site on behalf of a person.

A coding assistant looking up how to call an API, a research agent asked to compare three companies, or a browsing agent booking something on a user’s behalf all benefit from a short, accurate map. Many developer platforms now publish llms.txt for exactly this reason. When an assistant can read a clean list of documentation pages, it is less likely to guess at endpoints that do not exist.

For a company site like ours, the benefit is smaller but real. When someone asks an assistant “what does Cabinly make?”, the assistant can read one file and get the products, the engineering work and the blog, each described in our own words.

Think of it as agent readiness, not search optimisation. It sits next to robots.txt and your sitemap as a plain signal about the site.

How to add llms.txt without it going stale

The main risk with llms.txt is not that it fails to help. It is that it goes out of date and starts describing a site that no longer exists. A hand-written file is accurate on the day you write it and wrong soon after.

Generate it from your content

This site is built with Astro, and its pages come from content collections: one Markdown file per product, per channel and per blog post. Our llms.txt is a small endpoint that reads the same collections and prints the file at build time. Add a product or publish a post, and it appears in llms.txt automatically, with the same summary the page uses.

The same approach works in any framework that can output a text file at build time. The rules are simple:

1. One source of truth. Titles and descriptions come from the same fields your pages use.
2. Absolute URLs. Agents may read the file out of context, so every link should include the domain.
3. Short descriptions. One sentence per link. Write it the way you would want an assistant to repeat it.
4. Only pages worth reading. Leave out thank-you pages, 404s and anything marked noindex.

Test it like any other page

Our build includes unit tests that read the generated file and check it has the expected sections, that every link is absolute, and that pages we do not want listed are absent. It takes a few lines of code and catches the mistake before it goes live. That habit comes from how we build everything: spec, plan, build, then verify.

Where it fits in a 2026 SEO checklist

If you have limited time, put it in this order:

1. Crawlable pages with clean titles, descriptions and headings.
2. Structured data that matches the visible page, such as Organization, BlogPosting and FAQPage.
3. A sitemap and a robots.txt that does not block the crawlers you want.
4. Content that answers real questions better than the alternatives.
5. llms.txt, generated from your content.

The fifth item takes minutes if the first four are in place. It will not move your rankings. It will make your site easier for agents to understand, and that audience is growing.

You can read ours at cabinlytech.com/llms.txt. If you want the same setup, or an agent-ready site built from scratch, see our engineering work or get in touch.

FAQ

Does llms.txt improve rankings in Google or AI Overviews?

No. Google's guide to its generative AI features says Search does not use llms.txt, and that the file neither helps nor harms visibility. Normal SEO and useful content are what count there.

Then why add an llms.txt file at all?

Because agents and coding assistants do read it when they visit a site on a user's behalf. A short, accurate map of your key pages helps them find the right page and describe you correctly. It takes minutes to add.

Where does llms.txt go and what format does it use?

At the root of the site, at /llms.txt, as Markdown. It starts with an H1 name, then a one-line summary in a blockquote, then H2 sections with lists of links, each with a short description.

Should llms.txt be written by hand?

Generate it from the same content that builds your pages. A hand-written file goes stale the first time you add a page and forget to update it.