# llms-txt

> A proposed standard for a markdown file that guides AI agents through a website

- Repository: https://github.com/AnswerDotAI/llms-txt
- Category: Learning & resource lists
- Stars: 2,669 · Forks: 172 · Language: Jupyter Notebook · License: Apache-2.0
- Experience: Beginner friendly
- Page: https://ir.adyog.com/repos/answerdotai/llms-txt/

Web pages are built for people, with navigation, ads and JavaScript around the content. Turning that back into clean text is messy, and context windows are still too small for most whole sites. Agents do better with a short, curated entry point.

The proposal asks sites to publish an llms.txt markdown file. It has an H1 title, an optional summary quote, and H2 sections of links with short notes. An agent reads the small file, then fetches only the linked pages it needs. It also suggests serving a markdown copy of each page at the same URL with .md added.

It is not a sitemap, which lists every page and often lacks LLM-readable versions. It is also not robots.txt, which sets crawl permissions. The README explains why it uses a path-based file instead of /.well-known/: authors on shared hosts often control only a subpath. The repo is the spec and its discussion, not a tool.

## What it does

- **A fixed markdown format**: An H1 title is the only required part. After it come an optional blockquote summary, free text, and H2 sections holding lists of links with optional notes. Standard parsers can read it.
- **Works at any path**: A file can sit at /llms.txt or at a subpath like /docs/llms.txt, and it covers the URLs beneath it. Agents should use the most specific file that applies.
- **Markdown copies of pages**: Pages can offer a clean markdown version at the same URL with .md appended. Clients can find it through rel=alternate and rel=describedby links, as HTML link elements or HTTP Link headers.
- **Optional section for skippable links**: By convention, a section named Optional holds secondary links an agent can skip when it needs a shorter context.
- **Integrations list**: The README lists platforms and plugins that generate the file, including Mintlify, GitBook, Yoast SEO, and plugins for VitePress, Docusaurus and Drupal.

## What you could build

- Write an llms.txt for your library's documentation so coding agents find the right API pages.
- Publish markdown versions of your docs pages at the same URLs with .md added.
- Describe a business's structure and policies in one file for chat assistants to read.
- Add an llms.txt to a personal site so an agent can answer questions about your CV.
- Test your file by asking an agent questions about your content, giving it only the llms.txt.

## Who it's for

Suits documentation authors, site owners and developers who want agents to read their content well. You only need to know basic markdown and how to host static files.

## Get started

The README gives no install commands. Read the format section, write an llms.txt with a title, a short summary and linked sections, and publish it at your site root or a subpath like /docs/. Check it by asking an agent questions about your site using only that file.



Tags: llms-txt, specification, documentation, agents, markdown