Skip to content

Document Accept header Markdown content negotiation for AI tools #4153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fern-api wants to merge 2 commits into
base: main
Choose a base branch
from
+6 −4
Open

Document Accept header Markdown content negotiation for AI tools #4153

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
eed93b0
Document Accept header Markdown content negotiation for AI tools
fern-api[bot] Mar 10, 2026
dfafd35
wording update
devalog Mar 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions fern/products/docs/pages/ai/llms-txt.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
---
title: llms.txt and llms-full.txt
title: "`llms.txt` and `llms-full.txt`"
description: Fern automatically creates llms.txt files for AI developer tools. Filter by language, control visibility, and track analytics for LLM traffic.
---

<Markdown src="/snippets/agent-directive.mdx"/>

[llms.txt](https://llmstxt.org/) is a standard for exposing website content to AI developer tools. Fern implements this standard, automatically generating and maintaining `llms.txt` and `llms-full.txt` Markdown files for your documentation site. When an LLM bot accesses your docs, Fern serves these Markdown files instead of HTML, reducing token consumption by 90%+.
[`llms.txt`](https://llmstxt.org/) is a standard for exposing website content to AI developer tools. Fern implements this standard, automatically generating and maintaining `llms.txt` and `llms-full.txt` Markdown files so AI tools can discover and index your documentation.

Fern also serves [clean Markdown instead of HTML](https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation) on any page URL when the request includes an `Accept: text/markdown` header, ensuring agents only receive the content they need. Together, these features reduce token consumption by 90%+.

<Frame>
<img
Expand Down
4 changes: 2 additions & 2 deletions fern/products/docs/pages/ai/overview.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ AI helps keep your documentation current. Fern Writer is a Slack-based technical

## Optimize for AI

Your site is automatically optimized for AI tools and search engines. Fern serves Markdown instead of HTML to AI agents, reducing token consumption and helping these tools process your content faster. Your site hosts `llms.txt` and `llms-full.txt` files so LLMs can index your documentation efficiently.
Your site is automatically optimized for AI tools and search engines. Fern hosts `llms.txt` and `llms-full.txt` files so LLMs can index your documentation efficiently, and supports `Accept` header content negotiation to serve Markdown instead of HTML on any page when requested by AI agents. These features reduce token consumption by 90%+ with no configuration required.

<CardGroup cols={2}>
<Card title="llms.txt" icon="file-text" href="/learn/docs/ai-features/llms-txt" />
<Card title="llms.txt and Markdown serving" icon="file-text" href="/learn/docs/ai-features/llms-txt" />
</CardGroup>
Loading