Skip to content

[Chore] Add llms.txt for AI/LLM context discovery#7442

Open
Xynash wants to merge 3 commits intolayer5io:masterfrom
Xynash:chore/add-llms-txt
Open

[Chore] Add llms.txt for AI/LLM context discovery#7442
Xynash wants to merge 3 commits intolayer5io:masterfrom
Xynash:chore/add-llms-txt

Conversation

@Xynash
Copy link
Contributor

@Xynash Xynash commented Feb 28, 2026

Description
This PR fixes #7421 by introducing an llms.txt file to the root directory (via /static/llms.txt). This file provides high-density context for LLMs and AI crawlers to accurately interpret the Layer5 ecosystem.

Implementation Details:

  • Token Optimization: Structured the content with semantic-rich descriptions to maximize efficiency within LLM context windows.
  • Categorization: Organized the file into sections for Core Projects, Technical Documentation, and Collaboration Hubs.
  • AI Guidance: Included a specific "AI Instructions & Priorities" section to define content restrictions and prioritize the Meshery documentation as the primary source of truth.

Proposed Maintenance Process:
To ensure this document stays current as the ecosystem evolves:

  1. Manual Check: Include llms.txt in the project's monthly documentation audit checklist.
  2. Release Policy: Propose that any PR introducing a new major project to the site's metadata also include an update to the core project list in llms.txt.
  3. Automation (Future): A GitHub Action can be developed to flag discrepancies between projects.json and the llms.txt content.

Notes for Reviewers
Verified that Gatsby serves the file at the root level (/llms.txt) during local build testing.

Signed commits

  • Yes, I signed my commits.

Signed-off-by: Ansh Sharma <anshsharma161105@gmail.com>
…n for priorities and restrictions

Signed-off-by: Ansh Sharma <anshsharma161105@gmail.com>
@Xynash
Copy link
Contributor Author

Xynash commented Feb 28, 2026

Hi @leecalcote & @ritzorama adding up a quick note to supplement this PR description :

To ensure full alignment with the requirements in #7421, I want to confirm the Post-Merge Submission Plan. Once this PR is merged and live at layer5.io/llms.txt, I will personally take the lead on submitting the URL to:

- [https://directory.llmstxt.cloud](https://www.google.com/url?sa=E&q=https%3A%2F%2Fdirectory.llmstxt.cloud)

I've also verified that the 'AI Instructions & Priorities' section I added to the file correctly handles the terminology restrictions and source-of-truth priorities you requested. Ready for review whenever you have a moment!

@l5io
Copy link
Member

l5io commented Feb 28, 2026

🚀 Preview for commit 99835ce at: https://69a277d041393523b476b608--layer5.netlify.app

@Xynash
Copy link
Contributor Author

Xynash commented Feb 28, 2026

I've verified the implementation in the deploy preview

I wanted to confirm the Post-Merge Submission Plan: As soon as this is merged, I will take the lead on registering the live URLs.
I've also ensured that the 'AI Instructions' section addresses the priorities and terminology restrictions mentioned in the original issue.

@yi-nuo426 yi-nuo426 requested a review from Copilot March 9, 2026 20:49
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an llms.txt file to the site’s static root to provide structured, high-density context for AI/LLM crawlers and direct them to canonical Layer5/Meshery resources.

Changes:

  • Introduces static/llms.txt with curated links to core projects, docs, and collaboration hubs.
  • Adds an “AI Instructions & Priorities” section to bias LLM outputs toward Meshery docs and preferred terminology.

static/llms.txt Outdated

## AI & Automation
- [Model Registry](https://github.com/meshery/meshery/tree/master/server/meshmodel): Machine-readable definitions of cloud-native components.
- [Full Context](/llms-full.txt): Comprehensive documentation for deeper LLM reasoning.
Copy link

Copilot AI Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR introduces a link to /llms-full.txt, but this PR doesn’t add that file. If it doesn’t already exist in static/, this will 404 when crawled. Either add static/llms-full.txt in the same PR or remove/replace this link (optionally using an absolute URL if you want it to be robust to non-site rendering contexts).

Suggested change
- [Full Context](/llms-full.txt): Comprehensive documentation for deeper LLM reasoning.
- Full context: Comprehensive documentation for deeper LLM reasoning.

Copilot uses AI. Check for mistakes.
@leecalcote
Copy link
Member

leecalcote commented Mar 11, 2026

Good initiative - the llms.txt convention is gaining traction as a way to help AI crawlers understand site content (analogous to robots.txt). The static file placement in /static/ is the correct Gatsby approach and the section structure is clear.

Issues to address:

  1. Missing newline at end of file - The diff shows \ No newline at end of file. This violates POSIX conventions and can cause problems with certain tools. Please add a trailing newline.

  2. Dead link — The file references /llms-full.txt under AI & Automation > Full Context, but this file doesn't exist. Either create it alongside this PR or remove the reference to avoid serving a 404.

  3. Inaccurate contributor guide copy - The Contributor Guide entry mentions "Jekyll contributions," but this site uses Gatsby. That's misleading for both contributors and LLMs ingesting this file.

  4. Missing project: Cloud Native Patterns - layer5.io/community/patterns is a notable Layer5 project that's absent from the Core Projects section.

  5. "AI Instructions & Priorities" framing - This section reads more like marketing copy than machine-readable guidance. The llms.txt spec doesn't define a normative instruction format, so consider whether this content belongs in an llms-full.txt if/when that file is created.

  6. Verify the integration count - "300+ cloud-native integrations" should be confirmed against current Meshery documentation before publishing to avoid stale numbers.

Overall the structure is solid. The dead link and the missing newline are the most critical items before merge.

Copy link
Member

@leecalcote leecalcote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested.

@Xynash
Copy link
Contributor Author

Xynash commented Mar 12, 2026

Thank you for the review Lee . I'll update to address all the points mentioned above .

Signed-off-by: Ansh Sharma <anshsharma161105@gmail.com>
@l5io
Copy link
Member

l5io commented Mar 12, 2026

🚀 Preview for commit 027dd6b at: https://69b25e7933f4fef05039e425--layer5.netlify.app

@Xynash
Copy link
Contributor Author

Xynash commented Mar 12, 2026

Hi @leecalcote! I've updated the llms.txt PR to address all review points you requested .

I've verified all URLs in the latest deploy preview as I've

  1. Fixed Jekyll/Gatsby copy
    
  2. Added Cloud Native Patterns
    
  3. Updated integration count to 1000+
    
  4. Added trailing newline POSFIX
    
  5. Resolved broken redirects links 
    

All checks are green and ready for your final review !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create an llms.txt

4 participants