diff --git a/.agents/skills/afdocs-audit/SKILL.md b/.agents/skills/afdocs-audit/SKILL.md new file mode 100644 index 0000000..9a74871 --- /dev/null +++ b/.agents/skills/afdocs-audit/SKILL.md @@ -0,0 +1,141 @@ +--- +name: afdocs-audit +description: >- + Audit docs.warp.dev for agent-friendly documentation issues using the AFDocs + scorecard. Checks llms.txt, markdown availability, content negotiation, page + size, URL stability, and content structure. Use when asked to check agent + readiness, run an AFDocs audit, improve the docs score, or verify llms.txt + and markdown support. +--- + +# AFDocs Audit + +Run the [AFDocs scorecard](https://agentdocsspec.com/spec/) against docs.warp.dev and report results. + +## Running the audit + +From the docs repo root: + +```bash +node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs \ + --output /tmp/afdocs-report.json +``` + +The script runs `npx afdocs check https://docs.warp.dev --format json`, parses the output, and writes a structured report. + +### Options + +- `--output FILE` — Write the JSON report to a file (otherwise prints to stdout). +- `--url URL` — Override the site URL (default: `https://docs.warp.dev`). + +## Reading the report + +The JSON report contains: +- `score` — Overall score out of 100 +- `grade` — Letter grade (A+ through F) +- `total_checks` — Number of checks run +- `summary` — Counts by status (`pass`, `fail`, `warn`, `skip`) +- `categories` — Per-category scores and grades +- `issues` — Array of failing and warning checks with details and fix guidance + +Each issue includes: +- `id` — Check identifier (e.g., `llms-txt-directive-html`) +- `category` — Check category (e.g., `content-discoverability`) +- `status` — `fail` or `warn` +- `message` — Human-readable description +- `fix` — Suggested fix from the AFDocs spec + +### Known exceptions + +Before reporting, cross-reference every issue against the known exceptions in `references/known-exceptions.md`. Classify each issue into exactly one bucket: +- **Allowlisted** — known exceptions that are intentional (not problems) +- **Remaining** — genuine issues that need attention + +Only include a section if its count is > 0. Never list allowlisted issues under "Remaining." + +## Reporting results + +After running the audit, ALWAYS report the results to the user before taking any action. Include: + +1. **Score**: Overall score and grade +2. **Failures first**: List every fail-severity check with its message and fix guidance. These are the most impactful. +3. **Warnings**: List warning-severity checks with context. +4. **Allowlisted**: Briefly note any known exceptions that were flagged. +5. **If all checks pass**: Explicitly tell the user everything looks clean. + +Example report format: +``` +AFDocs audit complete: 23 checks run, score 82/100 (B). + +**Failures (5):** +- llms-txt-directive-html: No llms.txt directive in HTML pages + Fix: Add a visually-hidden element near the top of each page with a link to /llms.txt +- content-negotiation: Server ignores Accept: text/markdown + Fix: Add middleware to serve .md variants when Accept: text/markdown is requested + +**Warnings (1):** +- llms-txt-coverage: 80% of sitemap pages covered (247/308) + +**Allowlisted (2):** +- page-size-markdown: 1 page over 50K (changelog — intentionally long) +- markdown-content-parity: 7 pages with minor diffs (Turndown escaping, not real content gaps) +``` + +After reporting, ask the user which issues they want to address. + +## Slack notification (optional) + +If instructed to send a report to Slack, post a summary after the audit completes. + +1. Check if `BUZZ_SLACK_TOKEN` environment variable exists. +2. If the token exists, send a summary to the channel the user specified (or the channel configured in the agent's instructions). + +**Format:** + +``` +*AFDocs Audit — * +Score: /100 () | checks | pass, fail, warn + +*Failures ():* +• : + +*Warnings ():* +• : + +*Allowlisted ():* +• : +``` + +Send using: + +```bash +curl -X POST https://slack.com/api/chat.postMessage \ + -H "Authorization: Bearer $BUZZ_SLACK_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "channel": "", + "text": "", + "unfurl_links": false, + "unfurl_media": false + }' +``` + +If `BUZZ_SLACK_TOKEN` is not set, skip the notification and note that the token is required. + +## Dependencies + +Node.js 18+ with npm (for `npx afdocs`). No additional install required — `afdocs` is fetched on demand by npx. + +## Checks performed + +The AFDocs scorecard evaluates these categories: + +**Content Discoverability** — llms.txt existence, validity, size, link resolution, markdown links, and in-page directives +**Markdown Availability** — .md URL support and Accept: text/markdown content negotiation +**Page Size and Truncation Risk** — rendering strategy, page sizes (markdown and HTML), and content start position +**Content Structure** — tabbed content serialization, section header quality, code fence validity +**URL Stability and Redirects** — HTTP status codes and redirect behavior +**Observability and Content Health** — llms.txt coverage, markdown/HTML parity, cache headers +**Authentication and Access** — auth gate detection and alternative access paths + +Full spec: https://agentdocsspec.com/spec/ diff --git a/.agents/skills/afdocs-audit/references/known-exceptions.md b/.agents/skills/afdocs-audit/references/known-exceptions.md new file mode 100644 index 0000000..dee3589 --- /dev/null +++ b/.agents/skills/afdocs-audit/references/known-exceptions.md @@ -0,0 +1,42 @@ +# AFDocs Known Exceptions + +This file lists checks from the afdocs-audit skill that may flag as warnings or failures but are expected and intentional. When reporting audit results, classify these as "Allowlisted" rather than "Remaining." + +## content-start-position + +**Expected status**: fail or warn +**Reason**: Sampled pages may have content starting past 50% of the HTML output. This is inherent to Starlight's layout — sidebar navigation, header markup, and JavaScript/CSS precede the `
` content area. +**Mitigation**: The llms.txt directive, `` in ``, and `Accept: text/markdown` content negotiation middleware all steer agents to the clean markdown version, bypassing the HTML boilerplate entirely. +**Action**: No fix needed. This is a structural property of Starlight sites. + +## markdown-content-parity + +**Expected status**: warn (several pages, ~2% average difference) +**Reason**: False positive. The "missing" segments are numbered heading text like "2. Tabbed File Viewer" where Turndown correctly escapes the period (`### 2\. Tabbed File Viewer`) to prevent markdown parsers from interpreting it as a list item. The content IS present in the markdown — the AFDocs checker's text comparison doesn't account for markdown escaping. +**Affected pages** (as of 2026-05-05): +- `/agent-platform/cloud-agents/triggers/scheduled-agents-quickstart/` — step headings +- `/agent-platform/cloud-agents/integrations/github-actions/` — numbered use case headings +- `/support-and-community/troubleshooting-and-support/troubleshooting-login-issues/` — URLs with special chars +- `/reference/cli/quickstart/` — optional step headings +- `/guides/getting-started/welcome-to-warp/` — numbered section headings +- `/terminal/editor/vim/` — "See Vim docs:" link text +- `/guides/getting-started/10-coding-features-you-should-know/` — numbered feature headings +**Action**: No fix needed. Content is intact. + +## page-size-markdown / page-size-html + +**Expected status**: warn — but only allowlist `/changelog/` +**Reason**: The changelog page (`/changelog/`) is intentionally a single long page (~4,000 lines of MDX). It is excluded from `llms-full.txt` generation due to a `hast-util-to-text` stack overflow, but is still accessible at its URL and indexed by the sitemap. +**Action**: If the only flagged page is `/changelog/`, classify as allowlisted. If other pages are flagged, treat those as genuine issues that may need splitting. + +## section-header-quality + +**Expected status**: skip +**Reason**: Only evaluated when tab panels contain section headers. Most sampled pages with tabs don't have headers inside the tab panels, so the check is skipped. +**Action**: None needed. + +## auth-alternative-access + +**Expected status**: skip +**Reason**: All docs pages are publicly accessible, so no alternative access path is needed. +**Action**: None needed. diff --git a/.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs b/.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs new file mode 100755 index 0000000..b49d369 --- /dev/null +++ b/.agents/skills/afdocs-audit/scripts/afdocs_audit.mjs @@ -0,0 +1,193 @@ +#!/usr/bin/env node +/** + * AFDocs audit wrapper script. + * + * Runs `npx afdocs check` against docs.warp.dev, parses the JSON output, + * and produces a structured report with scores, issues, and fix guidance. + * + * Usage: + * node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs + * node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs --output /tmp/report.json + * node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs --url https://preview.docs.warp.dev + */ + +import { execSync } from 'node:child_process'; +import { writeFileSync } from 'node:fs'; +import { resolve } from 'node:path'; + +const GRADE_THRESHOLDS = [ + [97, 'A+'], + [93, 'A'], + [90, 'A-'], + [87, 'B+'], + [83, 'B'], + [80, 'B-'], + [77, 'C+'], + [73, 'C'], + [70, 'C-'], + [67, 'D+'], + [63, 'D'], + [60, 'D-'], + [0, 'F'], +]; + +function scoreToGrade(score) { + for (const [threshold, grade] of GRADE_THRESHOLDS) { + if (score >= threshold) return grade; + } + return 'F'; +} + +function parseArgs(argv) { + const args = { output: null, url: 'https://docs.warp.dev' }; + for (let i = 0; i < argv.length; i++) { + if (argv[i] === '--output') args.output = argv[++i]; + else if (argv[i] === '--url') args.url = argv[++i]; + else if (argv[i] === '--help' || argv[i] === '-h') { + console.log('Usage: node afdocs_audit.mjs [--output FILE] [--url URL]'); + process.exit(0); + } + } + return args; +} + +function runAfdocsCheck(url) { + // Validate URL to prevent shell injection + try { + const parsed = new URL(url); + if (!['http:', 'https:'].includes(parsed.protocol)) { + throw new Error(`Invalid protocol: ${parsed.protocol}`); + } + } catch (e) { + throw new Error(`Invalid URL "${url}": ${e.message}`); + } + + try { + const stdout = execSync(`npx afdocs check ${url} --format json`, { + encoding: 'utf8', + maxBuffer: 10 * 1024 * 1024, // 10 MB — the JSON output can be large + timeout: 300_000, // 5 minutes + stdio: ['pipe', 'pipe', 'pipe'], + }); + return JSON.parse(stdout); + } catch (error) { + // npx afdocs exits with code 1 when there are failures, but still + // prints valid JSON to stdout. Try to parse it. + if (error.stdout) { + try { + return JSON.parse(error.stdout); + } catch { + // Fall through to error + } + } + throw new Error(`Failed to run afdocs check: ${error.message}`); + } +} + +function buildReport(raw) { + const { summary, results } = raw; + const score = raw.summary?.score ?? estimateScore(results); + const grade = scoreToGrade(score); + + // Group results by category + const categories = {}; + for (const r of results) { + if (!categories[r.category]) { + categories[r.category] = { checks: [], pass: 0, fail: 0, warn: 0, skip: 0 }; + } + categories[r.category].checks.push(r); + categories[r.category][r.status] = (categories[r.category][r.status] || 0) + 1; + } + + // Extract issues (fail + warn) + const issues = results + .filter((r) => r.status === 'fail' || r.status === 'warn') + .map((r) => ({ + id: r.id, + category: r.category, + status: r.status, + message: r.message, + fix: r.details?.fix || r.fix || null, + })); + + return { + url: raw.url, + timestamp: raw.timestamp || new Date().toISOString(), + score, + grade, + total_checks: summary.total, + summary: { + pass: summary.pass, + fail: summary.fail, + warn: summary.warn, + skip: summary.skip, + }, + categories: Object.fromEntries( + Object.entries(categories).map(([name, cat]) => [ + name, + { pass: cat.pass, fail: cat.fail, warn: cat.warn, skip: cat.skip }, + ]) + ), + issues, + all_results: results.map((r) => ({ id: r.id, category: r.category, status: r.status, message: r.message })), + }; +} + +/** + * Estimate score from results when the raw JSON doesn't include a score field. + * Uses a simple formula: (pass / (total - skip)) * 100. + */ +function estimateScore(results) { + const scored = results.filter((r) => r.status !== 'skip'); + if (scored.length === 0) return 100; + const passing = scored.filter((r) => r.status === 'pass').length; + // Warnings count as half-pass + const warnings = scored.filter((r) => r.status === 'warn').length; + return Math.round(((passing + warnings * 0.5) / scored.length) * 100); +} + +function printSummary(report) { + console.log(`\nAFDocs Audit — ${report.url}`); + console.log(`Score: ${report.score}/100 (${report.grade})`); + console.log( + `Checks: ${report.total_checks} total | ${report.summary.pass} pass, ${report.summary.fail} fail, ${report.summary.warn} warn, ${report.summary.skip} skip` + ); + + if (report.issues.length === 0) { + console.log('\n✅ All checks passed!'); + return; + } + + const failures = report.issues.filter((i) => i.status === 'fail'); + const warnings = report.issues.filter((i) => i.status === 'warn'); + + if (failures.length > 0) { + console.log(`\nFailures (${failures.length}):`); + for (const f of failures) { + console.log(` ✗ ${f.id}: ${f.message}`); + if (f.fix) console.log(` Fix: ${f.fix}`); + } + } + + if (warnings.length > 0) { + console.log(`\nWarnings (${warnings.length}):`); + for (const w of warnings) { + console.log(` ⚠ ${w.id}: ${w.message}`); + } + } +} + +// Main +const args = parseArgs(process.argv.slice(2)); +console.log(`Running AFDocs check on ${args.url}...`); + +const raw = runAfdocsCheck(args.url); +const report = buildReport(raw); + +printSummary(report); + +if (args.output) { + const outputPath = resolve(args.output); + writeFileSync(outputPath, JSON.stringify(report, null, 2)); + console.log(`\nReport written to ${outputPath}`); +} diff --git a/.agents/skills/afdocs-fix/SKILL.md b/.agents/skills/afdocs-fix/SKILL.md new file mode 100644 index 0000000..e7f72ea --- /dev/null +++ b/.agents/skills/afdocs-fix/SKILL.md @@ -0,0 +1,164 @@ +--- +name: afdocs-fix +description: >- + Fix agent-friendly documentation issues found by the afdocs-audit skill. + Reads the AFDocs audit report and applies automated fixes for failing checks. + Use after running the afdocs-audit skill, or when asked to fix AFDocs issues, + improve the agent-ready score, or remediate llms.txt / markdown availability + problems. +--- + +# AFDocs Fix + +Apply automated fixes for issues found by the `afdocs-audit` skill. This skill reads an AFDocs audit report and remediates fixable issues. + +## Prerequisites + +1. Run the audit skill first to produce a report: + ```bash + node .agents/skills/afdocs-audit/scripts/afdocs_audit.mjs --output /tmp/afdocs-report.json + ``` +2. Read the report and cross-reference against `afdocs-audit/references/known-exceptions.md` to identify which issues are genuine (vs. allowlisted). + +## Fix procedures by check + +For each failing or warning check, follow the procedure below. Skip checks listed in `known-exceptions.md`. + +### llms-txt-directive-html (Content Discoverability) + +**What's wrong**: No agent-facing directive pointing to llms.txt in the HTML pages. + +**Fix**: Add a visually-hidden element in `src/components/CustomHeader.astro` before the `.header` div: + +```html + +``` + +The `.sr-only` class is Starlight's built-in screen-reader-only utility. The `sanitizeRoot` function in `docs-markdown-integration.js` strips `.sr-only` elements during HTML → markdown conversion, so this directive will appear only in the HTML version (the markdown version gets its own directive — see below). + +**Files**: `src/components/CustomHeader.astro` + +### llms-txt-directive-md (Content Discoverability) + +**What's wrong**: No llms.txt directive in the generated markdown pages. + +**Fix**: In `src/integrations/docs-markdown-integration.js`, in the `convertHtmlToMarkdown` function, prepend a blockquote before the title: + +```javascript +const llmsDirective = + '> For the complete documentation index, see [llms.txt](/llms.txt).\n' + + '> Markdown versions of each page are available by appending .md to any URL.'; +const sections = [llmsDirective, `# ${normalizeWhitespace(title)}`]; +``` + +**Files**: `src/integrations/docs-markdown-integration.js` + +### content-negotiation (Markdown Availability) + +**What's wrong**: Server ignores `Accept: text/markdown` header and returns HTML. + +**Fix**: Create Astro middleware at `src/middleware.ts` that uses the existing `shouldServeMarkdown()` helper from `src/lib/docs-markdown.js`: + +```typescript +import { defineMiddleware } from 'astro:middleware'; +import { + shouldServeMarkdown, + isEligibleDocHtmlPath, + getMarkdownPathFromHtmlPath, +} from './lib/docs-markdown.js'; + +export const onRequest = defineMiddleware(async (context, next) => { + const { request, url } = context; + if (!isEligibleDocHtmlPath(url.pathname)) return next(); + if (!shouldServeMarkdown(request)) return next(); + const mdPath = getMarkdownPathFromHtmlPath(url.pathname); + return context.rewrite(mdPath); +}); +``` + +The `shouldServeMarkdown` helper checks both the `Accept` header (for `text/markdown` and `text/plain`) and user-agent tokens (ChatGPT, Claude, Cursor, etc.). + +**Files**: `src/middleware.ts` (new file) + +### llms-txt-coverage (Observability) + +**What's wrong**: llms.txt covers less than 95% of sitemap pages. + +**Fix**: Update the `customSets` paths in `astro.config.mjs` under `starlightLlmsTxt()`: + +1. Check that every content directory under `src/content/docs/` has a matching customSet entry. +2. Common mismatches: + - Directory was renamed but customSet path wasn't updated (e.g., `university/**` → `guides/**`) + - New content directories added without a corresponding customSet + - Sub-paths excluded too aggressively (e.g., `support-and-community/community/` pages) +3. For large pages that cause `hast-util-to-text` stack overflows: the `exclude` option only applies to `llms-small.txt`, NOT `llms-full.txt`. Do not add pages to `exclude` expecting them to be skipped in `llms-full.txt`. + +**Diagnostic steps**: +```bash +# List content directories +ls src/content/docs/ + +# Compare against customSets in astro.config.mjs +grep -A1 "label:" astro.config.mjs | grep "paths:" +``` + +**Files**: `astro.config.mjs` + +### markdown-url-support (Markdown Availability) + +**What's wrong**: Some pages don't return markdown when `.md` is appended to the URL. + +**Fix**: Check that `src/integrations/docs-markdown-integration.js` and `src/pages/[...slug].md.ts` are both present and correctly configured. The integration generates static `.md` files at build time; the page route serves them in dev mode. + +**Files**: `src/integrations/docs-markdown-integration.js`, `src/pages/[...slug].md.ts` + +### http-status-codes (URL Stability) + +**What's wrong**: The site returns 200 for non-existent pages (soft 404). + +**Fix**: Ensure `public/404.html` or an Astro 404 page exists and returns a proper 404 status code. In Vercel, configure `cleanUrls` and ensure the adapter handles 404 responses correctly. + +**Files**: `vercel.json`, `src/pages/404.astro` (if it exists) + +### MCP Server Discoverable + +**What's wrong**: No MCP server discovery endpoint found. + +**Fix**: Add a static discovery file at `public/.well-known/mcp.json`: + +```json +{ + "name": "Warp Documentation", + "description": "Search and retrieve Warp documentation.", + "url": "https://warp.mcp.kapa.ai" +} +``` + +This points to the existing Kapa-hosted MCP server (OAuth-protected). No custom implementation needed. + +**Files**: `public/.well-known/mcp.json` (new file) + +## Checks with no automated fix + +These checks require infrastructure or design changes that can't be automated: + +- **content-start-position** — Inherent to Starlight's layout. Mitigated by content negotiation and llms.txt directives. See `known-exceptions.md`. +- **page-size-markdown / page-size-html** — Requires editorial decision to split long pages. Flag in the report but do not auto-fix. +- **section-header-quality** — Content-level change requiring human judgment. + +## Applying fixes + +1. Create a branch: `git checkout -b afdocs-fixes origin/main` +2. Apply the fixes for each failing check (skip allowlisted checks). +3. Validate: `npm run build` (the build must succeed). +4. Commit with the prefix: `AFDocs fixes: ` +5. Open a PR: `gh pr create` + +## PR conventions + +- Title must be prefixed with `AFDocs fixes:` (e.g., `AFDocs fixes: add llms.txt directive and content negotiation middleware`) +- Include the audit score (before/after if known) in the PR description +- Include the co-author line: `Co-Authored-By: Oz ` diff --git a/astro.config.mjs b/astro.config.mjs index f66ae03..6b41d7a 100644 --- a/astro.config.mjs +++ b/astro.config.mjs @@ -163,5 +163,9 @@ export default defineConfig({ }), docsMarkdownIntegration(), ], - adapter: vercel(), + // Deploy Astro middleware as a Vercel Edge Function so it runs at + // request time for ALL pages, including pre-rendered static pages. + // Without this, middleware only runs at build time for static pages + // and content negotiation (Accept: text/markdown) doesn't work. + adapter: vercel({ middlewareMode: 'edge' }), });