docs: Generate CLI reference docs for kagent and kmcp

[yashrajshuklaaa]

Closes #262.

What this does

Adds automated CLI reference doc generation for both kagent and kmcp, replacing kagent's hand written CLI pages and kmcp's flat manual pages. Wired into the existing update-ref-docs.yaml workflow, same as the API and Helm docs.

Why this approach

kagent and kmcp are not built the same way internally. kmcp exports its Cobra root command from a public package, so it could be documented natively using cobra/doc. kagent's root command sits inside package main and uses internal/ packages, so it cannot be imported from an external repo at all, this is a Go language restriction, not something fixable from the website side.

Since native generation is not available for both CLIs equally, this PR uses one approach for both: build the real binary in CI, then walk <binary> <path> --help recursively to reconstruct the full command tree. This is the same category of approach used in agentgateway/website's generate-agctl-ref.py, linked as prior art in the issue, though that script parses source statically while this one reads the actual compiled binary's output.

Calling --help recursively is safe, Cobra intercepts it before any PreRun/RunE hooks run, so no side effects like docker-compose or cluster calls get triggered during generation.

What is new

• scripts/generate-cli-docs.py: the generator script. Produces one Hugo page per command, matching the existing kagent-<command>.md naming style, with frontmatter, flags, subcommand links, and examples.
• scripts/audit-cli-docs.py: a small check script that verifies frontmatter, code fences, and section formatting before the generated pages get committed.
• Four new steps in .github/workflows/update-ref-docs.yaml: build each CLI binary, smoke test it with --help, then generate docs into the right content folders.

Content changes

• docs-site/content/kagent/resources/cli/: fully regenerated, as agreed in the issue. Nested subcommands now get their own pages instead of being folded into one parent page, so future subcommands get documented automatically.
• docs-site/content/kmcp/reference/: the 9 existing manual CLI pages moved into a new cli/ subfolder and are now generated, matching kagent's structure.
• kagent-help.md and kmcp-help.md removed, Cobra's built in help command has no real content of its own. completion is kept, since it documents actual shell specific subcommands that are genuinely useful.

Drift this caught

Regenerating from real --help output showed the old manual docs had gone stale in a few places:

• kagent deploy still documented --api-key/--api-key-secret, which do not exist anymore (--env-file replaced them)
• kagent invoke still documented --url-override, replaced by --token
• default model listed was gemini-2.0-flash, actual default is now gemini-2.5-flash

Known trade-off

The old kagent-mcp.md had hand written links to kmcp's dedicated reference pages. --help text cannot express that kind of cross-repo relationship, so those specific links are not present in the generated version. Happy to add them back manually if maintainers would like that.

Testing done

• Checked script output against the real --help text of both CLIs, not just a sample fixture. This caught and fixed a few real issues:
  • kmcp uses non standard header formatting in places (Available Commands:: with double colon, GlobalFlags: with no space), now handled
  • kagent's init command prints a duplicated Examples: section, only the first one is kept now
  • subcommand links initially pointed to raw .md filenames, which do not resolve under Hugo's URL scheme, fixed to use correct relative paths
  • the --config flag default was leaking the machine's actual home directory into the docs, now shown as the generic $HOME placeholder
• scripts/audit-cli-docs.py runs clean across all 61 generated pages
• Verified rendering locally with hugo server, including clicking through nested subcommand links

Follow ups (not blocking this PR)

• If kagent later exports its root command from a public package, it could switch to native cobra/doc generation like kmcp, without changing the page layout.
• The kagent-mcp.md cross-links mentioned above, if maintainers want them restored.

[yashrajshuklaaa]

cc @kristin-kronstain-brown PTAL

[yashrajshuklaaa]

All checks are green except "Preview / deploy" which looks like it needs Cloudflare secrets that aren't available to fork PRs, seems unrelated to the actual changes.

[kristin-kronstain-brown]

@yashrajshuklaaa
Thanks for this. I'm really excited to have this functionality added! I built both CLIs and reran your generator against them, so here's what I found.

Looking good!

• The pipeline reproduces exactly. Built kagent-dev/kagent@426a1330 and kagent-dev/kmcp@1cec647, ran generate-cli-docs.py against both, and got output byte-identical to all 61 committed pages on a different machine, different OS, different home directory.
• Both of the architectural claims hold. I compiled an external module against github.com/kagent-dev/kmcp/pkg/cli and called cli.Root() successfully, so kmcp really could use cobra/doc. And kagent's root command really is in package main with six internal/ imports, so it really cannot. Your kagent init observation checks out too: --help genuinely prints Examples: twice, at lines 8 and 16.
• The $HOME scrub is a real catch, and I want to flag it because it is easy to skim past. Without your replacement, whoever ran the generator last would be published in the docs.
• Your read on the failing check is also right. Preview / deploy dies at wrangler versions upload because CLOUDFLARE_API_TOKEN is not exposed to fork PRs; the site build itself completed. And your drift findings hold up, with one small correction: --url-override was not removed, it is marked hidden at go/core/cli/cmd/kagent/main.go:129-130. Same documentation outcome, and it is actually a good argument for generation, since hidden flags drop out on their own.

Three blocking items, then a decision I would like your view on.

Biggest things to look at

1. The ../ links break in the Markdown export. They resolve correctly in the HTML because each page is served at a directory URL, but this site also publishes a .md version of every page, and that one is served at a file URL:

/docs/kagent/resources/cli/kagent-get.md  +  ../kagent-get-agent/
  -> /docs/kagent/resources/kagent-get-agent/     404

The link checker only scans public/**/*.html, which is why CI is green. Emitting root-relative links (/docs/kagent/resources/cli/kagent-get-agent/) fixes both output formats, matches the rest of the repo (144 root-relative links in content/ against 36 relative ones that all come from this PR), and lets you delete the long comment in render_page explaining the workaround. The link checker side is something I need to look at.

2. Five pages ship a truncated meta description. node.short = node.long.splitlines()[0] takes the first physical line of a hard-wrapped Long, so kagent-mcp.md gets description: MCP server management commands for creating and managing. Same on kagent-db-migrate, kagent-db-migrate-up, kagent-db-migrate-force, and kmcp-install.

The script already parses the correct string as child_short[name] from the parent's Available Commands: block, and I confirmed against the binaries that all five have a clean one available:

kagent mcp               -> MCP (Model Context Protocol) server management
kagent db migrate        -> Apply, roll back, and inspect database migrations
kagent db migrate up     -> Apply all pending migrations across every registered source
kagent db migrate force  -> Mark version V as applied without running its SQL
kmcp install             -> Install the KMCP controller on a Kubernetes cluster

All five are non-root commands, so child_short covers the whole set.

While you are in that function: the frontmatter is emitted as unquoted YAML built from --help text. All 61 pages parse today, but the nightly job regenerates from upstream main, so the first Cobra Short that gains a ": " breaks the Hugo build inside an auto-created PR. yaml.safe_dump removes that class of failure.

3. Ten pages lose their worked examples, because those commands have no Cobra Example field: kagent add-mcp, bug-report, dashboard, get, install, mcp, uninstall, version, and kmcp init, secrets. kagent get loses five worked examples, and kmcp init loses its FastMCP and MCP Go links along with them. For the most-read pages that is a step backwards for readers.

init, install, and secrets are the only kmcp commands without an Example field, and install never had a documented example anyway. So the kmcp regression is two commands, and both could be closed by adding an Example: to pkg/cli/internal/commands/init.go and secrets.go — small changes that can be done later.

Should fix

Generating docs is not side-effect free. Your reasoning about PreRun and RunE is correct as far as it goes, but kagent calls config.Init() at go/core/cli/cmd/kagent/main.go:58, which is before rootCmd.Execute() and therefore outside Cobra entirely. Cobra never gets the chance to intercept it. That function runs os.MkdirAll(~/.kagent, 0755) and viper.WriteConfigAs(), so every one of the 41 --help calls touches the filesystem. Running your generator created ~/.kagent/config.yaml on my machine, which had not existed a minute earlier.

In CI the practical impact is nil, since the runner's home is thrown away. Two reasons to fix it anyway:

• This is also the cause of the $HOME leak you patched. The default that gets printed is read back out of the file that config.Init() just wrote. Treating it at the source is tidier than scrubbing the symptom.
• "No side effects" is currently a property of what kagent's main() happens to do today, not of the approach. If someone later adds a cluster ping or a telemetry call before Execute(), the nightly docs job silently starts making it 41 times per run, and nobody will connect the two.

Pointing HOME at a throwaway directory for the duration fixes both at once. One wrinkle if you take that route: _HOME is computed at import time from the generator's own environment, so it must be derived from the same value you hand the child process, otherwise the scrub silently stops matching.

Worth correcting the claim in the PR description either way, since it is the sentence a reviewer leans on when deciding this is safe to run unattended.

ANSI color silently destroys parsing, and nothing catches it. This one is kmcp-only. kagent uses the stock Cobra template and emits no escapes at all, but kmcp's sectionHeader in pkg/cli/root.go wraps every section header in color. Under a pipe, fatih/color disables it, which is why your run and CI both work. Under a pty it does not, and I confirmed what comes out:

^[[38;2;148;45;231;1mAvailable Commands^[[0;22;0;0;0;22m:

Every header comparison in parse_help fails against that, so the entire help text falls into desc_lines. The result is a two-page site with an empty _index.md, exit code 0, no exception — and audit-cli-docs.py passes it, because the frontmatter parses, the fences balance, and the body is not empty. GitHub Actions uses a pipe today, so this does not fire, but it is a silent failure in an unattended nightly job and the cost of ruling it out is small:

• pass NO_COLOR=1 and TERM=dumb in the child environment, and strip ANSI escapes before parsing
• assert that root.children is non-empty and exit non-zero otherwise

That second assertion is the general guard, and it also covers the next item.

run_help ignores result.returncode. A subcommand whose --help fails produces an empty page and a green run. To be fair to the current state: I ran --help across all 41 kagent commands and all 18 kmcp ones, and every single one exits 0 with non-empty output, so this is latent rather than active. It is cheap insurance on a job that runs unattended against a moving upstream.

Wire audit-cli-docs.py into update-ref-docs.yaml, between generation and create-pull-request, with pip3 install pyyaml. Since that job opens PRs unattended, a checker nobody runs is not much of a net.

textwrap.dedent the example blocks. "\n".join(...).strip() only strips the first line, so 16 of 17 example blocks render with the first command flush and the rest indented two spaces.

#### Linux: and #### macOS: leak into real headings on the completion pages. The rendered "On this page" panel for kagent completion bash now lists Linux: and macOS:. Escaping or demoting any description line starting with # handles it.

setup-go pins 1.24. kmcp declares go 1.24.0 / toolchain go1.24.3, but kagent declares go 1.27.0. The toolchain will auto-download, so it should work, but since one setup step now has to serve two repos with independent Go versions and compile a full CLI, go-version: 'stable' is the safer pin.

The generator overwrites _index.md wholesale, including title, description, and intro sentence. It matched byte-for-byte this time, which is why the diff shows no deletions, but it means any future editorial change to that page gets silently reverted by the nightly job. This might be fine.

Two suggestions

Fix the kmcp template typos upstream and delete two parser special-cases. You worked around Available Commands:: and GlobalFlags: in the parser, which was the right call for getting this landed. The causes are pkg/cli/root.go:51, where a stray : sits outside the sectionHeader call, and line 60, which is missing a space. Two one-character fixes in a repo the same org owns, and then the parser does not need to know about them.

On "one approach for both": the uniformity argument is real and I am not asking you to change course. But it is worth naming that for kmcp the native path would sidestep every workaround in this script at once — cobra/doc reads the command struct fields directly rather than the rendered help, so the doubled colon, the missing space, the ANSI risk, and the truncated-description bug all stop existing, because .Short is available as a field. The costs are real too: two codepaths to maintain, page layouts that would drift apart between the two products, and one wrinkle I hit while testing — cli.Root() returns only 7 subcommands, because Cobra adds completion and help lazily during Execute(), so you would need InitDefaultCompletionCmd() first. Worth weighing rather than deciding on uniformity alone.

Related: the output is already not fully uniform. kmcp's custom template uses {{.UseLine}}, so its usage blocks are one line, while kagent's are two.

Two things for us, not you

• The sidebar flattens rather than nests: 15 entries to 41 for kagent, 9 to 20 for kmcp, with kagent get agent rendering as a sibling of kagent get. Your weights produce a sensible order, so it reads fine, but real nesting would need directory bundles and different URLs.
• Nine kmcp pages change URL and two -help pages disappear. Nothing inside content/ links to them, so nothing breaks internally, but this repo has no redirect mechanism at all.

On the kagent-mcp.md cross-links you flagged: yes please, once we settle the examples question, since it is the same mechanism.

Thank you for the time you are spending on this!

[kristin-kronstain-brown]

Left a comment.