Skip to content

FIX: Sync docs TOC with generated API pages + fail-fast TOC validation#1793

Open
romanlutz wants to merge 2 commits into
microsoft:mainfrom
romanlutz:romanlutz/fix-readthedocs-build
Open

FIX: Sync docs TOC with generated API pages + fail-fast TOC validation#1793
romanlutz wants to merge 2 commits into
microsoft:mainfrom
romanlutz:romanlutz/fix-readthedocs-build

Conversation

@romanlutz
Copy link
Copy Markdown
Contributor

@romanlutz romanlutz commented May 23, 2026

The Read the Docs build was failing (Success: False) because doc/myst.yml referenced api/pyrit_common_cli_helpers.md, which is never produced by build_scripts/gen_api_md.py. The generator only emits per-submodule pages when the parent module is a pure aggregate (only submodules, no direct API). pyrit.common has its own public API, so its cli_helpers submodule is not expanded into a separate page, causing the build log to emit hundreds of ⛔️ myst.yml Table of contents entry does not exist: api/pyrit_common_cli_helpers.md lines.

Two changes in this PR

1. Fix the immediate build break (doc/myst.yml)

  • Removed the bad entry for api/pyrit_common_cli_helpers.md (the helper functions in pyrit/common/cli_helpers.py are internal CLI plumbing and don't need their own API page).
  • Added the missing api/pyrit_output.md entry, which is generated by the script but was not listed in the TOC.

2. Prevent this class of regression

The reason this slipped past CI is that jupyter-book build --strict does not treat ⛔️ Table of contents entry does not exist as an error. PR #1753's deploy-book check passed (exit 0); every per-PR check was green. Only Read the Docs surfaced the failure because its build harness flagged the lack of generated output downstream.

This PR moves the check upstream into the generator itself:

  • build_scripts/validate_docs.py now auto-detects whether doc/api/ exists. Pre-commit (before any docs build) keeps skipping api/* like today; after gen_api_md.py runs, api/* entries are validated like any other TOC file, and orphan detection covers generated pages too.
  • build_scripts/gen_api_md.py invokes validate_docs.main() after writing the pages and propagates its non-zero exit code, so the Makefile, CI, and RTD all fail fast when doc/myst.yml drifts from the generator output in either direction (stale entry or missing entry).
  • tests/unit/build_scripts/test_validate_docs.py is extended to cover the new conditional behavior in both directions.

Verification

Verified locally with both directions of mismatch:

  1. Stale TOC entry (added a bogus api/pyrit_nonexistent.md to the TOC):

    [myst.yml] File referenced in myst.yml TOC not found: 'api/pyrit_nonexistent.md'
    === EXIT: 1 ===
    
  2. Missing TOC entry (removed api/pyrit_output.md from the TOC):

    [orphaned] File exists but not in myst.yml: api\pyrit_output.md
    === EXIT: 1 ===
    
  3. Happy path (good TOC): [OK] All documentation validations passed! → exit 0.

  4. Pre-commit invocation (no doc/api/ directory): still skips api/* entries — unchanged from today.

  5. Full unit test suite: 73 passed in 1.16s.

romanlutz and others added 2 commits May 23, 2026 14:02
The Read the Docs build was failing because doc/myst.yml referenced
api/pyrit_common_cli_helpers.md, which is never produced by
build_scripts/gen_api_md.py. The generator only emits per-submodule
pages when the parent module is a pure aggregate (only submodules,
no direct API). pyrit.common has its own public API, so its
cli_helpers submodule is not expanded into a separate page.

Also adds the missing api/pyrit_output.md entry, which is generated
by the script but was not listed in the TOC.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Read the Docs build for microsoft#1753 quietly failed because jupyter-book emits "Table of contents entry does not exist" as warnings rather than errors, and --strict does not promote them. The CI deploy-book job and every per-PR check passed; only RTD surfaced the failure.

Move that check upstream into the generator itself:

- validate_docs.py now auto-detects whether doc/api/ exists. Pre-commit (before any docs build) keeps skipping api/* like today; after gen_api_md.py runs, api/* entries are validated like any other TOC file and orphan detection covers generated pages too.

- gen_api_md.py invokes validate_docs.main() after writing the pages and propagates a non-zero exit code, so Makefile / CI / RTD all fail fast when doc/myst.yml drifts from the generator output in either direction (stale entry or missing entry).

- Extended test_validate_docs.py to cover the new conditional behavior in both directions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@romanlutz romanlutz changed the title FIX: Sync docs TOC with generated API pages FIX: Sync docs TOC with generated API pages + fail-fast TOC validation May 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant