Skip to content

ci: syntax-check fenced code blocks on changed docs - #3267

Open
chaitanyas-maker wants to merge 1 commit into
LambdaTest:stagefrom
chaitanyas-maker:docs/ci-code-block-linter
Open

ci: syntax-check fenced code blocks on changed docs#3267
chaitanyas-maker wants to merge 1 commit into
LambdaTest:stagefrom
chaitanyas-maker:docs/ci-code-block-linter

Conversation

@chaitanyas-maker

Copy link
Copy Markdown
Contributor

The documentation audit found 118 code blocks that could not parse or compile. Every one would have been caught on the commit that introduced it by a check this simple.

This adds scripts/lint-code-blocks.js and a PR workflow that runs it over the docs changed by that PR.

What it checks

Using the real toolchains, not regexes:

javascript node --check python ast.parse
ruby ruby -c bash bash -n
json JSON.parse yaml yaml.safe_load_all

Plus two whole-file checks that both caught real defects in the audit:

  • Unbalanced code fences. An unclosed block silently inverts every block after it — prose renders as code and code as prose. This is live on smartui-cli-env-variables.md right now, where sections 5–8 render backwards.
  • Typographic quotes inside code blocks. ‘ ’ “ ” are a hard syntax error in every language. Currently live in 12 Cypress snippets.

Scoped to changed files, on purpose

The repo has pre-existing debt — a repo-wide gate would block every PR on unrelated pages. This stops new breakage landing while the backlog is worked down. node scripts/lint-code-blocks.js --all sweeps everything when you want the full picture.

False positives were the main design concern

A noisy linter trains reviewers to ignore it, so:

  • Blocks are dedented before parsing. Markdown nests blocks inside lists and <TabItem>s, so content arrives uniformly indented. Without this, every indented Python block looks like an IndentationError — I hit exactly this trap during the audit and reported false defects because of it.
  • Deliberate fragments are skipped — bare object literals like 'LT:Options': { ... }, key: { fragments, and elision markers.
  • JSON samples with // comments are stripped before parsing, so the house convention is tolerated while genuinely malformed JSON still fails.
  • Languages with no runtime on the runner are skipped, not failed.

Verified in both directions: clean pages report zero problems, and the known-bad pages still report their real defects.

$ node scripts/lint-code-blocks.js docs/smartui-ignore-colors.md
lint-code-blocks: 3 blocks checked, 0 problem(s)

$ node scripts/lint-code-blocks.js docs/smartui-cypress-sdk.md
✖ docs/smartui-cypress-sdk.md:561  [json]
  Expected property name or '}' in JSON at position 10
✖ docs/smartui-cypress-sdk.md  curly quotes inside code blocks at line(s) 267, 279, 291…

It found defects immediately

Running it against pages the audit had already been through surfaced residual issues the earlier passes missed, now fixed here:

  • npx smartui exec -P 5000 -- <command"> in 9 SDK pages — the stray quote makes it an unterminated shell string
  • 23 unbalanced set / export / $env: credential lines across 10 pages

It also swept the 25 pages the audit had never reached and found 28 more defects there, including set LT_USERNAME=YOUR_USERNAME" on the k6 page and // used as a shell comment across the git-branching pages. Those are filed separately.

Suggested follow-up

Once the open fix PRs land, consider flipping to --all in a non-blocking scheduled job to track the remaining backlog.

An audit of the SmartUI doc set found 118 code blocks that could not parse or
compile. Every one would have been caught on the commit that introduced it by
a check this simple.

Adds scripts/lint-code-blocks.js and a PR workflow that runs it over the docs
changed by that PR.

What it checks, using the real toolchains:
  javascript  node --check          python  ast.parse
  ruby        ruby -c               bash    bash -n
  json        JSON.parse            yaml    yaml.safe_load_all
plus two whole-file checks that caught real defects in the audit:
  - unbalanced code fences (an unclosed block silently inverts every block
    after it, so prose renders as code and code as prose)
  - typographic quotes inside code blocks, which are a hard syntax error in
    every language

Deliberately scoped to files changed by the PR. The repo has pre-existing debt
and a repo-wide gate would block every PR on unrelated pages; this stops new
breakage landing while the backlog is worked down. `--all` sweeps everything.

Care taken to avoid false positives, which would train reviewers to ignore it:
  - block content is dedented before parsing, because Markdown nests blocks
    inside lists and TabItems and every indented Python block would otherwise
    look like an IndentationError
  - deliberate partial snippets (bare object literals, `key: {` fragments,
    elision markers) are skipped
  - JSON samples carrying // comments are stripped before parsing, so the
    house convention is tolerated while genuinely malformed JSON still fails
  - languages with no runtime on the runner are skipped, not failed
Verified both directions: clean pages report zero, and the known-bad pages
still report their real defects.

Also fixes the defects this linter found immediately on already-audited pages,
which the earlier passes had missed:
  - `npx smartui exec -P 5000 -- <command">` in 9 SDK pages (stray quote makes
    it an unterminated shell string)
  - 23 unbalanced `set`/`export`/`$env:` credential lines across 10 pages

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0197inyHsQ3V3CxPicLLvFJy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant