Skip to content

fix: parse README pin and changelog as markdown - #1482

Open
John-David Dalton (jdalton) wants to merge 2 commits into
mainfrom
test/main-ast-markdown
Open

fix: parse README pin and changelog as markdown#1482
John-David Dalton (jdalton) wants to merge 2 commits into
mainfrom
test/main-ast-markdown

Conversation

@jdalton

@jdalton John-David Dalton (jdalton) commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-fable-5

Port of the v1.x AST-markdown rework (landed there as 6e5adcb) to main's fleet shape: markdown structure questions are answered from a position-tracked GFM mdast parse instead of line scans and string patterns, so content inside code fences can never read as structure.

What ported where (main's surfaces differ from v1.x):

v1.x surface main surface Change
.github/scripts/pin-readme-assets.mjs (standalone script) scripts/fleet/publish-infra/pin-readme.mts pinReadmeAssets() (pure function inside the pin/restore bracket) Rewrite ported: mdast image/link/definition urls + parse5-located src/srcset values, edits applied at parser-reported byte offsets, no serializer round-trip. Signature unchanged.
scripts/release/changelog.mts scripts/fleet/lib/changelog-render.mts unreleasedRange() + scripts/fleet/lib/changelog.mts sectionHasEntries() and mergeUnreleased()'s insertion scan Same port: ## boundaries from heading node positions, entries from real listItem nodes. Exported APIs unchanged.
(new) scripts/fleet/_shared/markdown-ast.mts One shared GFM parse helper (parseMarkdownGfm, h2LineIndexes, hasListItem) so the pin and the changelog flows read the same tree.

Behavior fixed on main by the port, each with a test: assets/ inside fenced code blocks or inline code spans stays as written; srcset attributes are now pinned (the string version only handled src=/](); reference-style definitions are now pinned; a ## line inside a code fence can no longer truncate a promoted [Unreleased] block or misplace the merge insertion point; a - lookalike in fenced shell output no longer satisfies the empty-changelog guard. Differential receipt: with the port reverted (tests kept), 10 of the 18 new tests fail; with the port, 18/18 pass.

Dependencies: mdast-util-gfm@3.1.0, micromark-extension-gfm@3.0.0, and parse5@8.0.1 added to the pnpm catalog + root devDependencies (mdast-util-from-markdown, micromark, and @types/mdast were already in the catalog).

Cascade note: scripts/fleet/** is managed by the shared fleet template. This lands the fix downstream to stop the bleeding; the same change needs a matching upstream template pass (including parseSectionBullets, which keeps a line loop here since it only sees generated sections) or the next cascade reverts it.

Tests: 13 in test/fleet/pin-readme.test.mts, 5 in test/fleet/changelog-markdown-structure.test.mts.

Suite: pnpm test --all locally — 7552 passed, 425 skipped, 8 failed. All 8 failures are cmd-optimize integration snapshot mismatches (packages/cli/test/integration/cli/cmd-optimize.test.mts, cmd-optimize-output-and-paths.test.mts) that fail identically on an untouched origin/main checkout (verified by stashing this branch's changes and rerunning both files) — pre-existing local-environment snapshot drift, not from this diff. pnpm run check passes on the modified scope except the pre-existing dispatch-table-is-current failure, which also fails on untouched origin/main.


Note

Medium Risk
Touches release changelog promotion/merge and publish-time README rewriting; behavior changes are intentional but could affect edge-case CHANGELOGs or READMEs that relied on the old heuristics.

Overview
Replaces line scans and regex with a shared GFM mdast parse (scripts/fleet/_shared/markdown-ast.mts) so markdown inside code fences is never treated as structure.

Publish README pin (pinReadmeAssets): relative assets/… refs are rewritten at parser byte offsets (markdown image/link/definition plus parse5-located src/srcset), with no serializer round-trip. Adds coverage for srcset, reference definitions, and skipping lookalikes in fences or inline code.

Changelog flows: [Unreleased] section bounds, merge insertion, and sectionHasEntries use real ## headings and listItem nodes instead of startsWith('## ') / /^\s*-\s/ patterns.

Adds catalog deps mdast-util-gfm, micromark-extension-gfm, and parse5, plus fleet tests for pin and changelog structure.

Reviewed by Cursor Bugbot for commit 227a305. Configure here.

pinReadmeAssets matched raw markdown with string patterns
(src="assets/ and ](assets/), so an assets/ ref inside a fenced code
block or inline code span got rewritten even though registries render
it as literal text, srcset attributes were never pinned at all, and
reference-style definitions ([ref]: assets/…) were missed.

The README now parses to a position-tracked GFM mdast tree
(mdast-util-from-markdown was already in the catalog; the GFM pair is
added so tables and footnotes parse the way GitHub renders them) and
every edit lands on a parser-reported byte offset. Raw HTML arrives as
mdast html nodes whose source slices go through parse5 with source
locations on, so only real src/srcset attribute values are touched.
No serializer round-trip — untouched bytes stay byte-identical. The
exported signature and the pin/restore bracket are unchanged.

The shared parse helper lives in _shared/markdown-ast.mts so the
changelog flows read the same tree. Ported from the v1.x rework
(6e5adcb) to this branch's publish-infra shape; needs the
matching pass in the shared fleet template or the next cascade
reverts it.
The changelog flows located the [Unreleased] block, the next release
heading, and the merge insertion point by scanning raw lines for
'## ', and counted entries with a '- ' pattern. A '## ' or '- ' line
inside a fenced code block read as structure: a promote could truncate
the Unreleased block at a fence line, and a bullet lookalike in fenced
shell output satisfied the empty-changelog guard.

Structure now comes from the same GFM mdast parse the README pin uses:
level-2 headings from parser positions, entries from real listItem
nodes. The exported APIs and output for well-formed changelogs are
unchanged; the new tests fail on the line-scan version and pass here.
Ported from the v1.x rework (6e5adcb); parseSectionBullets keeps
its line loop (generated-section inputs only) and is noted for the
upstream template pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant