Skip to content

docs: migration analysis skill improvements#5995

Merged
rise-erpelding merged 10 commits intomainfrom
rise-erpelding/component-analysis-prompt-improvements
Mar 31, 2026
Merged

docs: migration analysis skill improvements#5995
rise-erpelding merged 10 commits intomainfrom
rise-erpelding/component-analysis-prompt-improvements

Conversation

@rise-erpelding
Copy link
Copy Markdown
Collaborator

@rise-erpelding rise-erpelding commented Jan 30, 2026

Description

Improves the migration-analysis-prompt.md used for generating component migration documentation. Changes address recurring issues identified during human review of AI-generated docs, including branch confusion, missing diffs, verbose summaries, and incorrect status classifications.

Motivation and context

  • During review of generated migration docs (link, tag, taggroup, tooltip), several patterns of errors were identified:
    TagGroup incorrectly stated "no changes in Spectrum 2" due to branch verification failures
  • Diff sections were often prose descriptions instead of actual unified diffs
  • Summary sections included structure descriptions instead of focusing on changes
  • Features removed in Spectrum 2 were marked "Missing from CSS" instead of "Deprecated"
  • These changes make the prompt more explicit about requirements to reduce errors and improve output quality.

Related issue(s)

  • Related to ongoing 2nd-gen component analysis work

  • fixes [Issue Number]

Summary of changes

Commits are separated for easier review:

  1. Branch verification protocol - Requires proof of correct branch (git output, quoted imports, distinguishing features)
  2. Mandatory diff generation - Adds git diff command and makes diff section required
  3. HTML diff format - Provides realistic example showing actual structural changes
  4. Summary writing guidelines - Constrains summaries to change-focused bullets only
  5. CSS selector categorization - Requires organizing selectors by type (base, variants, states, etc.)
  6. Name equivalence mapping - Guides recognition of same concepts with different names
  7. --mod change tracking - Notes --mod changes as signals for new/deprecated features
  8. Deprecation detection criteria - Expands when to use "Deprecated" vs "Missing from CSS"
  9. Documentation goal statement - Frames output for engineers estimating migration work
  10. Quality gates checklist - Self-check before completion

Author's checklist

  • I have read the CONTRIBUTING and PULL_REQUESTS documents.
  • I have reviewed at the Accessibility Practices for this feature, see: Aria Practices
  • I have added automated tests to cover my changes.
  • I have included a well-written changeset if my change needs to be published.
  • I have included updated documentation if my change required it.

Reviewer's checklist

  • Includes a Github Issue with appropriate flag or Jira ticket number without a link
  • Includes thoughtfully written changeset if changes suggested include patch, minor, or major features
  • Automated tests cover all use cases and follow best practices for writing
  • Validated on all supported browsers
  • All VRTs are approved before the author can update Golden Hash

Manual review test cases

  • Descriptive Test Statement

    1. Go here
    2. Do this action
    3. Expect this result
  • Descriptive Test Statement

    1. Go here
    2. Do this action
    3. Expect this result

Device review

  • Did it pass in Desktop?
  • Did it pass in (emulated) Mobile?
  • Did it pass in (emulated) iPad?

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Jan 30, 2026

⚠️ No Changeset found

Latest commit: 5f7224e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jan 30, 2026

📚 Branch Preview Links

🔍 First Generation Visual Regression Test Results

When a visual regression test fails (or has previously failed while working on this branch), its results can be found in the following URLs:

Deployed to Azure Blob Storage: pr-5995

If the changes are expected, update the current_golden_images_cache hash in the circleci config to accept the new images. Instructions are included in that file.
If the changes are unexpected, you can investigate the cause of the differences and update the code accordingly.

@marissahuysentruyt marissahuysentruyt force-pushed the 2nd-gen-component-analysis branch from 7e7651f to 8b8fe5c Compare February 2, 2026 19:55
@rise-erpelding rise-erpelding force-pushed the rise-erpelding/component-analysis-prompt-improvements branch from 917c8d1 to 1bc5429 Compare February 4, 2026 19:13
@rise-erpelding rise-erpelding force-pushed the 2nd-gen-component-analysis branch 9 times, most recently from dfad5ac to 5dece21 Compare February 16, 2026 19:41
@rise-erpelding rise-erpelding force-pushed the rise-erpelding/component-analysis-prompt-improvements branch from 1bc5429 to 2d544f4 Compare February 16, 2026 23:44
@rise-erpelding rise-erpelding self-assigned this Feb 16, 2026
@rise-erpelding rise-erpelding force-pushed the 2nd-gen-component-analysis branch 3 times, most recently from 1ba9747 to 9f0701b Compare February 23, 2026 16:17
Base automatically changed from 2nd-gen-component-analysis to main February 23, 2026 19:09
@rise-erpelding rise-erpelding force-pushed the rise-erpelding/component-analysis-prompt-improvements branch from 4bf7e1d to e08a24b Compare March 30, 2026 18:38
@rise-erpelding rise-erpelding marked this pull request as ready for review March 30, 2026 18:50
@rise-erpelding rise-erpelding requested a review from a team as a code owner March 30, 2026 18:50
@rise-erpelding rise-erpelding changed the title docs: component analysis prompt improvements docs: migration analysis skill improvements Mar 30, 2026
@rise-erpelding rise-erpelding force-pushed the rise-erpelding/component-analysis-prompt-improvements branch 2 times, most recently from d2656f8 to 47eb0cc Compare March 30, 2026 19:47
@caseyisonit caseyisonit added the Status:Ready for merge PR has 2 approvals, all tests pass, and is ready to merge label Mar 30, 2026
@rise-erpelding rise-erpelding force-pushed the rise-erpelding/component-analysis-prompt-improvements branch from 47eb0cc to 0b2fbf6 Compare March 30, 2026 22:43
@rise-erpelding rise-erpelding force-pushed the rise-erpelding/component-analysis-prompt-improvements branch from 0b2fbf6 to 5f7224e Compare March 31, 2026 14:02
@rise-erpelding rise-erpelding merged commit f308b3e into main Mar 31, 2026
27 of 28 checks passed
@rise-erpelding rise-erpelding deleted the rise-erpelding/component-analysis-prompt-improvements branch March 31, 2026 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status:Ready for merge PR has 2 approvals, all tests pass, and is ready to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants