Skip to content

feat(verify): add consolidated end-of-run summary report#35

Merged
Bccorb merged 1 commit into
mainfrom
feat/verify-consolidated-report
Jul 8, 2026
Merged

feat(verify): add consolidated end-of-run summary report#35
Bccorb merged 1 commit into
mainfrom
feat/verify-consolidated-report

Conversation

@Bccorb

@Bccorb Bccorb commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What

seamless verify now prints a single consolidated summary report at the very end of a run, so you no longer have to scroll back through phase output to see how each layer did.

The report includes:

  • Packages under test: source versions for --local runs (@seamless-auth/core, /express, /react), or the declared pins for released runs (@seamless-auth/express from the adapter-app, @seamless-auth/react from each web template), plus the source seamless-auth-api version in both modes.
  • One line per conformance layer (API / adapter, and each web template) with its pass/fail status and duration.
  • An overall verdict.

It is printed after teardown so it stays on screen. The existing live per-phase output (build logs, Playwright grids) is unchanged and still streams as the run progresses.

Why

The only end-of-run output before was a single pass/fail line, and teardown noise printed after it, so the result got pushed off screen on any real run. This gives a stable, scannable recap in one place.

Implementation notes

  • Each layer now runs through a runLayer() helper that times it and records the outcome; the shared failed flag is unchanged.
  • Version collection is fully best-effort (guarded reads); a missing manifest just drops that line and can never fail a run.
  • Setup failures (for example Docker missing) surface in the same report as Verify aborted: <message>.

Testing

  • npm run build (tsc) passes.
  • Ran the full default seamless verify end to end: exit 0, all three layers pass (API / adapter, react-vite all flows, react-oauth scoped to @oauth).
────────────────────────────────────────────────────────
  Seamless Verify — summary
  SDK: released (npm)  ·  1m 31s total
────────────────────────────────────────────────────────
  Packages
    seamless-auth-api       0.2.3
    @seamless-auth/express  ^0.6.0
    @seamless-auth/react    ^0.4.0
────────────────────────────────────────────────────────
  ✔  API / adapter                 6.5s
  ✔  Web · react-vite (all flows)  7.6s
  ✔  Web · react-oauth (@oauth)    1.7s
────────────────────────────────────────────────────────
  ✔ Conformance passed — 3 layer(s).
────────────────────────────────────────────────────────

Collect each conformance layer's pass/fail and duration as it runs and print
a single report after teardown, so results stay on screen without scrolling
back through phase output. The report also lists the seamless package versions
under test (source versions for --local, declared pins for released runs) and an
overall verdict.
@Bccorb Bccorb merged commit ff94bb9 into main Jul 8, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant