Skip to content

Add SPFxScaffoldLog structured event log#206

Open
nick-pape wants to merge 16 commits intoSharePoint:mainfrom
nick-pape:nick-pape/scaffold-event-log
Open

Add SPFxScaffoldLog structured event log#206
nick-pape wants to merge 16 commits intoSharePoint:mainfrom
nick-pape:nick-pape/scaffold-event-log

Conversation

@nick-pape
Copy link
Copy Markdown
Contributor

@nick-pape nick-pape commented Mar 27, 2026

Description

Introduces a typed, JSONL-serializable event log (SPFxScaffoldLog) that records key decisions and outcomes during scaffolding. This is the foundation for the audit log file described in #100.

New logging module in @microsoft/spfx-template-api:

  • SPFxScaffoldEvent — discriminated union of four event kinds: template-rendered, package-manager-selected, file-write, package-manager-install-completed
  • SPFxScaffoldLog — append-only log class with getEventsOfKind() filtering and toJsonl()/fromJsonl() serialization

SPFxTemplateWriter integration:

  • writeAsync() now accepts an optional { log } parameter
  • Emits a file-write event per non-deleted file with non-null contents, recording outcome (new / merged / preserved / unchanged)
  • Fully backward-compatible — no behavior change when log is omitted

CLI integration (CreateAction):

  • Creates a SPFxScaffoldLog at the start of scaffolding
  • Appends template-rendered (with full render context), package-manager-selected, and package-manager-install-completed events
  • Passes log to the writer for file-level event recording

The log is created and populated in-memory but not yet persisted to disk — that will be handled by #100.

How was this tested?

  • 15 new unit tests for SPFxScaffoldLog (append/read, filtering, JSONL round-trips, timestamp auto-fill, edge cases)
  • 7 new integration tests for SPFxTemplateWriter log events (one per file outcome + multi-file + backward-compat + skip deleted)
  • All 391 tests pass (281 API + 91 CLI + 19 template), clean rush build

Type of change

  • New feature / enhancement

Introduces a typed, JSONL-serializable event log that records key
decisions and outcomes during scaffolding. This is the foundation for
the audit log file described in SharePoint#100.

New logging module in spfx-template-api:
- SPFxScaffoldEvent discriminated union (template-rendered,
  package-manager-selected, file-write, package-manager-install-completed)
- SPFxScaffoldLog class with append, getEventsByKind, toJsonl/fromJsonl

SPFxTemplateWriter now accepts an optional log and emits a file-write
event per file with outcome (new/merged/preserved/unchanged).

CreateAction creates a log and appends template-rendered,
package-manager-selected, and install-completed events.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 27, 2026 16:23
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new structured scaffolding event log to @microsoft/spfx-template-api and wires it into the CLI scaffolding flow to record template render decisions, per-file write outcomes, and package manager actions (as groundwork for the audit log in #100).

Changes:

  • Introduces logging module with SPFxScaffoldEvent (discriminated union) and SPFxScaffoldLog (append/filter + JSONL serialize/deserialize).
  • Extends SPFxTemplateWriter.writeAsync() with optional { log } to emit file-write events (new/merged/preserved/unchanged).
  • Integrates log creation + event emission into spfx create (template-rendered, package-manager-selected, package-manager-install-completed).

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
common/changes/@microsoft/spfx-cli/scaffold-log_2026-03-27.json Adds Rush change file for the lockstep policy.
apps/spfx-cli/src/cli/actions/CreateAction.ts Creates/uses SPFxScaffoldLog, emits scaffold events, and threads log into the writer.
api/spfx-template-api/src/writing/SPFxTemplateWriter.ts Adds IWriteOptions + optional log emission for per-file outcomes.
api/spfx-template-api/src/writing/index.ts Re-exports IWriteOptions from the writing barrel.
api/spfx-template-api/src/writing/test/SPFxTemplateWriter.test.ts Adds integration tests validating file-write events across outcomes and backward compatibility.
api/spfx-template-api/src/logging/SPFxScaffoldEvent.ts Defines the scaffold event union and related event interfaces/types.
api/spfx-template-api/src/logging/SPFxScaffoldLog.ts Implements append-only in-memory log + JSONL (de)serialization and kind filtering.
api/spfx-template-api/src/logging/index.ts Adds logging barrel exports.
api/spfx-template-api/src/logging/test/SPFxScaffoldLog.test.ts Adds unit tests for append/filtering/JSONL round-trips and timestamp autofill.
api/spfx-template-api/src/index.ts Exports new logging types and SPFxScaffoldLog from the package root.
api/spfx-template-api/etc/spfx-template-api.api.md Updates API extractor output for the new public types and updated writer signature.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@nick-pape
Copy link
Copy Markdown
Contributor Author

Manual Test Report

Environment

  • Node.js 22.16.0 (LTS)
  • Rush 5.172.0 / pnpm 10.15.1
  • Branch: nick-pape/scaffold-event-log

1. Automated Tests

Package Tests Result
@microsoft/spfx-template-api 240 (15 new) ✅ All pass
@microsoft/spfx-cli 88 ✅ All pass
Total 328

Full rush build — clean, no warnings.

2. SPFxScaffoldLog Unit Behavior (programmatic)

Scenario Result
Append 4 events, read back in order
getEventsByKind('file-write') filters correctly
getEventsByKind('template-rendered') filters correctly
Empty-string timestamps auto-filled with valid ISO 8601
toJsonl()fromJsonl() round-trip: events match

3. Writer + Log Integration — Fresh Scaffold

Scaffolded webpart-minimal into empty directory via API:

  • 23 file-write events recorded
  • All outcomes: new (23)
  • ✅ Every rendered file tracked

4. Writer + Log Integration — Merge Scenario (Add Component)

Scaffolded extension-application-customizer into existing webpart project:

  • 22 file-write events recorded
  • Outcome breakdown:
    • preserved: 8 (files with no merge helper, kept existing)
    • merged: 4 (package.json, config/config.json, config/package-solution.json, config/serve.json)
    • unchanged: 4 (identical content, e.g. tsconfig.json)
    • new: 6 (extension source files + SharePoint assets)
  • mergeHelper field populated on all merged events
  • mergeHelper field absent on non-merged events

5. CLI End-to-End

Command Result
spfx create --template webpart-minimal --local-source ... (fresh) ✅ 23 files written
spfx create --template extension-application-customizer --local-source ... (into existing) ✅ Merges applied, new extension files added

Both runs completed without errors. The log is created and populated in-memory (not persisted to disk yet — that's #100).

6. Backward Compatibility

  • writeAsync(editor, targetDir) without log parameter: ✅ No errors, identical behavior to before
  • All 15 pre-existing SPFxTemplateWriter tests pass unchanged
  • All 53 CreateAction tests pass unchanged (including snapshot tests)

- Return defensive copy from SPFxScaffoldLog.events getter
- Use append() in fromJsonl() so timestamps are normalized
- Narrow readFile catch to ENOENT only, rethrow other errors
- Record package-manager-selected unconditionally (including 'none')
- Restructure install flow so completion event is logged before throw
- Add test for non-ENOENT readFile error propagation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
nick-pape and others added 2 commits March 27, 2026 16:33
…ent-log

# Conflicts:
#	api/spfx-template-api/etc/spfx-template-api.api.md
#	api/spfx-template-api/src/index.ts
#	apps/spfx-cli/src/cli/actions/CreateAction.ts
- Rename getEventsByKind → getEventsOfKind
- Use @remarks JSDoc tag for mergeHelper field
- Add signal field to IPackageManagerInstallCompletedEvent
- Make timestamp optional on append() via SPFxScaffoldEventInput type
- Doc comment: "defensive shallow copy"
- Optimize fromJsonl to single-pass character scanning
- Add snapshot assertions to writer log integration tests
- Extract _logFileWrite to loose function, use log?.append()
- Simplify mergeHelper serialization (JSON.stringify drops undefined)
- Remove redundant timestamp: '' from CLI event appends

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 27, 2026 21:40
@nick-pape
Copy link
Copy Markdown
Contributor Author

Manual Test Report (v2 — post review fixes)

Re-run after addressing all review feedback (Copilot + iclanton) and merging from main (CasedString PR #194, cwd mock PR #204, CRLF PR #208).

Environment

  • Node.js 22.16.0 (LTS)
  • Rush 5.172.0 / pnpm 10.15.1
  • Branch: nick-pape/scaffold-event-log @ 30f0823

1. Automated Tests

Package Tests Result
@microsoft/spfx-template-api 267 (15 log + 8 writer log + 5 snapshots) All pass
@microsoft/spfx-cli 88 All pass
Total 355 All pass

rush build — clean, no warnings.

2. SPFxScaffoldLog Unit Behavior (26 programmatic assertions)

Scenario Result
Append 4 events, read back in order Pass
getEventsOfKind('file-write') filters correctly Pass
getEventsOfKind('template-rendered') filters correctly Pass
Timestamps auto-filled with valid ISO 8601 when omitted Pass
Explicit timestamp preserved when provided Pass
events getter returns defensive shallow copy (different refs) Pass
toJsonl() produces correct line count Pass
fromJsonl() round-trip matches Pass
fromJsonl("") returns empty log Pass
fromJsonl handles trailing newlines Pass

3. Writer + Log — Fresh Scaffold

Scaffolded webpart-minimal into empty directory:

  • 23 file-write events — all new
  • No merged/preserved/unchanged (correct for fresh project)

4. Writer + Log — Merge Scenario

Scaffolded extension-application-customizer into existing project:

  • 22 file-write events
  • merged: 4 (package.json, config/config.json, config/serve.json, config/package-solution.json)
  • new: 6 (extension source + SharePoint assets)
  • preserved: 6 (no merge helper, kept existing)
  • unchanged: 6 (identical content)
  • All merged events have mergeHelper field; non-merged events do not

5. Backward Compatibility

  • writeAsync(editor, targetDir) without log: completes without error

6. CLI End-to-End

spfx create --template webpart-minimal --local-source ... — 10 top-level entries scaffolded, package.json and tsconfig.json present.

7. Review Changes Verified

  • getEventsOfKind (renamed from getEventsByKind)
  • SPFxScaffoldEventInput type (timestamp optional via distributive Omit)
  • signal field on IPackageManagerInstallCompletedEvent
  • _logFileWrite is a loose function using log?.append()
  • Single-pass fromJsonl parser
  • Snapshot assertions on all writer log tests

@nick-pape nick-pape requested a review from iclanton March 27, 2026 21:50
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Inline distributive Omit to avoid _DistributiveOmit leaking into API
- Handle \r\n line endings in fromJsonl for cross-platform support
- Only include mergeHelper field on file-write events when defined

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@nick-pape nick-pape linked an issue Mar 30, 2026 that may be closed by this pull request
6 tasks
The _logFileWrite fix correctly omits mergeHelper when undefined,
so the snapshots no longer contain "mergeHelper": undefined entries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 30, 2026 18:23
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ent-log

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@nick-pape nick-pape enabled auto-merge (squash) March 30, 2026 19:45
Enable resolveJsonModule in spfx-cli tsconfig and replace the require()
call with a typed import, as suggested by iclanton.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 30, 2026 21:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

nick-pape and others added 2 commits March 30, 2026 17:14
- fromJsonl: use indexOf('\n') instead of character-by-character scanning
- _logFileWrite: simplify to just log?.append({...}) — no elaborate inline type
- SPFxTemplateWriter: use FileSystem.isNotExistError instead of manual ENOENT check
- IWriteOptions.log doc: clarify it covers non-deleted entries with non-null contents
- CreateAction: fix CRLF line endings to LF
- CreateAction: use IWaitForExitResultWithoutOutput from Executable.waitForExitAsync
- CreateAction: move log append + error handling into _runInstallAsync
- CreateAction: move package-manager-selected event before writeAsync
- resolveJsonModule: move from spfx-cli tsconfig to the rig

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…r package.json

The resolveJsonModule setting was added to the shared rig tsconfig in a previous
review round, but it should be handled by a separate PR. Revert CLI_VERSION back
to require() since the rig no longer enables resolveJsonModule.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 31, 2026 16:35
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

nick-pape and others added 3 commits March 31, 2026 13:46
…ent-log

# Conflicts:
#	apps/spfx-cli/src/cli/actions/CreateAction.ts
- Switch require() to import for package.json in CreateAction (iclanton)
- Replace delete with destructuring in snapshotLog test helper (iclanton)
- Remove unnecessary spread in SPFxScaffoldLog.append, mutate timestamp directly (iclanton)
- Fix IFileWriteEvent doc to clarify deleted/null entries are skipped (copilot)
- Fix enoent() test helper to include errno/syscall for FileSystem.isNotExistError compat
- Update snapshots

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ent-log

Integrates PR SharePoint#207 (Remove mem-fs-editor) with scaffold event log.
Adapts logging to the new TemplateOutput-based SPFxTemplateWriter API.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 31, 2026 21:45
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Resolve merge conflicts from SharePoint#212 (CRLF line endings fix)
- Fix append() to shallow-clone events instead of mutating caller's object
- Normalize exitCode consistently in _runInstallAsync using a single variable
- Clarify FileWriteOutcome 'new' docs to cover binary file overwrite case

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@nick-pape
Copy link
Copy Markdown
Contributor Author

Manual Test Report — Post-merge with PR #207 (Remove mem-fs-editor)

Branch: nick-pape/scaffold-event-log
Date: 2026-03-31


Test 1: Unit Test Suites

Suite Tests Result
@microsoft/spfx-template-api 292 PASSED
@microsoft/spfx-cli 91 PASSED
@microsoft/spfx-template-test 19 PASSED

Test 2: End-to-End Scaffolding

Ran the CLI with --local-source and --template webpart-react. 17 templates discovered, 27 files scaffolded. Log events are created in-memory for template-rendered, package-manager-selected, and file-write events.

Result: PASSED

Test 3: SPFxScaffoldLog API

Operation Result
Create log and append 5 events Events count: 5
getEventsOfKind('file-write') Returns 2 file-write events
toJsonl() round-trip via fromJsonl() Round-trip match: true
All timestamps auto-populated true
mergeHelper field preserved through serialization true

Result: PASSED

Test 4: append() Input Immutability

Verified that append() does NOT mutate the caller's input object — timestamp is added only to the stored copy.

Result: PASSED


Overall: All 6 test scenarios PASSED. Logging integrates correctly with the new TemplateOutput-based writer from PR #207.

Copilot AI review requested due to automatic review settings March 31, 2026 23:24
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +124 to 128
if (existingBuffer === undefined) {
contentToWrite = contents;
_logFileWrite(log, relativePath, 'new');
} else if (!existingBuffer.equals(contents)) {
contentToWrite = contents;
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_logFileWrite() is called before writeFileAsync() executes. If the subsequent write fails (e.g., EACCES/disk full), the log will still record outcomes like new/merged even though the file was not actually written, which makes the audit trail misleading. Consider appending file-write events only after a successful write, and/or introducing a failure outcome/event when writeFileAsync() throws.

Copilot uses AI. Check for mistakes.
Comment on lines +24 to +25
* When provided, a `file-write` event is appended for each non-deleted file with
* non-null contents processed during the write phase.
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The IWriteOptions.log doc refers to “non-deleted” files and “non-null contents”, but TemplateOutput/ITemplateOutputEntry don’t represent deletions and contents can’t be null. This wording is misleading for API consumers; consider rephrasing to simply describe that a file-write event is recorded for each file in templateOutput.files that is evaluated for writing/merge.

Suggested change
* When provided, a `file-write` event is appended for each non-deleted file with
* non-null contents processed during the write phase.
* When provided, a `file-write` event is appended for each file in the template
* output that is evaluated for writing or merging during the write phase.

Copilot uses AI. Check for mistakes.
export interface IPackageManagerSelectedEvent extends ISPFxScaffoldEventBase {
kind: 'package-manager-selected';
packageManager: string;
targetDir: string;
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IPackageManagerSelectedEvent.targetDir is currently an arbitrary string and in CLI usage it’s an absolute path. If this log is later persisted/committed (per #100’s audit-log goals), absolute paths can leak PII (e.g., usernames/home directories) and reduce portability. Consider storing a path relative to the project root (or just the folder name), or omitting/redacting this field from the public event schema.

Suggested change
targetDir: string;
/**
* Target directory for the package manager operation, expressed as a path
* relative to the project root (for example, ".",
* "src", or "packages/my-project").
*
* Absolute paths MUST NOT be used here to avoid leaking PII (such as user
* home directories) into persisted logs and to improve portability.
*/
targetDirRelativePath: string;

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We aren't sending these logs anywhere, so PII isn't a concern.


// ---- getEventsOfKind ---------------------------------------------------

describe('getEventsOfKind', () => {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
describe('getEventsOfKind', () => {
describe(SPFxScaffoldLog.prototype.getEventsOfKind.name, () => {


// ---- JSONL serialization -----------------------------------------------

describe('toJsonl', () => {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
describe('toJsonl', () => {
describe(SPFxScaffoldLog.prototype.toJsonl.name, () => {

});
});

describe('fromJsonl', () => {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
describe('fromJsonl', () => {
describe(SPFxScaffoldLog.fromJsonl.name, () => {

export interface IPackageManagerSelectedEvent extends ISPFxScaffoldEventBase {
kind: 'package-manager-selected';
packageManager: string;
targetDir: string;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We aren't sending these logs anywhere, so PII isn't a concern.

*
* @public
*/
export type SPFxScaffoldEvent =
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
export type SPFxScaffoldEvent =
export type ISPFxScaffoldEvent =

*
* @public
*/
export type SPFxScaffoldEventInput = SPFxScaffoldEvent extends infer E
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
export type SPFxScaffoldEventInput = SPFxScaffoldEvent extends infer E
export type ISPFxScaffoldEventInput = SPFxScaffoldEvent extends infer E

const normalizedEvent: SPFxScaffoldEvent = {
...event,
timestamp: event.timestamp || new Date().toISOString()
} as SPFxScaffoldEvent;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this cast? Shouldn't adding timestamp make the typechecker happy?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement SPFxCreationAuditLog

3 participants