Skip to content

Switch local SIL dependency development to local packages#762

Open
johnml1135 wants to merge 3 commits intomainfrom
liblcm_debug
Open

Switch local SIL dependency development to local packages#762
johnml1135 wants to merge 3 commits intomainfrom
liblcm_debug

Conversation

@johnml1135
Copy link
Contributor

@johnml1135 johnml1135 commented Mar 13, 2026 •
edited
Loading

Summary

This PR removes the old nested Localizations/LCM source workflow and replaces it with a package-only local dependency workflow for libpalaso, liblcm, and chorus.

The supported inner loop now:

  • packs selected local dependency repos into Output/LocalNuGetFeed
  • writes temporary version overrides to Build/SilVersions.Local.props
  • restores FieldWorks against those locally packed packages
  • reuses previously packed local artifacts when the dependency repo state has not changed
  • gives VS Code a guarded portable-PDB prelaunch path that understands both pinned-package mode and the full local-package shortcut

What Changed

  • removed the FieldWorks.LocalLcm.sln / UseLocalLcmSource path
  • added -LocalPalaso, -LocalLcm, and -LocalChorus switches to build.ps1 and test.ps1
  • added repo-local package restore behavior so the extracted package cache stays under packages/
  • added Build/Agent/Pack-LocalDependencies.ps1 to:
    • pack libpalaso first, then liblcm and chorus in parallel
    • write and read per-dependency reuse stamps in Output/LocalNuGetFeed/.stamp
    • fingerprint dependency repos from git state so unchanged local packages can be reused instead of repacked
    • tolerate benign local-repo CRLF safety warnings during fingerprinting
    • sync the selected local package contents back into the repo-local packages/ cache
  • updated version/property flow so the local-package path can override the SIL dependency versions cleanly
  • updated build.ps1 debug stamps to record:
    • the requested local dependency set
    • the local package version
    • local dependency fingerprint state
    • managed debug symbol mode
    • relevant workspace pathspecs and worktree status for VS Code prelaunch reuse checks
  • expanded Build/Agent/Invoke-VsCodeDebugBuild.ps1 so the VS Code prelaunch helper now:
    • understands package mode versus full local-package mode
    • forwards local dependency switches and -LocalPackageVersion into build.ps1
    • rebuilds when selected local dependency repo fingerprints changed
    • rebuilds when the FieldWorks launch binary or matching PDB were rewritten after the last stamped debug build
    • skips the prelaunch build only when the stamp, relevant git state, selected dependency state, and launch outputs still match
  • updated .vscode/tasks.json so Prepare Debug (Local Packages) goes through the helper instead of always rebuilding
  • updated .vscode/launch.json to:
    • keep the supported clr-based .NET Framework launchers explicit
    • add diagnostics launch variants for package and local-package sessions
    • enable NuGet.org symbol lookup without narrowing symbol loading to a brittle module allowlist
    • remove the misleading old dotnet test launch config for this net48 repo
  • updated Docs/architecture/liblcm-debugging.md to document:
    • Visual Studio as the primary debugger for mixed managed/native scenarios
    • VS Code as the managed-only best-effort path
    • the full-local-stack assumption behind the built-in VS Code Local Packages launchers
    • the prelaunch helper's rebuild and skip rules

Reuse Behavior

Unchanged local dependency artifacts are reused when all of the following still match:

  • dependency repo path
  • build configuration
  • requested local package version
  • git-state fingerprint for the dependency repo
  • expected .nupkg artifacts still exist in Output/LocalNuGetFeed

The git fingerprint uses:

  • clean:<HEAD> for clean repos
  • dirty:<sha256(...)> for dirty repos, based on status, diff, and untracked file metadata/content

VS Code Debug Behavior

The VS Code Prepare Debug (*) path now skips work only when it can prove the last successful portable-PDB debug build still matches the requested session:

  • same local dependency mode and local package version
  • same relevant workspace git state for the tracked debug inputs
  • same local dependency fingerprints for the selected local repos
  • same stamped FieldWorks.exe and matching FieldWorks.pdb

If those checks fail, the helper rebuilds before launch.

Validation

Completed locally with the repo-standard entrypoints and targeted debug-helper validation:

  • ./build.ps1 -LocalPalaso -LocalLcm -LocalChorus
  • repeated the same build and confirmed reuse of Palaso/Lcm/Chorus packages instead of repacking
  • ./build.ps1 -BuildTests -LocalPalaso -LocalLcm -LocalChorus
  • ./test.ps1 -NoBuild -LocalPalaso -LocalLcm -LocalChorus
  • ./Build/Agent/Invoke-VsCodeDebugBuild.ps1 -ManagedDebugType portable
  • repeated the same package-mode helper run and confirmed skip behavior
  • touched Output/Debug/FieldWorks.pdb, reran the package-mode helper, and confirmed it rebuilt before launch, then skipped on the next run
  • ./Build/Agent/Invoke-VsCodeDebugBuild.ps1 -LocalPalaso -LocalLcm -LocalChorus -ManagedDebugType portable
  • repeated the same full local-package helper run and confirmed skip behavior
  • CI: Full local check
  • git diff --check -- .vscode/launch.json .vscode/tasks.json Build/Agent/Invoke-VsCodeDebugBuild.ps1 Build/Agent/Pack-LocalDependencies.ps1 Docs/architecture/liblcm-debugging.md build.ps1

Notes:

  • the first test.ps1 attempt was blocked by a stale VS Code PowerShell host holding output test DLLs open; rerunning after clearing that stale shell made the -BuildTests path succeed
  • the no-build managed test run completed but still reports these existing test failures, which do not appear to be caused by the package-workflow changes:
    • BackTranslationNonInterleaved_MissingPicture
    • BackTranslationNonInterleaved_DefaultParaCharsStart
    • ApplyWS_MultipleWritingSystems
    • MigrateFrom83Alpha_MoveStemToLexeme (timeout)
  • CI: Full local check ran, but the repo whitespace task printed a noisy git diff usage block in this worktree state; the changed files themselves passed a direct git diff --check whitespace check

Review Focus

Please focus review on:

  • the new package-only local dependency control surface in build.ps1 / test.ps1
  • the reuse and invalidation logic in Build/Agent/Pack-LocalDependencies.ps1
  • the VS Code prelaunch reuse and invalidation logic in Build/Agent/Invoke-VsCodeDebugBuild.ps1
  • the scoped VS Code debug/task changes and their alignment with the documented Visual Studio-first workflow
  • the cleanup of the old nested local-LCM workflow from build config, tasks, and docs

This change is Reviewable

Copilot AI review requested due to automatic review settings March 13, 2026 19:07
@github-actions

This comment has been minimized.

Sign in to view
Copilot AI reviewed Mar 13, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces the previous “overlay/copy local liblcm outputs” approach with an explicit, repo-wide local-source mode rooted at Localizations/LCM, including dedicated build entrypoints and improved VS Code debugging ergonomics.

Changes:

  • Introduces -LcmMode Auto|Package|Local and -ManagedDebugType into build.ps1, plus local-source detection/state reporting and runtime-output syncing for VS Code symbols.
  • Adds FieldWorks.LocalLcm.sln and MSBuild wiring (UseLocalLcmSource, package→project reference switching, build-task bootstrapping).
  • Updates developer setup + VS Code tasks/launchers, and improves toolchain discovery (MSVC path selection).

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
scripts/Agent/Copy-LocalLcm.ps1 Removes the old “copy locally-built LCM DLLs over NuGet outputs” flow.
build.ps1 Adds LcmMode + ManagedDebugType, local-source selection, and symbol/runtime-output refresh.
Src/Common/SimpleRootSite/EditingHelper.cs Disposes VwPropertyStoreManaged via using.
Src/Common/FwUtils/FwUtilsTests/TestFwStylesheetTests.cs Disposes VwPropertyStoreManaged in tests via using.
Src/Common/FieldWorks/FieldWorks.csproj Enables unmanaged debugging for the FieldWorks project.
Src/Common/Controls/Widgets/FontHeightAdjuster.cs Disposes VwPropertyStoreManaged via using.
Setup-Developer-Machine.ps1 Installs .NET SDK 8.x when missing; adjusts helper-repo cloning/junction behavior for worktrees; clones liblcm nested.
FieldWorks.LocalLcm.sln Adds a dedicated solution that includes local liblcm projects for local-source mode.
Docs/architecture/liblcm-local-source-checklist.md Documents the migration checklist and expectations for local-source mode.
Docs/architecture/liblcm-debugging.md Adds detailed debugging guidance for package vs local-source liblcm workflows.
Docs/architecture/dependencies.md Points dependency debugging guidance to the new liblcm debugging doc and summarizes the workflow.
DistFiles/Parts/StandardParts.xml Adds new part definitions (ImportResidue/Pictures).
Directory.Build.targets Adds package→project reference switching for LCM + build-task bootstrapping/validation targets.
Directory.Build.props Adds UseLocalLcmSource defaults for FieldWorks.LocalLcm and redirects LCM paths/artifacts to nested checkout.
Build/Src/FwBuildTasks/Make.cs Improves MSVC tool path detection under modern VS layouts.
Build/Agent/Invoke-VsCodeDebugBuild.ps1 Adds VS Code prelaunch build wrapper with change detection + mode/debug-type stamping.
Build/Agent/FwBuildEnvironment.psm1 Prefers HostX64\x64 MSVC bin path early in PATH.
.vscode/tasks.json Adds explicit package/local build tasks and “prepare debug” tasks; defaults Build to -LcmMode Auto.
.vscode/launch.json Adds package vs local-LCM launch configs with prelaunch tasks + symbol search paths.
.vscode/extensions.json Recommends the official C# extension while keeping Dev Kit unwanted (ReSharper-first).
.serena/project.yml Adds Serena config keys for line ending and read-only memory patterns.

@github-actions
Copy link

github-actions bot commented Mar 13, 2026 •
edited
Loading

NUnit Tests

    1 files  ±  0      1 suites  ±0   5m 50s ⏱️ -11s
4 430 tests +356  4 343 ✅ +340  87 💤 +16  0 ❌ ±0 
4 439 runs  +356  4 352 ✅ +340  87 💤 +16  0 ❌ ±0 

Results for commit 8468561. ± Comparison against base commit de383e0.

This pull request removes 26 and adds 382 tests. Note that renamed tests count towards both. 
SIL.FieldWorks.Common.FwUtils.ParseCharacterSequencesTests ‑ BasicAscii_EachCharSeparate
SIL.FieldWorks.Common.FwUtils.ParseCharacterSequencesTests ‑ CombiningDiacriticFollowsBase_GroupedWithBase
SIL.FieldWorks.Common.FwUtils.ParseCharacterSequencesTests ‑ DiacriticsPrecedeBase_DiacriticBeforeBase_ThenAnotherBase
SIL.FieldWorks.Common.FwUtils.ParseCharacterSequencesTests ‑ DiacriticsPrecedeBase_DiacriticSeparateFromFollowingBase
SIL.FieldWorks.Common.FwUtils.ParseCharacterSequencesTests ‑ DiacriticsPrecedeBase_MultipleDiacritics_EachSeparate
SIL.FieldWorks.Common.FwUtils.ParseCharacterSequencesTests ‑ Digits_EachSeparate
SIL.FieldWorks.Common.FwUtils.ParseCharacterSequencesTests ‑ EmptyString_ReturnsEmpty
SIL.FieldWorks.Common.FwUtils.ParseCharacterSequencesTests ‑ GetReferences_EmptyLines_Skipped
SIL.FieldWorks.Common.FwUtils.ParseCharacterSequencesTests ‑ GetReferences_ReturnsCorrectOffsetsAndLengths
SIL.FieldWorks.Common.FwUtils.ParseCharacterSequencesTests ‑ GetReferences_SupplementaryCharacters_CorrectOffsets
…

SILUBS.ScriptureChecks.CapitalizationCheckSilUnitTest ‑ CapitalizedHeading
SILUBS.ScriptureChecks.CapitalizationCheckSilUnitTest ‑ CapitalizedList
SILUBS.ScriptureChecks.CapitalizationCheckSilUnitTest ‑ CapitalizedProperName_NotParaStart
SILUBS.ScriptureChecks.CapitalizationCheckSilUnitTest ‑ CapitalizedProperName_ParaStart
SILUBS.ScriptureChecks.CapitalizationCheckSilUnitTest ‑ CapitalizedTableCellHead
SILUBS.ScriptureChecks.CapitalizationCheckSilUnitTest ‑ CapitalizedTitle
SILUBS.ScriptureChecks.CapitalizationCheckSilUnitTest ‑ Footnotes_TreatedSeparately
SILUBS.ScriptureChecks.CapitalizationCheckSilUnitTest ‑ GetLengthOfChar
SILUBS.ScriptureChecks.CapitalizationCheckSilUnitTest ‑ LCaseInRunAfterNote
SILUBS.ScriptureChecks.CapitalizationCheckSilUnitTest ‑ LCaseInRunAfterPicture
…

♻️ This comment has been updated with latest results.

@johnml1135
Copy link
Contributor Author

johnml1135 commented Mar 16, 2026

Cleaned up this PR to match the review feedback and the split described in the local-source plan.

What changed:

  • split the Windows build-toolchain work into a separate PR: Improve Visual Studio toolchain detection for Windows builds #766
  • removed the toolchain-only files from this branch:
    • Build/Agent/FwBuildEnvironment.psm1
    • Build/Agent/Verify-FwDependencies.ps1
    • Build/Src/FwBuildTasks/Make.cs
  • removed the unrelated DistFiles/Parts/StandardParts.xml change from this branch
  • removed the planning/checklist markdown that was only used to drive the implementation:
    • Docs/architecture/liblcm-local-source-plan.md
    • Docs/architecture/liblcm-local-source-checklist.md
  • squashed the remaining liblcm workflow work on this branch down to one commit for easier review

What remains in this PR is the local liblcm source-mode implementation and the supporting docs/debugger/build-flow changes that are directly part of that workflow.

The split toolchain PR is here for separate review: #766

@johnml1135 johnml1135 force-pushed the liblcm_debug branch 2 times, most recently from bf0bd50 to 899d70c Compare March 17, 2026 00:39
jasonleenaylor
jasonleenaylor reviewed Mar 18, 2026
View reviewed changes
Copy link
Contributor

@jasonleenaylor jasonleenaylor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR looks like it has conflated two different uses of LCM locally, there is a localization related path used mostly by installers or by local testing of localization and the current focus of being able to test LCM fixes by integrating locally built packages and symbols.

The developer is presumed to have LCM cloned locally. The previous helper script would run pack on lcm and capture the version information out of the build log and modify the package references to point at the new version. It would also copy the symbol artifacts from the lcm build output into the fieldworks build output.

I think a cleaner and more developer configurable change would be to have a -LocalLcm parameter which takes a path to the lcm root folder. Validate that location, run dotnet pack in that location, change the lcm package version in the centralized packaging location for fieldworks, copy symbol files to output if necessary.

@jasonleenaylor reviewed all commit messages and made 1 comment.
Reviewable status: 0 of 131 files reviewed, all discussions resolved.

@johnml1135 johnml1135 changed the title Add local LCM source mode workflow Switch local SIL dependency development to local packages Mar 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jasonleenaylor jasonleenaylor jasonleenaylor left review comments

Copilot code review Copilot Copilot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@johnml1135 @jasonleenaylor