PR3-C migration: resolve formal source canonicals before prod migration#37
Open
paynejd wants to merge 3 commits into
Open
PR3-C migration: resolve formal source canonicals before prod migration#37paynejd wants to merge 3 commits into
paynejd wants to merge 3 commits into
Conversation
…ce canonicals Pre-prod hardening of the v1->v2 candidates migration (PR3-C, #2540/#2541) ahead of the production run. Keys concepts on each source's formal canonical_url (matching the live app) instead of a generated ns.openconceptlab.org URL. - dump-projects.sql: dump resolving target_repo + custom/bridge canonicals (ICD-11 -> http://id.who.int/icd/release/11/mms; PIH -> generated; else source canonical). Recovers 1703 dropped custom-algo results; collapses 10 split-identity projects. migrate.mjs unchanged. - fix-algorithm-canonicals.sql: one-time --rw fix of map_projects.algorithms so the live app stays consistent with migrated keys on re-run. - validate.mjs: independent v1<->v2 equivalence + referential-integrity validator (0 hard failures on the live 69-project snapshot). - README: runbook fixes (run migrate.mjs from 315e9b0; 69 targets; retain backup permanently; never re-dump+migrate after apply). Verified read-only/dry-run against the 2026-06-02 prod snapshot; see verification-report.md. Orphan-tag drops (~975) tracked separately. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e.mjs (vendor normalizer) Removes the fragile "run from commit 315e9b0" requirement. migrate.mjs imported normalizeLegacyAllCandidates from src/normalizers.js, which the v2-only PR (9f8f4b3) deleted from main 29 min after the script merged — so `node migrate.mjs` crashed from a current main checkout. Vendor the normalizer (normalizers.js + its only dep conceptKey.js, pinned to the validated 315e9b0 versions) into ./vendored/ and repoint the import. (Not ./lib/ — the root .gitignore ignores lib/.) migrate.mjs now runs from ANY checkout, including main. README reverted to the plain `node migrate.mjs` command (no worktree / no SHA). No behavior change: the vendored normalizer is byte-identical to 315e9b0, which every prior dry-run + Sunny's testing used. Verified the self-contained run from a main-based checkout: 69 ok / 0 skipped / 0 errors, validator 0 hard failures (unchanged). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
paynejd
force-pushed
the
pr3c-migration-canonical-fix
branch
from
93a56dd to
96083f0
Compare
… candidates Adds a migration-only result-derived fallback so candidates whose algorithm tag isn't in the project's current algorithms[] (reconfigured / mistagged projects) are recovered instead of dropped at the normalizer's `if(!algoDef) return`. - vendored/normalizers.js: new `reference_source: 'result'` in resolveReference + source-path helpers. When a tag has no matched/typed algo, derive the concept's identity from the RESULT itself: if its source is the project's target source -> anchor to the target canonical+version (formal; visible + recommendable); otherwise key on its own generated source (correctly NON-target, so the UI's target filter keeps it inert, never mis-surfaced). Bridge/AI-payload visibility is gated by conceptBelongsToTargetRepo, never by algorithm_id, so an unrecognized id is a benign label. - validate.mjs: recovery-aware — reports orphan codes recovered vs still-missing and treats any still-missing as a hard failure. - README: documents the third fallback; vendored note reflects the patch. Measured on the live 69-project snapshot: 731/731 recoverable codes recovered, 0 still-missing, 0 hard failures, 0 referential errors; target/non-target keying verified (e.g. id=114 stubs -> formal loinc.org; id=132/135 CIEL intermediaries -> generated CIEL / inert). The by-algorithm-view UI nicety is a separate app PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes implementation of
OpenConceptLab/ocl_issues#2555(prod migration run still pending — bare ref, mirroring how the migration script referenced #2540).What & why
Pre-production hardening of the PR3-C v1→v2 candidates migration (#2540/#2541), ahead of the prod run. A read-only verification against the full live
map_projectssnapshot (69 non-empty projects, seeverification-report.md) found the migration keys concepts on the generatedns.openconceptlab.orgURL while the live app keys on each source's formalcanonical_url, causing custom-algo drops, concept-key splits, and migrated↔app key mismatches.Changes (migration tooling only — no
src// app code)dump-projects.sql— migration dump that resolves formal canonicals inline (self-contained; dry-run-able):ICD-11-WHO-Agent) →http://id.who.int/icd/release/11/mmsns.openconceptlab.org(no formal canonical)canonical_url(CIEL/LOINC/SNOMED-GPS)migrate.mjsis unchanged.fix-algorithm-canonicals.sql— one-time--rwUPDATE ofmap_projects.algorithmsapplying the same resolution to custom + bridge algos, so the live app produces formal keys on re-run (it reads these from the algo config).validate.mjs— independent v1↔v2 equivalence + referential-integrity validator.migrate.mjsfrom315e9b0; 69 targets (not 68); retain the backup permanently; never re-dump+migrate after apply.Verified (dry-run + validator on the live 69)
Sandbox apply remained 69/69 →
mapper_schema_version: 2; rollback byte-identical.Out of scope
The ~975 orphan-tag drops (results tagged with an algorithm not in the project's current
algorithms[]) are a separate normalizer fix, not a canonical issue — to be ticketed separately.🤖 Generated with Claude Code