Skip to content

ci: keep pins.json in sync with pyproject.toml; cap torch <2.12#9351

Open
lstein wants to merge 6 commits into
invoke-ai:mainfrom
lstein:ci/pins-consistency-check
Open

ci: keep pins.json in sync with pyproject.toml; cap torch <2.12#9351
lstein wants to merge 6 commits into
invoke-ai:mainfrom
lstein:ci/pins-consistency-check

Conversation

@lstein

@lstein lstein commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

The launcher installs InvokeAI's dependencies from two sources that nothing keeps in sync: pins.json (fetched at the release tag, used for the torch index URL in legacy installs) and pyproject.toml/uv.lock. pins.json drifted — its Linux ROCm index stayed at rocm6.3 after pyproject.toml moved to rocm7.1 — so launcher installs on AMD got ROCm 6.3 wheels, which cannot work at all on RDNA4 cards (#9328). #9337 fixed the immediate URL; this PR adds the guardrail so it can't silently drift again, and caps torch below the version that breaks ROCm installs.

Changes

  • New scripts/check_pins.py: validates pins.json against an explicit required platform/backend matrix (linux: cpu/cuda/rocm, win32: cpu/cuda, darwin: none — MPS has no separate torch index). Missing entries, missing platform sections, unknown platforms, unsupported backends, and URLs that disagree with the matching torch-<backend> [[tool.uv.index]] in pyproject.toml all fail the check. A missing entry matters as much as a stale one: the launcher's schema makes every backend key optional and it only passes --index when the entry exists, so an absent entry silently falls back to the default PyPI index.
  • New tests/test_check_pins.py: removes each required entry in turn (and each platform section) and asserts a nonzero exit, plus stale-URL, unknown-platform and unsupported-backend cases. The parametrized cases are generated from the matrix itself so coverage can't silently shrink.
  • pins.json: adds the win32.cpu entry the new matrix exposes as missing. Without it a Windows "No dedicated GPU" legacy install gets no --index and pulls the CUDA-bundled PyPI torch wheel rather than the +cpu build; linux.cpu has always been there.
  • uv-lock-checks workflow: runs the new check, triggered by changes to pyproject.toml, uv.lock, pins.json, or the check script.
  • python-tests workflow: path filter extended to scripts/** and pins.json, so editing the checker runs its own tests.
  • pyproject.toml + uv.lock: base torch constraint capped at <2.12 on linux/windows. torch 2.12.x+rocm7.1 breaks generation ([bug]: Release 6.13.5rc1 not installing ROCm 7.1 as expected #9328; 2.11 works, and the rocm extra pins 2.10.0). The cap only affects legacy (pre-6.14) launcher installs, which resolve this range live against the pytorch indexes instead of using uv.lock — no locked versions changed (uv lock --locked passes; the lock diff is just the recorded specifier). Remove the cap once the 2.12 ROCm incompatibility is diagnosed.

Caveat

Because the launcher fetches pins.json at the release tag, none of this retroactively fixes already-tagged releases (e.g. v6.13.5.rc1 carries rocm6.3 forever). It takes effect from the next tag onward.

QA

  • python3 scripts/check_pins.py passes on the fixed files.
  • pytest tests/test_check_pins.py — 19 passed.
  • uv lock --locked passes after the constraint change; no package versions moved in the lock.
  • Workflow YAML validated.

Related: #9328, #9337, invoke-ai/launcher#131

🤖 Generated with Claude Code

lstein and others added 2 commits July 11, 2026 11:54
pins.json is consumed only by the Invoke Launcher (fetched at the release
tag) to pick the torch wheel index for legacy installs. Nothing in-repo
references it, so it silently drifted from pyproject.toml: its Linux ROCm
index stayed at rocm6.3 after the torch-rocm index moved to rocm7.1,
causing the launcher to install ROCm 6.3 wheels (invoke-ai#9328, launcher#131).

- Add scripts/check_pins.py, which fails if any torchIndexUrl entry in
  pins.json differs from the matching [[tool.uv.index]] URL in
  pyproject.toml.
- Run it from the uv-lock-checks workflow, triggered by changes to any of
  pyproject.toml, uv.lock, pins.json, or the check script.
- Bump pins.json rocm index to rocm7.1 (matches invoke-ai#9337) so the check
  passes.
- Bump the workflow's uv from 0.6.10 to 0.11.28: uv.lock is already lock
  format revision 3, which 0.6.x cannot parse.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
torch 2.12.x+rocm7.1 breaks generation (invoke-ai#9328; 2.11 works, and 2.10 is
what the rocm extra pins). The cap only matters for legacy (pre-6.14)
launcher installs, which resolve the base range live against the pytorch
wheel indexes instead of using uv.lock — with the range open to <3.0,
AMD users could land on the broken 2.12.x. Lockfile-based installs are
unaffected: uv.lock stays at 2.7.1 / 2.10.0+rocm7.1 (no locked versions
changed, only the recorded specifier).

Remove the cap once the 2.12 ROCm incompatibility is diagnosed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@lstein
lstein requested a review from blessedcoolant as a code owner July 11, 2026 15:56
@github-actions github-actions Bot added CI-CD Continuous integration / Continuous delivery Root python-deps PRs that change python dependencies labels Jul 11, 2026
@lstein lstein added the 6.14.0 label Jul 11, 2026
@lstein lstein moved this to 6.14.x Theme: USER EXPERIENCE in Invoke - Community Roadmap Jul 11, 2026

@JPPhoto JPPhoto left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • invokeai/scripts/check_pins.py:32-43: The consistency check only validates entries that already exist in pins.json; it does not require every launcher-supported platform/backend entry to be present. Removing torchIndexUrl.linux.rocm still prints pins.json is consistent with pyproject.toml and exits 0. The launcher's schema (main over in the launcher repo) permits backend keys to be absent, and its legacy install path only adds --index when the selected entry exists. An omitted ROCm entry therefore silently falls back from the ROCm wheel index, causing AMD installations to resolve an unsuitable build or fail. This recreates the exact installation failure the new guardrail is intended to prevent. To expose this issue, add a test that removes each required CPU, CUDA, or ROCm entry in turn and asserts that the checker exits nonzero; the implementation should validate an explicit required platform/backend matrix, not merely iterate existing entries.

JPPhoto and others added 3 commits July 15, 2026 13:52
Resolved uv.lock conflict by regenerating from main's lock with `uv lock`
rather than merging hunks. The only substantive change is the recorded
`torch` specifier (>=2.7.0,<2.12); no package versions moved.
Addresses JPPhoto's review. The checker only iterated the entries that
already existed in pins.json, so deleting torchIndexUrl.linux.rocm still
exited 0 — recreating the exact silent-fallback failure the guardrail was
meant to prevent (the launcher only passes --index when the entry exists,
and its schema makes every backend key optional).

- Validate pins.json against REQUIRED_BACKENDS, an explicit matrix of the
  platform/backend combinations the launcher can select and Invoke supports.
  Missing entries, missing platform sections, unknown platforms and
  unsupported backends are now all errors, alongside the existing URL check.
- Split the logic into check_pins(), with main() taking a repo root, so the
  exit code is directly testable.
- Add tests/test_check_pins.py: removes each required entry in turn and
  asserts a nonzero exit, plus stale-URL, unknown-platform and
  unsupported-backend cases.
- pins.json: add the win32.cpu entry the matrix exposes as missing. Without
  it, a Windows "no dedicated GPU" legacy install gets no --index and pulls
  the CUDA-bundled PyPI torch wheel instead of the +cpu build. Linux already
  had this entry.
- python-tests: trigger on scripts/** and pins.json so these tests actually
  run when the checker or the pins change.
- Revert the uv 0.6.10 -> 0.11.28 bump in uv-lock-checks: 0.6.10 parses the
  revision-3 lock fine (verified), so the stated justification was wrong and
  the bump left this workflow out of step with python-tests.
@github-actions github-actions Bot added the python-tests PRs that change python tests label Jul 24, 2026
@lstein

lstein commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author

Good catch — you're right, and the failure mode was exactly as you described: del pins["torchIndexUrl"]["linux"]["rocm"] printed "consistent" and exited 0, because the loop only walked entries that were already there. Fixed at 7f8809b.

Explicit required matrix. scripts/check_pins.py now validates against a declared matrix rather than iterating what happens to exist:

REQUIRED_BACKENDS: dict[str, set[str]] = {
    "win32": {"cpu", "cuda"},
    "linux": {"cpu", "cuda", "rocm"},
    "darwin": set(),
}

It's an exact match in both directions, so four classes of drift now fail:

  • missing entrytorchIndexUrl.linux.rocm removed → exit 1, with the message spelling out the consequence ("the launcher would omit --index for rocm installs on linux and resolve torch from the default index")
  • missing platform section — including darwin, whose empty-but-present section is still required
  • unsupported backend — e.g. a win32.rocm entry, which PyTorch publishes no wheels for and which pyproject.toml marks sys_platform == 'linux'
  • stale URL — the original [bug]: Release 6.13.5rc1 not installing ROCm 7.1 as expected #9328 case, unchanged

I derived the matrix from the launcher rather than guessing: getTorchPlatform() in src/main/util.ts maps all four GPU choices to cuda/rocm/cpu on every OS, and InstallFlowStepConfigureGpuPicker.tsx offers all four regardless of platform — so every cell the launcher can actually reach and that Invoke supports is required. darwin is empty on purpose (MPS has no separate torch index) and that intent is now asserted, not implied by absence.

Tests. New tests/test_check_pins.py (19 cases). The parametrized ones are generated from check_pins.REQUIRED_BACKENDS itself, so they can't silently stop covering a backend if the matrix is edited, and there's a separate assertion that the matrix still contains linux cpu/cuda/rocm and win32 cpu/cuda — narrowing the matrix to make a failure go away trips that. To make exit codes directly testable, main() now takes a repo root and the logic moved into a pure check_pins(pins, pyproject).

Also added scripts/** and pins.json to the python tests workflow's path filter — without that, editing check_pins.py wouldn't have run its own tests. (This incidentally also fixes the same gap for the existing test_docs_json_export.py.)

One consequence worth flagging: the new matrix immediately caught a second missing cell, win32.cpu, so this PR now adds it to pins.json. Today a Windows "No dedicated GPU" legacy install gets no --index and resolves torch from the default PyPI index — which on Windows is the CUDA-bundled wheel, not the +cpu build. Linux has had linux.cpu all along; this just brings Windows in line. If that was a deliberate omission rather than an oversight, say the word and I'll drop cpu from the win32 row instead.

Unrelated to your review, two other changes in this push:

  • uv.lock conflict resolved by regenerating from main's lock with uv lock rather than merging hunks. uv lock --locked passes; the only substantive change is the recorded torch specifier — no package versions moved.
  • Reverted the uv 0.6.100.11.28 bump in uv-lock-checks.yml. I'd justified it in the PR description by claiming uv 0.6.x can't parse a revision-3 lockfile. I actually tested it this time — 0.6.10 resolves this lock fine — so the premise was wrong and the bump only left this workflow out of step with python-tests. The workflow diff is now just the pins check.

@lstein
lstein requested a review from JPPhoto July 24, 2026 23:07

@JPPhoto JPPhoto left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing found:

  • invokeai/pyproject.toml:52-54: The reported incompatibility is specifically torch 2.12.x+rocm7.1, but the new sys_platform != 'darwin' upper bound is emitted into package metadata for every Linux and Windows installation, including CUDA, CPU-only, and Linux ARM64 environments that do not use ROCm. Direct/manual installs in those environments can therefore reject an otherwise required torch>=2.12, downgrade an existing Torch installation, or fail dependency resolution; this also contradicts the PR's claim that the cap only affects legacy launcher installs. Test: resolve and smoke-test InvokeAI with Torch 2.12 on representative Windows CUDA, Windows CPU, Linux CUDA, Linux CPU, and Linux ARM64 environments, and either scope the cap to the affected legacy ROCm resolution path or document evidence that Torch 2.12 is incompatible on every capped platform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.14.0 CI-CD Continuous integration / Continuous delivery python-deps PRs that change python dependencies python-tests PRs that change python tests Root

Projects

Status: 6.14.x Theme: USER EXPERIENCE

Development

Successfully merging this pull request may close these issues.

2 participants