chore(ci): Add CI flow for post-release verification - #1225
Conversation
Teingi
left a comment
There was a problem hiding this comment.
Thanks for adding automated post-release verification. I found two correctness issues that need to be addressed before merge: the clean-environment smoke test cannot locate the CLI it just installed, and the verifier rejects version formats that the release producer currently accepts. Details are inline.
| f"Installed PowerContext version mismatch: expected {expected_version}, received {installed_version}" | ||
| ) | ||
|
|
||
| executable = shutil.which("powercontext") |
There was a problem hiding this comment.
[P1] Please resolve the console script from the verification environment rather than from the ambient PATH. The workflow invokes $RUNNER_TEMP/powercontext-release-verify/bin/python directly without activating that virtual environment or adding its bin directory to PATH, so shutil.which("powercontext") returns None even though the entrypoint exists in the virtual environment. This makes the smoke step fail on a clean release runner, or it can accidentally select an unrelated ambient installation. Derive the executable from sys.executable (for example, Path(sys.executable).with_name("powercontext")) or explicitly prepend the verification environment's bin directory to PATH.
There was a problem hiding this comment.
Thanks—the stable-version change addresses P2. P1 still reproduces because Path.resolve() follows the virtual environment bin/python symlink to the base interpreter before replacing the filename. On Ubuntu, this makes the code look for powercontext outside the virtual environment. Please use Path(python_executable).with_name(name) without resolve(), and update the regression test to use a real venv-style Python symlink.
There was a problem hiding this comment.
Concrete fix:
- executable = Path(python_executable).resolve().with_name(name)
+ executable = Path(python_executable).with_name(name)The regression test should reproduce the POSIX venv symlink instead of creating python as a regular file:
@pytest.mark.skipif(smoke.os.name == "nt", reason="POSIX venv symlink regression")
def test_release_smoke_resolves_console_script_from_verification_python(tmp_path) -> None:
scripts = tmp_path / "verification" / "bin"
scripts.mkdir(parents=True)
base_python = tmp_path / "base-python"
base_python.touch()
python = scripts / "python"
python.symlink_to(base_python)
console_script = scripts / "powercontext"
console_script.touch()
assert smoke._console_script(python) == console_scriptThis test fails with the current .resolve() implementation because it follows python to base_python before replacing the filename, and passes once .resolve() is removed. The existing copy-based case can be kept separately for Windows if desired.
Which issue or RFC does this PR close?
Closes #1121.
Rationale for this change
Release verification previously required manual checks across GitHub Release assets, PyPI availability, clean installation, entrypoints, and basic runtime behavior. This made missing distributions and broken published packages difficult to detect immediately.
This PR adds repeatable post-release verification for the current PowerContext release contract. It also supports manually checking existing PowerMem releases so historical releases can be verified without failing due to package-name differences.
What changes are included in this PR?
Verify releaseworkflow.softprops/action-gh-release.pip checkand verify the installed package version.autopowercontextpowermemrelease_package=auto.Are there any user-facing changes?
no
How was this change tested?
The following repository checks were run locally:
The following checks passed:
ty.The local PowerContext smoke test was also run:
This successfully verified:
The PowerMem compatibility path was tested against the existing public
v1.1.7release:Both GitHub Release and PyPI verification passed.
powermem==1.1.7was also installed from the public PyPI index into a clean Python 3.12 virtual environment. The package version check andpip checkpassed for all installed dependencies.CI verification workflow
The
Verify releaseworkflow can be triggered in two ways.Automatic trigger
The main
Releaseworkflow runs when a GitHub Release is published:The release flow is:
The verification job starts only after both of these jobs succeed:
pypi-publishrelease-assetsFor an automated PowerContext release, the workflow explicitly uses:
The CI verification then checks:
pip check.Manual trigger
The workflow can be run from GitHub:
v1.1.7.autopowercontextpowermemFor old PowerMem releases, use:
For current PowerContext releases, use:
The
autooption detects the package contract from the GitHub Release wheel asset. It recognizes:If the package cannot be uniquely detected, the workflow fails with a message asking for an explicit package selection.
The workflow can also be triggered with GitHub CLI:
To list the latest verification run:
To wait for the workflow result:
AI usage statement
Coworked with OpenAI Codex.