Skip to content

chore(ci): Add CI flow for post-release verification - #1225

Open
Alanxtl wants to merge 3 commits into
oceanbase:masterfrom
Alanxtl:add_ci
Open

chore(ci): Add CI flow for post-release verification#1225
Alanxtl wants to merge 3 commits into
oceanbase:masterfrom
Alanxtl:add_ci

Conversation

@Alanxtl

@Alanxtl Alanxtl commented Aug 13, 2026

Copy link
Copy Markdown

Which issue or RFC does this PR close?

Closes #1121.

Rationale for this change

Release verification previously required manual checks across GitHub Release assets, PyPI availability, clean installation, entrypoints, and basic runtime behavior. This made missing distributions and broken published packages difficult to detect immediately.

This PR adds repeatable post-release verification for the current PowerContext release contract. It also supports manually checking existing PowerMem releases so historical releases can be verified without failing due to package-name differences.

What changes are included in this PR?

  • Add a reusable and manually dispatchable Verify release workflow.
  • Publish PowerContext wheel and source distributions as GitHub Release assets using softprops/action-gh-release.
  • Run release verification after both PyPI publication and GitHub Release asset upload succeed.
  • Verify that the expected wheel and source distribution exist on GitHub Releases.
  • Verify that the expected wheel and source distribution are available on PyPI.
  • Retry PyPI verification to account for publication propagation delays.
  • Install the released package from public PyPI in a clean virtual environment.
  • Run pip check and verify the installed package version.
  • Verify the PowerContext CLI and Server entrypoints.
  • Start a real PowerContext Server backed by a temporary SQLite database.
  • Verify Server readiness and the Streamable HTTP MCP endpoint.
  • Run a Memory write and FTS search smoke test through the public SDK.
  • Produce a concise release health summary in GitHub Actions.
  • Support manually verifying existing releases with package selection:
    • auto
    • powercontext
    • powermem
  • Automatically detect legacy PowerMem releases from their GitHub Release wheel assets when release_package=auto.
  • Skip PowerContext-specific CLI, Server, MCP, and Memory smoke tests for PowerMem releases.
  • Update GitHub Actions dependencies to their latest available release tags.
  • Add focused tests for release metadata validation, missing distributions, missing assets, and PyPI retry behavior.

Are there any user-facing changes?

no

How was this change tested?

The following repository checks were run locally:

uv lock --locked
uv run prek run -a
uv run ty check
uv run python -m pytest \
  tests/test_ci_release_verification.py \
  tests/test_cli.py \
  tests/e2e/test_server_cli.py \
  tests/e2e/test_mcp_transport.py

The following checks passed:

  • Dependency lock consistency.
  • All repository pre-commit hooks.
  • Ruff linting and formatting.
  • Static type checking with ty.
  • 29 focused release, CLI, Server, and MCP tests.
  • Workflow YAML validation.
  • Git diff validation.

The local PowerContext smoke test was also run:

$version = uv run python -c "from importlib.metadata import version; print(version('powercontext'))"
uv run python scripts/ci_release_smoke.py \
  --version $version \
  --timeout-seconds 30

This successfully verified:

  • Installed PowerContext version.
  • CLI availability.
  • Server startup.
  • Server readiness.
  • SQLite database initialization.
  • Memory write through the public SDK.
  • FTS Memory search.
  • MCP endpoint startup.
  • MCP tool discovery.
  • Server process cleanup.

The PowerMem compatibility path was tested against the existing public v1.1.7 release:

uv run python scripts/ci_verify_github_release.py \
  --repository oceanbase/powercontext \
  --tag v1.1.7 \
  --expected-asset powermem-1.1.7-py3-none-any.whl \
  --expected-asset powermem-1.1.7.tar.gz

uv run python scripts/ci_verify_pypi_release.py \
  --package powermem \
  --version 1.1.7 \
  --attempts 1

Both GitHub Release and PyPI verification passed.

powermem==1.1.7 was also installed from the public PyPI index into a clean Python 3.12 virtual environment. The package version check and pip check passed for all installed dependencies.

CI verification workflow

The Verify release workflow can be triggered in two ways.

Automatic trigger

The main Release workflow runs when a GitHub Release is published:

on:
  release:
    types: [published]

The release flow is:

Release published
    |
    v
Build PowerContext wheel and sdist
    |
    +--> Publish distributions to PyPI
    |
    +--> Upload distributions to GitHub Release
              |
              v
       Verify release

The verification job starts only after both of these jobs succeed:

  • pypi-publish
  • release-assets

For an automated PowerContext release, the workflow explicitly uses:

release_package: powercontext

The CI verification then checks:

  • GitHub Release state and assets.
  • PyPI package availability.
  • Clean installation from public PyPI.
  • Dependency consistency with pip check.
  • Package version metadata.
  • PowerContext CLI entrypoints.
  • Server readiness.
  • MCP endpoint and tools.
  • SQLite Memory write and FTS search.

Manual trigger

The workflow can be run from GitHub:

  1. Open the repository on GitHub.
  2. Go to Actions.
  3. Select Verify release.
  4. Click Run workflow.
  5. Select the branch containing the verification workflow.
  6. Enter the published release tag, for example v1.1.7.
  7. Select the package contract:
    • auto
    • powercontext
    • powermem
  8. Start the workflow.

For old PowerMem releases, use:

release_tag: v1.1.7
release_package: powermem

For current PowerContext releases, use:

release_tag: vX.Y.Z
release_package: powercontext

The auto option detects the package contract from the GitHub Release wheel asset. It recognizes:

powercontext-X.Y.Z-py3-none-any.whl
powermem-X.Y.Z-py3-none-any.whl

If the package cannot be uniquely detected, the workflow fails with a message asking for an explicit package selection.

The workflow can also be triggered with GitHub CLI:

gh workflow run release-verify.yml \
  --repo oceanbase/powercontext \
  --ref master \
  -f release_tag=v1.1.7 \
  -f release_package=auto

To list the latest verification run:

gh run list \
  --repo oceanbase/powercontext \
  --workflow release-verify.yml \
  --limit 1

To wait for the workflow result:

gh run watch <run-id> \
  --repo oceanbase/powercontext \
  --exit-status

AI usage statement

Coworked with OpenAI Codex.

@CLAassistant

CLAassistant commented Aug 13, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@PsiACE

PsiACE commented Aug 13, 2026

Copy link
Copy Markdown
Member

Thanks for the contribution, @Alanxtl!

cc @Teingi, could you help review this since you're preparing the 0.0.1 release?

@PsiACE
PsiACE requested a review from Teingi August 13, 2026 03:24

@Teingi Teingi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding automated post-release verification. I found two correctness issues that need to be addressed before merge: the clean-environment smoke test cannot locate the CLI it just installed, and the verifier rejects version formats that the release producer currently accepts. Details are inline.

Comment thread scripts/ci_release_smoke.py Outdated
f"Installed PowerContext version mismatch: expected {expected_version}, received {installed_version}"
)

executable = shutil.which("powercontext")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Please resolve the console script from the verification environment rather than from the ambient PATH. The workflow invokes $RUNNER_TEMP/powercontext-release-verify/bin/python directly without activating that virtual environment or adding its bin directory to PATH, so shutil.which("powercontext") returns None even though the entrypoint exists in the virtual environment. This makes the smoke step fail on a clean release runner, or it can accidentally select an unrelated ambient installation. Derive the executable from sys.executable (for example, Path(sys.executable).with_name("powercontext")) or explicitly prepend the verification environment's bin directory to PATH.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks—the stable-version change addresses P2. P1 still reproduces because Path.resolve() follows the virtual environment bin/python symlink to the base interpreter before replacing the filename. On Ubuntu, this makes the code look for powercontext outside the virtual environment. Please use Path(python_executable).with_name(name) without resolve(), and update the regression test to use a real venv-style Python symlink.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concrete fix:

-    executable = Path(python_executable).resolve().with_name(name)
+    executable = Path(python_executable).with_name(name)

The regression test should reproduce the POSIX venv symlink instead of creating python as a regular file:

@pytest.mark.skipif(smoke.os.name == "nt", reason="POSIX venv symlink regression")
def test_release_smoke_resolves_console_script_from_verification_python(tmp_path) -> None:
    scripts = tmp_path / "verification" / "bin"
    scripts.mkdir(parents=True)
    base_python = tmp_path / "base-python"
    base_python.touch()
    python = scripts / "python"
    python.symlink_to(base_python)
    console_script = scripts / "powercontext"
    console_script.touch()

    assert smoke._console_script(python) == console_script

This test fails with the current .resolve() implementation because it follows python to base_python before replacing the filename, and passes once .resolve() is removed. The existing copy-based case can be kept separately for Windows if desired.

Comment thread .github/workflows/release-verify.yml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement]: Add automated post-release verification for assets, PyPI packages, entrypoints, and smoke tests

4 participants