Skip to content

fix(ci): move PyPI publish out of a reusable workflow - #92

Merged
jimisola merged 3 commits into
mainfrom
fix/pypi-trusted-publishing-reusable-workflow
Aug 23, 2026
Merged

fix(ci): move PyPI publish out of a reusable workflow#92
jimisola merged 3 commits into
mainfrom
fix/pypi-trusted-publishing-reusable-workflow

Conversation

@jimisola

@jimisola jimisola commented Aug 23, 2026

Copy link
Copy Markdown
Member

Description

PyPI does not accept a reusable workflow as a trusted publisher, full stop —
pypi/warehouse#11096, unresolved. The
workaround, confirmed working elsewhere
via OpenAstronomy's docs:
the job that calls pypa/gh-action-pypi-publish must be defined directly in the caller's
own workflow file. A composite action is fine — the OIDC claim is about which workflow file
the job runs in, not what its steps reference — but workflow_call is not.

python-publish-to-pypi.yml was exactly the shape PyPI rejects. Confirmed two ways today:

  1. Four real release runs — reqstool-client, reqstool-python-decorators,
    reqstool-python-hatch-plugin, reqstool-python-poetry-plugin — all failed identically
    at the publish step, after tag/build/assets all succeeded, with
    invalid-publisher: valid token, but no corresponding publisher. This is a different,
    deeper problem than the workflow-filename/environment staleness fixed for npm's trusted
    publisher — no config value on pypi.org could have fixed it.
  2. publish-dev-to-testpypi.yml — a separate workflow publishing a dev build to Test PyPI
    on every push to main — has been failing on every run since the 2026-08-16
    migration
    , across all four repos, for a week, unnoticed.

What changes

  • New: .github/actions/publish-to-pypi, a composite action wrapping the
    download-artifact → (dry-run twine check | pypa/gh-action-pypi-publish) steps
    python-publish-to-pypi.yml had.
  • Deleted: python-publish-to-pypi.yml and its test stub, rather than kept around
    deprecated — a known-broken-for-purpose reusable workflow left in place is exactly what
    invites someone to wire it up again.
  • Test PyPI is gone, not carried forward. PyPI accepts a pre-release version identifier
    (0.3.0rc1) on the real index directly, and pip ignores it without --pre — there was
    never a need for a separate staging index to hold a candidate safely, so the action has no
    target input at all, just PyPI.
  • RELEASING.md: updated throughout — the staging-publish step is gone, the pipeline
    diagram points at the composite action, the test environment is now unused (left
    declared rather than removed, in case a future index needs it), the trusted-publishing
    warnings drop the test.pypi.org references, and the release-candidate section documents
    PyPI's new behavior (publishes straight to the real index, like npm and Maven Central)
    next to the existing VS Code exception.

What this loses

The old workflow derived environment: stable from target automatically for the real
index. A composite action can't set the caller's environment: — that's a job-level
attribute only the workflow file itself can declare. Every caller now sets it explicitly.

Companion PRs

Every PyPI-publishing repo's release.yml needs to stop calling python-publish-to-pypi.yml
and switch to this action instead, and drop its own Test PyPI jobs:
client#444,
decorators#91,
hatch-plugin#102,
poetry-plugin#139.

Merge order: this one first — the companions reference
reqstool/.github/.github/actions/publish-to-pypi@main, which doesn't exist until this
merges.

Checklist

Test plan

tests/actions/run-tests.sh — 40/40 passing locally. shellcheck, yamllint, and zizmor
all clean locally (zizmor caught a real template-injection finding mid-review — target
validation was interpolating ${{ inputs.target }} directly into a run: string instead of
going through env:; fixed, and now moot since the target input is gone entirely).
tests/python/publish-to-pypi.yml's actionlint coverage is gone with the file it tested;
the composite action has no workflow_call surface for actionlint to check the same way, so
the companion PRs' real dispatches are the actual test.

PyPI does not accept a reusable workflow as a trusted publisher, at all --
pypi/warehouse#11096, unresolved. python-publish-to-pypi.yml is exactly
that shape: every consumer's release.yml and publish-dev-to-*.yml called
it via workflow_call, so every real publish attempt failed at the OIDC
exchange with invalid-publisher, regardless of what the trusted publisher
config named. Confirmed against four real release runs today, and against
a week of failing publish-dev-to-testpypi.yml runs across every
PyPI-publishing repo since the 2026-08-16 migration -- nobody had noticed.

actions/publish-to-pypi replaces it as a composite action. The OIDC claim
is about which workflow file the job runs in, not what its steps
reference, so a job defined directly in the caller's own workflow that
calls this action for steps still satisfies PyPI's requirement. Consumer
repos need a matching change to stop calling the old reusable workflow --
tracked as companion PRs.

python-publish-to-pypi.yml and its test are deleted rather than kept
around deprecated: leaving a known-broken-for-purpose reusable workflow in
place is exactly what invites someone to wire it up again.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
Comment thread .github/actions/publish-to-pypi/action.yml Fixed
zizmor flagged the target-validation step for template injection --
${{ inputs.target }} was interpolated directly into the run: string
instead of going through env:, so a malicious target value could break
out of the quotes. Matches the pattern every other composite action in
this repo already uses.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
PyPI accepts a pre-release version identifier (0.3.0rc1) on the real
index directly, and pip ignores it without --pre -- there was never a
need for a separate staging index to hold a candidate safely. Test PyPI
existed here only as that safety net.

Simplifies actions/publish-to-pypi to a single target (drops the
target input, check-target.sh, and its tests). Companion PRs drop
publish-to-testpypi from each repo's release.yml and delete
publish-dev-to-testpypi.yml outright, and remove the prerelease gate on
publish-to-pypi -- which also makes PyPI's release-candidate handling
consistent with npm and Maven Central, which already publish candidates
straight to their real index rather than skipping.

RELEASING.md updated throughout: the staging-publish step is gone, the
`test` environment is now unused (left declared rather than removed
from safe-settings, in case a future index needs it), and the
release-candidate section documents PyPI's new behavior next to the
existing VS Code exception.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
jimisola added a commit to reqstool/reqstool-client that referenced this pull request Aug 23, 2026
PyPI accepts a pre-release version identifier (0.3.0rc1) on the real
index directly, and pip ignores it without --pre -- there was never a
need to route a release candidate to a separate staging index to hold
it safely. Test PyPI existed here only as that safety net.

publish-to-pypi now runs unconditionally instead of skipping for a
release candidate, which also makes this consistent with what npm and
Maven Central already do. publish-dev-to-testpypi.yml is deleted --
it's been failing on every push to main since the 2026-08-16 migration
anyway (see reqstool/.github#92), and there's nowhere left for it to
publish to.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
jimisola added a commit to reqstool/reqstool-python-decorators that referenced this pull request Aug 23, 2026
PyPI accepts a pre-release version identifier (0.3.0rc1) on the real
index directly, and pip ignores it without --pre -- there was never a
need to route a release candidate to a separate staging index to hold
it safely. Test PyPI existed here only as that safety net.

publish-to-pypi now runs unconditionally instead of skipping for a
release candidate, which also makes this consistent with what npm and
Maven Central already do. publish-dev-to-testpypi.yml is deleted --
it's been failing on every push to main since the 2026-08-16 migration
anyway (see reqstool/.github#92), and there's nowhere left for it to
publish to.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
jimisola added a commit to reqstool/reqstool-python-hatch-plugin that referenced this pull request Aug 23, 2026
PyPI accepts a pre-release version identifier (0.3.0rc1) on the real
index directly, and pip ignores it without --pre -- there was never a
need to route a release candidate to a separate staging index to hold
it safely. Test PyPI existed here only as that safety net.

publish-to-pypi now runs unconditionally instead of skipping for a
release candidate, which also makes this consistent with what npm and
Maven Central already do. publish-dev-to-testpypi.yml is deleted --
it's been failing on every push to main since the 2026-08-16 migration
anyway (see reqstool/.github#92), and there's nowhere left for it to
publish to.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
jimisola added a commit to reqstool/reqstool-python-poetry-plugin that referenced this pull request Aug 23, 2026
PyPI accepts a pre-release version identifier (0.3.0rc1) on the real
index directly, and pip ignores it without --pre -- there was never a
need to route a release candidate to a separate staging index to hold
it safely. Test PyPI existed here only as that safety net.

publish-to-pypi now runs unconditionally instead of skipping for a
release candidate, which also makes this consistent with what npm and
Maven Central already do. publish-dev-to-testpypi.yml is deleted --
it's been failing on every push to main since the 2026-08-16 migration
anyway (see reqstool/.github#92), and there's nowhere left for it to
publish to.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
@jimisola
jimisola merged commit d6445af into main Aug 23, 2026
12 checks passed
@jimisola
jimisola deleted the fix/pypi-trusted-publishing-reusable-workflow branch August 23, 2026 02:15
jimisola added a commit to reqstool/reqstool-client that referenced this pull request Aug 23, 2026
* fix(ci): stop publishing to PyPI through a reusable workflow

PyPI does not accept a reusable workflow as a trusted publisher --
pypi/warehouse#11096, unresolved. Both publish jobs here called
reqstool/.github's python-publish-to-pypi.yml via workflow_call, so
every real publish attempt failed at the OIDC exchange with
invalid-publisher regardless of what the trusted publisher config
named -- confirmed today, and it explains why publish-dev-to-testpypi.yml
has been failing on every push to main since the 2026-08-16 migration.

Switches both jobs to reqstool/.github's new actions/publish-to-pypi
composite action, called directly from a job defined in this workflow.
A composite action doesn't have the same problem: the OIDC claim is
about which workflow file the job runs in, not what its steps reference.

The two-index environment derivation (testpypi -> test, pypi -> stable)
that the reusable workflow computed internally now has to be explicit
per job -- a composite action can't set the caller's environment:.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

* fix(ci): drop Test PyPI entirely rather than fix its trusted publisher

PyPI accepts a pre-release version identifier (0.3.0rc1) on the real
index directly, and pip ignores it without --pre -- there was never a
need to route a release candidate to a separate staging index to hold
it safely. Test PyPI existed here only as that safety net.

publish-to-pypi now runs unconditionally instead of skipping for a
release candidate, which also makes this consistent with what npm and
Maven Central already do. publish-dev-to-testpypi.yml is deleted --
it's been failing on every push to main since the 2026-08-16 migration
anyway (see reqstool/.github#92), and there's nowhere left for it to
publish to.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

* fix(ci): pin the publish-to-pypi action reference to a commit

CodeQL flagged actions/unpinned-tag: @main is a mutable ref, so a
compromised or force-pushed reqstool/.github main would execute
through this reference with no review. Pinned to the commit main
pointed at when this action was added, matching the org's existing
`@<sha> # main YYYY-MM-DD` convention for third-party/internal action
references elsewhere in this repo.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

---------

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
jimisola added a commit to reqstool/reqstool-python-poetry-plugin that referenced this pull request Aug 23, 2026
* fix(ci): stop publishing to PyPI through a reusable workflow

PyPI does not accept a reusable workflow as a trusted publisher --
pypi/warehouse#11096, unresolved. Both publish jobs here called
reqstool/.github's python-publish-to-pypi.yml via workflow_call, so
every real publish attempt failed at the OIDC exchange with
invalid-publisher regardless of what the trusted publisher config
named -- confirmed today, and it explains why publish-dev-to-testpypi.yml
has been failing on every push to main since the 2026-08-16 migration.

Switches both jobs to reqstool/.github's new actions/publish-to-pypi
composite action, called directly from a job defined in this workflow.
A composite action doesn't have the same problem: the OIDC claim is
about which workflow file the job runs in, not what its steps reference.

The two-index environment derivation (testpypi -> test, pypi -> stable)
that the reusable workflow computed internally now has to be explicit
per job -- a composite action can't set the caller's environment:.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

* fix(ci): drop Test PyPI entirely rather than fix its trusted publisher

PyPI accepts a pre-release version identifier (0.3.0rc1) on the real
index directly, and pip ignores it without --pre -- there was never a
need to route a release candidate to a separate staging index to hold
it safely. Test PyPI existed here only as that safety net.

publish-to-pypi now runs unconditionally instead of skipping for a
release candidate, which also makes this consistent with what npm and
Maven Central already do. publish-dev-to-testpypi.yml is deleted --
it's been failing on every push to main since the 2026-08-16 migration
anyway (see reqstool/.github#92), and there's nowhere left for it to
publish to.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

* fix(ci): pin the publish-to-pypi action reference to a commit

CodeQL flagged actions/unpinned-tag: @main is a mutable ref, so a
compromised or force-pushed reqstool/.github main would execute
through this reference with no review. Pinned to the commit main
pointed at when this action was added, matching the org's existing
`@<sha> # main YYYY-MM-DD` convention for third-party/internal action
references elsewhere in this repo.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

---------

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
jimisola added a commit to reqstool/reqstool-python-hatch-plugin that referenced this pull request Aug 23, 2026
* fix(ci): stop publishing to PyPI through a reusable workflow

PyPI does not accept a reusable workflow as a trusted publisher --
pypi/warehouse#11096, unresolved. Both publish jobs here called
reqstool/.github's python-publish-to-pypi.yml via workflow_call, so
every real publish attempt failed at the OIDC exchange with
invalid-publisher regardless of what the trusted publisher config
named -- confirmed today, and it explains why publish-dev-to-testpypi.yml
has been failing on every push to main since the 2026-08-16 migration.

Switches both jobs to reqstool/.github's new actions/publish-to-pypi
composite action, called directly from a job defined in this workflow.
A composite action doesn't have the same problem: the OIDC claim is
about which workflow file the job runs in, not what its steps reference.

The two-index environment derivation (testpypi -> test, pypi -> stable)
that the reusable workflow computed internally now has to be explicit
per job -- a composite action can't set the caller's environment:.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

* fix(ci): drop Test PyPI entirely rather than fix its trusted publisher

PyPI accepts a pre-release version identifier (0.3.0rc1) on the real
index directly, and pip ignores it without --pre -- there was never a
need to route a release candidate to a separate staging index to hold
it safely. Test PyPI existed here only as that safety net.

publish-to-pypi now runs unconditionally instead of skipping for a
release candidate, which also makes this consistent with what npm and
Maven Central already do. publish-dev-to-testpypi.yml is deleted --
it's been failing on every push to main since the 2026-08-16 migration
anyway (see reqstool/.github#92), and there's nowhere left for it to
publish to.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

* fix(ci): pin the publish-to-pypi action reference to a commit

CodeQL flagged actions/unpinned-tag: @main is a mutable ref, so a
compromised or force-pushed reqstool/.github main would execute
through this reference with no review. Pinned to the commit main
pointed at when this action was added, matching the org's existing
`@<sha> # main YYYY-MM-DD` convention for third-party/internal action
references elsewhere in this repo.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

---------

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
jimisola added a commit to reqstool/reqstool-python-decorators that referenced this pull request Aug 23, 2026
* fix(ci): stop publishing to PyPI through a reusable workflow

PyPI does not accept a reusable workflow as a trusted publisher --
pypi/warehouse#11096, unresolved. Both publish jobs here called
reqstool/.github's python-publish-to-pypi.yml via workflow_call, so
every real publish attempt failed at the OIDC exchange with
invalid-publisher regardless of what the trusted publisher config
named -- confirmed today, and it explains why publish-dev-to-testpypi.yml
has been failing on every push to main since the 2026-08-16 migration.

Switches both jobs to reqstool/.github's new actions/publish-to-pypi
composite action, called directly from a job defined in this workflow.
A composite action doesn't have the same problem: the OIDC claim is
about which workflow file the job runs in, not what its steps reference.

The two-index environment derivation (testpypi -> test, pypi -> stable)
that the reusable workflow computed internally now has to be explicit
per job -- a composite action can't set the caller's environment:.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

* fix(ci): drop Test PyPI entirely rather than fix its trusted publisher

PyPI accepts a pre-release version identifier (0.3.0rc1) on the real
index directly, and pip ignores it without --pre -- there was never a
need to route a release candidate to a separate staging index to hold
it safely. Test PyPI existed here only as that safety net.

publish-to-pypi now runs unconditionally instead of skipping for a
release candidate, which also makes this consistent with what npm and
Maven Central already do. publish-dev-to-testpypi.yml is deleted --
it's been failing on every push to main since the 2026-08-16 migration
anyway (see reqstool/.github#92), and there's nowhere left for it to
publish to.

Needs reqstool/.github#92 merged first.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

* fix(ci): pin the publish-to-pypi action reference to a commit

CodeQL flagged actions/unpinned-tag: @main is a mutable ref, so a
compromised or force-pushed reqstool/.github main would execute
through this reference with no review. Pinned to the commit main
pointed at when this action was added, matching the org's existing
`@<sha> # main YYYY-MM-DD` convention for third-party/internal action
references elsewhere in this repo.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

---------

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
jimisola added a commit that referenced this pull request Aug 23, 2026
…on (#94)

* fix(ci): stop nesting pypa/gh-action-pypi-publish in a composite action

Confirmed against four real release runs today: pypa/gh-action-pypi-publish
is a Docker container action, and GitHub resolves its image using the
wrapping action's own repository and ref rather than the Docker action's
when nested inside another `uses:` -- every publish failed with `docker:
invalid reference format`, trying to pull
ghcr.io/reqstool/.github:<this-action's-own-sha>. The action's own
maintainers say this usage is untested and unsupported.

This is exactly what the workaround comment linked from #92's PR
description already said -- upload as a bare step in the caller, not
routed through anything else -- and actions/publish-to-pypi violated it
by wrapping the publish step instead of just inlining it. Deleting the
composite action rather than trimming it to only download-artifact: what
remains is one actions/download-artifact call, not enough indirection to
be worth a shared action, and every layer added here tonight has broken
in a new way.

Companion PRs inline both steps directly into each PyPI-publishing repo's
release.yml.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

* refactor(ci): keep the composite action, drop only the publish step

Reworks the previous commit, which deleted the action outright on the
mistaken premise that composite actions were the problem. They are not:
a composite action creates no new workflow context, so the job keeps the
caller's OIDC identity -- which is exactly why it works where a reusable
workflow does not. Someone makes the same point for Ruby in
pypi/warehouse#11096.

Only pypa/gh-action-pypi-publish has to come out. It is a Docker
container action, and GitHub resolves a nested Docker action's image
against the wrapping action's repository rather than its own -- the
action's own maintainer describes the same breakage in that thread.

Renamed publish-to-pypi -> download-dists, since what remains downloads
the distributions and deliberately does not publish them. Keeping the
old name would have been the misleading part.

RELEASING.md's PyPI notes corrected while here. Two claims in the
previous commit were wrong: PyPI matches `job_workflow_ref`, the
bottom-most workflow, not the top-level caller; and reusable workflows
are not rejected categorically -- one in the *same* repo as its caller
works by accident of that same rule. What fails is a cross-repo one like
this repo's, which puts reqstool/.github in the claim where the calling
project has to be.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>

---------

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants