Skip to content

Harden the CI harness itself: pip-audit, workflow linting, SHA pins - #245

Merged
danielrmerskine merged 1 commit into
mainfrom
de/ci-supply-chain
Aug 13, 2026
Merged

Harden the CI harness itself: pip-audit, workflow linting, SHA pins#245
danielrmerskine merged 1 commit into
mainfrom
de/ci-supply-chain

Conversation

@danielrmerskine

Copy link
Copy Markdown
Collaborator

The workflows gate everything else, but nothing gated the workflows -- and Python had no
in-gate dependency scan while Rust has cargo-audit. Four hardening changes, each measured
against the tree first so everything lands green:

pip-audit job (new, in static-checks.yml): audits the resolved CI environment
(cli + [mcp] extra + unit-test requirements) on every PR. Deliberately a separate job,
not a step inside the required static-checks context: it depends on the PyPI advisory
service, and that external flake must not couple into a required check until the job earns
trust during soak. tools/pip_audit_ignore.txt starts EMPTY -- the tree resolved clean.
The job upgrades the runner's seeded pip/setuptools before auditing (their advisories are
about the image, not this project). Scope stated honestly in the header: this covers what
CI resolves; the box runtime resolves unpinned at image build and needs its own audit
(follow-up).

zizmor (pinned, offline mode, findings at low+ fail) and actionlint (pinned,
shellcheck/pyflakes integrations disabled -- this job already applies the repo's chosen
shell floors) now run inside the required static-checks job. Deliberate exceptions carry
reasons: .github/zizmor.yml (the nightly's secrets: inherit to the bench workflows),
.github/actionlint.yaml (the lager-bench runner label).

SHA pins + credential hygiene: every third-party action pinned to a commit SHA with a
version comment (dependabot's github-actions ecosystem keeps them moving), and every
checkout sets persist-credentials: false -- nothing in these workflows pushes.

Default token permissions: every workflow now declares permissions: contents: read;
jobs needing more must say so themselves. Compatible with the reusable-workflow chain in
both directions.

MERGE ORDER NOTE: this PR should merge LAST of the currently open CI series (#231, #234,
#243, #244, and the Rust gate rework). Those add or touch workflow files with tag-pinned
actions; once zizmor gates here, unpinned refs go red. Before merging this one, it gets
rebased with a final pin sweep over whatever landed.

Validated locally: zizmor and actionlint exit 0 against this branch with the configs;
pip-audit runs clean over the freshly resolved environment (zero known vulnerabilities;
the ignore-file parsing exercised with a fake entry); all six workflows YAML-parse and
every embedded run block passes bash -n; run_pip_audit.sh is shellcheck-clean at the
gate's own floor. Bench workflow edits are trigger-neutral; a supervised
workflow_dispatch of integration-tests from this branch is the pre-merge validation.

The workflows gate everything else but nothing gated the workflows, and
Python dependencies had no in-gate vulnerability scan while Rust had
cargo-audit. Four changes, all measured against the tree before landing
so the gate is green on day one:

pip-audit job (static-checks.yml): audits the resolved CI environment
(cli + [mcp] extra + unit-test requirements) on every PR. A separate
job, not a step in the required static-checks context -- it depends on
the PyPI advisory service, and external flake must not couple into a
required check until it earns trust. The ignore file
(tools/pip_audit_ignore.txt) starts EMPTY: the tree resolved clean.
Stated honestly in the job header: this covers what CI resolves, not
what the box runtime resolves unpinned at image build -- that audit is
follow-up work. The env's own seeded pip/setuptools are upgraded before
auditing, so findings are about this project's tree, not the runner
image's bootstrap tooling.

zizmor (pinned, offline, low+ fails) and actionlint (pinned, its
shellcheck/pyflakes integrations disabled -- this job already applies
the repo's own chosen floors to shell) run inside the required
static-checks job. Deliberate exceptions carry reasons in
.github/zizmor.yml (the nightly's secrets: inherit) and
.github/actionlint.yaml (the bench runner label).

Every third-party action is SHA-pinned with a version comment, and
every checkout sets persist-credentials: false -- no job here pushes,
so the token has no business surviving into the workspace. The
dependabot github-actions ecosystem keeps the pins moving.

Every workflow now carries a default-token permissions block
(contents: read); jobs needing more must say so themselves. For the
reusable-workflow chain this is compatible both ways: the callers'
default caps the called workflows at exactly what they declare.

Bench workflows are touched (pins, permissions, persist-credentials)
but trigger-neutrally; validated by a supervised dispatch before merge.
@danielrmerskine

Copy link
Copy Markdown
Collaborator Author

Rebuilt on top of the merged series (#231, #234, #243, #244, #246, #252): the branch was
reset to main and the sweep re-applied fresh, so the pins and permissions now cover all
nine workflow files including the ones that merged after this PR first opened
(nightly-bench's notify jobs, bench-watchdog, packaging, release-validation, the reworked
rust-checks, and the mcp-extra resolution step). 13 checkouts, 9 setup-pythons, cache and
upload-artifact all SHA-pinned with version comments; zero tag refs remain;
persist-credentials off everywhere; every workflow carries a default-token permissions
block.

Validated against the full tree: zizmor exits clean at low+ with only the three
configured exceptions (nightly's secrets: inherit, the bench runner label), actionlint
clean, every embedded run block passes bash -n, pip-audit still resolves the tree with
zero known vulnerabilities.

Remaining before undraft: one supervised workflow_dispatch of integration-tests from
this branch to prove the bench workflow edits are behavior-neutral on real hardware.

@danielrmerskine
danielrmerskine marked this pull request as ready for review August 13, 2026 20:35
@danielrmerskine
danielrmerskine merged commit 93fba95 into main Aug 13, 2026
19 checks passed
@danielrmerskine
danielrmerskine deleted the de/ci-supply-chain branch August 13, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant