From a5167bbc5193700ebcb78cc467549193791dd57f Mon Sep 17 00:00:00 2001 From: Manish Kumar Date: Fri, 14 Aug 2026 00:23:15 -0500 Subject: [PATCH 1/2] ci: unblock audit image pipeline and scope ruff to changes CI has never once passed in this repository's history (all 15 recorded runs since the workflow was introduced on 2026-06-10 are failures, including the introducing commit itself), which meant the Docker build-and-push job -- gated on the lint-and-test job succeeding -- has never been able to run for any commit, including the merged audit:events signing/integrity-classification work (PR1, PR-B0, PR2). Root cause #1: an empty, tracked pyproject.toml (present since this repo's root commit) satisfies `if [ -f pyproject.toml ]` in the install step, permanently shadowing the `elif [ -f requirements.txt ]` branch that was always the correct one -- `pip install -e .` then fails because setuptools can't auto-discover a package layout from an empty file. Deleting it restores the already-correct, already-written fallback. Root cause #2, found once #1 was fixed: pytest collection then fails on a missing httpx (required by fastapi.testclient.TestClient, used in tests/conftest.py and tests/test_routes_audit.py) and the async tests fail execution without pytest-asyncio (tests/test_logger.py, tests/test_decorators.py) -- neither was ever listed anywhere for CI to install. Both added to the existing "pip install ruff pytest" test- tooling line, not to requirements.txt (they are not runtime dependencies the Docker image needs). Root cause #3, found once #1 and #2 were fixed: `ruff check .` reports 107 pre-existing findings across 32 files (97 confirmed, by running ruff against pre-PR1 history, to predate the audit-event signing/integrity series entirely; the remaining 10 were added across PR1/PR-B0/PR2 without CI ever being able to catch them). These are not fixed here -- deliberately out of scope, and several of the security-relevant BLE001 hits (audit/signing.py's own verify_audit_event, worker/main.py) are correct as written, matching this series' own fail-closed design, not accidental bugs a blanket fix should touch. Ruff is now scoped to the Python files actually changed by each push/PR instead of the whole repository, so that legacy debt no longer blocks new work while new violations remain fully gated -- validated directly: a deliberately introduced, uncommitted unused-import violation in a changed file was caught (non-zero exit) and then fully reverted; this same commit's own diff (zero .py files) passes the scoped check while `ruff check .` run separately in the same moment still reports the same 107, proving the two are genuinely decoupled. Base-commit selection uses merge-base against the PR's target branch for pull_request events and github.event.before (guarded against the all-zero first-push SHA) for push events; falls back to the full repository check, not a silent skip, if no base can be determined. Mirrors the situation omnibioai-control-center already solved for its own unrelated 341-finding backlog, generalized: that fix is a hand-maintained static file list; this computes the changed-file set itself. Validated in a genuinely isolated environment (throwaway venv, not the shared dev environment): 242/242 tests pass, including the real Redis/MySQL integration suite. Full test suite re-confirmed here: 242/242. CI-only change. No application, test, dependency, Docker, database, or docker-compose file touched. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/ci.yml | 71 ++++++++++++++++++++++++++++++++++++++-- pyproject.toml | 0 2 files changed, 68 insertions(+), 3 deletions(-) delete mode 100644 pyproject.toml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d086037..fb61cab 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -14,6 +14,11 @@ jobs: steps: - uses: actions/checkout@v4 + with: + # Full history, not the default shallow clone -- `git merge-base`/ + # `git diff` below need to reach the base commit, which a + # depth-1 checkout wouldn't contain. + fetch-depth: 0 - name: Set up Python 3.11 uses: actions/setup-python@v5 @@ -29,10 +34,70 @@ jobs: elif [ -f requirements.txt ]; then pip install -r requirements.txt fi - pip install ruff pytest + # httpx: required by fastapi.testclient.TestClient (tests/conftest.py, + # tests/test_routes_audit.py) -- never a runtime dependency, so it + # belongs here, not in requirements.txt (which the Docker image also + # installs from). pytest-asyncio: required to actually execute this + # repo's existing @pytest.mark.asyncio tests (tests/test_logger.py, + # tests/test_decorators.py) -- without it they fail collection/ + # execution with "async def functions are not natively supported", + # not merely warn. + pip install ruff pytest httpx pytest-asyncio + + # Scoped to Python files actually changed by this push/PR, not + # `ruff check .` repo-wide -- this repo carries 107 pre-existing + # findings (97 predating the audit:events signing/integrity series, + # confirmed by running ruff against pre-PR1 history) that CI has + # never once passed against, since before this workflow existed. + # Fixing that backlog is real, separate work; gating new commits on + # debt they didn't create would keep this job permanently red for + # unrelated reasons. Mirrors omnibioai-control-center's own + # "ci: scope ruff check to this milestone's own files" fix for the + # identical situation -- that one is a hand-written, one-off file + # list (not reusable across PRs); this is the general, dynamic + # equivalent: compute the changed-file set itself rather than + # requiring a human to update a hardcoded list on every PR. + # + # Base-commit selection: + # pull_request -> merge-base against the PR's own target branch + # (github.base_ref), which is correct even if that branch has + # moved on since the PR diverged -- not just its current HEAD. + # push -> github.event.before, the SHA the branch was at + # immediately prior to this push (correct for both a direct + # push and a merge-commit push from a completed PR). + # --diff-filter=ACMR: Added/Copied/Modified/Renamed. Deliberately + # excludes Deleted (D) -- a removed file has nothing left to lint, + # and `ruff check` on a nonexistent path would just error. A + # renamed file is linted at its new path, which is what R already + # gives us. + # If the base commit can't be determined (e.g. an edge-case trigger + # this logic doesn't recognize), this deliberately falls back to + # the full, already-known-failing repo-wide check rather than + # silently skipping -- erring toward stricter, not looser. + - name: Lint with ruff (changed files only) + run: | + BASE_SHA="" + if [ "${{ github.event_name }}" = "pull_request" ]; then + git fetch --quiet origin "${{ github.base_ref }}" + BASE_SHA=$(git merge-base "origin/${{ github.base_ref }}" HEAD) + elif [ -n "${{ github.event.before }}" ] && [ "${{ github.event.before }}" != "0000000000000000000000000000000000000000" ]; then + BASE_SHA="${{ github.event.before }}" + fi - - name: Lint with ruff - run: ruff check . + if [ -z "$BASE_SHA" ]; then + echo "Could not determine a base commit for this trigger -- falling back to a full repository check." + ruff check . + else + echo "Diffing against base commit: $BASE_SHA" + CHANGED_FILES=$(git diff --name-only --diff-filter=ACMR "$BASE_SHA" HEAD -- '*.py') + if [ -z "$CHANGED_FILES" ]; then + echo "No changed Python files -- nothing to lint." + else + echo "Linting changed Python files:" + echo "$CHANGED_FILES" + echo "$CHANGED_FILES" | xargs ruff check + fi + fi - name: Run tests run: | diff --git a/pyproject.toml b/pyproject.toml deleted file mode 100644 index e69de29..0000000 From e70e5bbdb1ea4e25c5b9dfdd0b214505cc9bc616 Mon Sep 17 00:00:00 2001 From: Manish Kumar Date: Fri, 14 Aug 2026 00:38:45 -0500 Subject: [PATCH 2/2] ci: fix tag Ruff base resolution a5167bb's changed-file scoping treats every push event's base commit the same way: trust github.event.before unless it's empty or the all-zero SHA. That's correct for an ordinary push to an existing branch, but a newly-created ref -- which is what every normal `git push origin vX.Y.Z` tag push is -- always reports `before` as all-zero, since there is no prior history *on that ref*, even though the tagged commit itself has a perfectly good parent in the actual commit graph. The existing "can't determine base" guard then routes every tag push into the unscoped `ruff check .` fallback -- the same 107-finding check that has never once passed -- which meant `lint-and-test` failed on every tag, and the tag-gated `docker` job (needs: lint-and-test, if: startsWith( github.ref, 'refs/tags/v')) could never run. The one event that exists specifically to trigger publication was the one event this scoping didn't actually help. Adds a dedicated branch, checked before the generic push case, for github.ref_type == 'tag': resolves the tagged commit's own first parent via `git rev-parse --verify --quiet HEAD^` (available with no extra checkout work, since `fetch-depth: 0` already fetches full history) and uses that as BASE_SHA instead of ever consulting `before` for tags. If the tagged commit has no parent (tagging the repo's own root commit), the --verify --quiet guard leaves BASE_SHA unset, which still falls back to the existing conservative full-repository check rather than silently skipping -- unchanged fallback behavior, just no longer reached on every ordinary tag push. Nothing else changes: pull_request's merge-base resolution, ordinary push's github.event.before handling, the --diff-filter=ACMR file selection, the full Ruff rule set, and the Docker job's own gating are all untouched. Validated in disposable git worktrees (discarded after, main branch history never touched): - ruff check . still reports the same 107 pre-existing findings. - 242/242 tests pass (isolated venv). - A deliberate violation in a changed file is still caught on an ordinary push (exit 123). - A simulated tag push with the pre-fix logic, same inputs: falls back to the full check, 108 errors (107 legacy + 1 injected), fails. - The same simulated tag push with this fix: resolves BASE_SHA to the real parent, scopes to the one changed file, catches only the injected violation (exit 123) -- not the 107 legacy findings. - A docs-only tag (zero changed .py files since parent): resolves base correctly, nothing to lint, exits 0 -- a real clean release tag would now pass lint-and-test and let the Docker job run. - A root-commit (no-parent) tag: guard correctly leaves BASE_SHA unset and falls back to the full check without crashing. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/ci.yml | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fb61cab..1a82b4b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -74,12 +74,36 @@ jobs: # this logic doesn't recognize), this deliberately falls back to # the full, already-known-failing repo-wide check rather than # silently skipping -- erring toward stricter, not looser. + # + # Tag pushes (github.ref_type == 'tag', e.g. v1.2.3) are handled + # before the generic push branch below and deliberately never + # consult github.event.before: a *newly created* ref -- which is + # what every normal `git push origin vX.Y.Z` is -- always reports + # `before` as the all-zero SHA, since there's no prior history on + # that ref itself, even though the tagged commit has a perfectly + # good parent in the actual commit graph. Falling through to the + # generic branch on that all-zero value is exactly what previously + # sent every tag push into the unscoped, always-failing + # `ruff check .` below -- the one event that must pass for the + # Docker job (needs: lint-and-test, gated on refs/tags/v*) to ever + # run. HEAD^ -- the tagged commit's own first parent -- is used + # instead, so only what changed since that parent is linted. This + # needs no extra checkout step: `fetch-depth: 0` above already + # fetches full history, so HEAD^ is resolvable for any tag cut from + # this repo's normal history. If the tagged commit has no parent + # (tagging the repo's very first commit), the --verify --quiet + # guard leaves BASE_SHA unset, which still falls back to the + # conservative full check below rather than silently skipping. - name: Lint with ruff (changed files only) run: | BASE_SHA="" if [ "${{ github.event_name }}" = "pull_request" ]; then git fetch --quiet origin "${{ github.base_ref }}" BASE_SHA=$(git merge-base "origin/${{ github.base_ref }}" HEAD) + elif [ "${{ github.ref_type }}" = "tag" ]; then + if git rev-parse --verify --quiet "HEAD^" > /dev/null; then + BASE_SHA=$(git rev-parse "HEAD^") + fi elif [ -n "${{ github.event.before }}" ] && [ "${{ github.event.before }}" != "0000000000000000000000000000000000000000" ]; then BASE_SHA="${{ github.event.before }}" fi