Skip to content

ci: support full validation on workflow_dispatch (no PR context)#181

Merged
felickz merged 3 commits into
mainfrom
ci/workflow-dispatch-full-validation
Jul 11, 2026
Merged

ci: support full validation on workflow_dispatch (no PR context)#181
felickz merged 3 commits into
mainfrom
ci/workflow-dispatch-full-validation

Conversation

@felickz

@felickz felickz commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

What

Fixes ci.yml ("Build CodeQL Packs") so it can be run via workflow_dispatch
on any branch and get a real, full compile-and-test validation - without
requiring an open pull request.

Why

While validating the CLI 2.24.3 bump (#179 / #180), we discovered ci.yml
silently assumed every run has a PR context (github.event.number). For
workflow_dispatch runs this value is empty, which broke things two ways:

  1. Crash before compiling anything. pr-compile.sh/pr-suites-packs.sh
    were invoked with unquoted ${{ github.event.number }}. When empty, bash
    word-splitting drops the argument entirely, shifting LANGUAGE into
    position $1 and leaving $2 unset - set -u then throws
    unbound variable immediately, before codeql query compile ever runs.
  2. Steps silently skipped. Every step is gated on
    steps.changes.outputs.src == 'true' from dorny/paths-filter, which is
    PR-diff-based and not meaningful for a manual "verify everything" run.

This is exactly the gap that let PR #179 get merged on the strength of the
wrong CI signal (the repo's own CodeQL self-scan Analyze jobs looked green,
but the real compile-and-test job never actually re-ran on the fix commit).

Changes

  • Quote the PR-number/language args passed to pr-compile.sh and
    pr-suites-packs.sh so an empty value is passed as an explicit empty
    string instead of vanishing.
  • Add || github.event_name == 'workflow_dispatch' to every
    steps.changes.outputs.src == 'true' condition (12 occurrences across
    compile-and-test, extensions, library-sources, and configs jobs)
    so a manual dispatch always runs every step regardless of the path filter.
  • Give pr-compile.sh, pr-suites-packs.sh, and pr-configs.sh a "full
    mode" fallback that activates whenever PR_NUMBER is empty: instead of
    gh pr view/gh pr comment (which require a PR), each script enumerates
    the relevant files directly from the filesystem and skips PR-comment
    logic. Kept as single dual-mode scripts (not separate PR/non-PR files) to
    avoid future logic drift.

Validation plan

  • This PR itself will exercise the existing PR-based path unchanged
    (real pull_request event) - confirms no regression.
  • After this is up, we'll workflow_dispatch this branch directly to prove
    the new non-PR path actually compiles all 7 languages without
    crashing.

The Build CodeQL Packs workflow (ci.yml) previously assumed every run had
a PR context (github.event.number). For workflow_dispatch runs this value
is empty, which caused two problems:

1. Unquoted script arguments (pr-compile.sh, pr-suites-packs.sh) meant an
   empty github.event.number was dropped entirely by bash word-splitting,
   shifting the language argument into position 1 and leaving position 2
   unset - crashing immediately under set -u ("unbound variable") before
   any query compilation happened.
2. Every step was gated on dorny/paths-filter's src output, which is
   PR-diff-based and not meaningful for a manual "verify everything" run.

Fixes:
- Quote the PR-number/language args passed to pr-compile.sh and
  pr-suites-packs.sh so an empty value is passed as an explicit empty
  string instead of vanishing.
- Add || github.event_name == 'workflow_dispatch' to every
  steps.changes.outputs.src == 'true' step condition (12 occurrences
  across compile-and-test, extensions, library-sources, and configs jobs)
  so a manual dispatch always runs every step regardless of path-filter.
- Give pr-compile.sh, pr-suites-packs.sh, and pr-configs.sh a "full mode"
  fallback that activates whenever PR_NUMBER is empty: instead of calling
  gh pr view / gh pr comment (which require a PR), each script now
  enumerates the relevant files directly from the filesystem (full
  --warnings=error compile of the language, find over suites/*.qls,
  glob over configs/*.yml) and skips any PR-comment logic. This keeps a
  single dual-mode script per concern instead of forking separate
  PR/non-PR script files, avoiding future logic drift between the two
  paths.

This lets ci.yml be dispatched manually on any branch (including main)
to get a real, full compile-and-test validation without requiring an
open pull request - closing the validation gap that let PR #179 get
merged on the strength of the wrong CI signal.
Copilot AI review requested due to automatic review settings July 11, 2026 03:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the “Build CodeQL Packs” CI workflow and its helper scripts so the workflow can be run via workflow_dispatch on any branch and still perform full compile/test validation without requiring a pull request context.

Changes:

  • Adjusts ci.yml step gating so manual dispatch runs all relevant steps regardless of paths-filter output.
  • Quotes PR number/language args passed into CI scripts to avoid argument shifting when PR context is missing.
  • Adds “no PR number” fallback logic to the PR-oriented scripts to enumerate files/configs from the filesystem instead of using gh pr view and PR comments.
Show a summary per file
File Description
.github/workflows/ci.yml Allows workflow_dispatch to run the full validation path by bypassing PR-diff-based gating and quoting script arguments.
.github/scripts/pr-suites-packs.sh Adds a non-PR mode that resolves all suites and rebuilds packs directly from the working tree.
.github/scripts/pr-configs.sh Adds a non-PR mode that validates all configs/*.yml directly, and refactors compilation into a helper function.
.github/scripts/pr-compile.sh Adds a non-PR mode for full strict compilation, but currently performs redundant compilation work (see PR comment).

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Low

Comment thread .github/scripts/pr-compile.sh
felickz and others added 2 commits July 10, 2026 23:56
…LANGUAGE

codeql pack install/create requires a directory containing a qlpack.yml
directly. No language has a top-level qlpack.yml (only nested src/, lib/,
ext/, ext-library-sources/) so `codeql pack install "$LANGUAGE"` fails
with "ERROR: The directory is not a qlpack." Loop over the actual pack
subdirectories instead, matching what codeql-workspace.yml's
`**/qlpack.yml` globs actually resolve to.

Caught by dispatching this branch's workflow_dispatch run directly -
cpp/go/ruby all failed identically before this fix.
…klog

publish.yml never runs --warnings=error (only `codeql pack install`/`publish`,
which don't treat warnings as fatal), so full-mode's strict recompile of the
entire language directory was a brand-new, stricter check than what actually
gates real publishing - and it tripped on 5 pre-existing deprecated-API
warnings (getResult, getAUse, SsaVariable) that no PR-mode diff has ever
caught because no PR has ever touched all of them at once.

The plain compile check above already covers the whole language directory
without treating warnings as fatal, so full-mode is done once that passes.
Filed a follow-up issue to clean up the warning backlog and then make
PR-mode itself trigger a full strict compile when a PR touches
.codeqlversion or a codeql-pack.lock.yml, since a dependency/CLI bump can
change behavior across every query, not just the files literally edited.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@felickz felickz merged commit 1b8eaba into main Jul 11, 2026
30 checks passed
@felickz felickz deleted the ci/workflow-dispatch-full-validation branch July 11, 2026 04:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants