Skip to content

feat(ci): add the GARM E2E driver and its credential path (ISD-5876) - #316

Draft
cbartz wants to merge 1 commit into
mainfrom
feat/garm-e2e-driver-ISD-5876
Draft

feat(ci): add the GARM E2E driver and its credential path (ISD-5876)#316
cbartz wants to merge 1 commit into
mainfrom
feat/garm-e2e-driver-ISD-5876

Conversation

@cbartz

@cbartz cbartz commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #315. Review that one first; this PR's diff shows only its own changes once
#315 merges and GitHub retargets the base to main.

What this PR does

Adds garm_e2e.yaml, the driver for the GARM end-to-end test, and the credential path
it needs: the OpenStack username and password from Vault via AppRole, the rest of the
tenant settings and a dedicated GitHub App from repository secrets. It runs on the
private-endpoint runner, is triggered manually, and is not a merge gate.

Also adds the tox -e garm-e2e environment, the charms/tests/e2e/ suite directory, and
a stub test that asserts only that the credentials reach pytest. The end-to-end test
itself follows in the next PR in the stack.

Why we need it

The credential path is the part most likely to be got wrong and the part that is
worthless to review in the abstract, so it lands on its own and is proven against the
real infrastructure before anything is built on top of it. That already paid for itself:
it surfaced a Vault KV v1/v2 response-shape mismatch and two derived values that the
runner's automatic masking does not cover.

Landing it also registers its workflow_dispatch, which gives the end-to-end
implementation an on-demand --ref loop on the private-endpoint runner instead of a
trigger that fires on every push to a scarce shared resource.

Test plan

  • Run on the real infrastructure, green:
    https://github.com/canonical/github-runner-operators/actions/runs/32379213143

    Every step passed on the private-endpoint runner: the tenant settings and GitHub App
    loaded from secrets, the OpenStack username and password came from Vault, a Keystone
    token was issued against the tenant, and the suite reported 11 passed. The run also
    confirms the declared token scope is what the job actually gets — the log records
    Contents: read / Metadata: read and nothing more. It was triggered by a temporary
    label-gated pull_request trigger, since removed. The content here is unchanged from
    the revision that produced that run.

  • The log was audited for leaked credentials: no PEM markers, no Vault token, no
    rendered config, no base64 run long enough to be the encoded private key, and no Vault
    hostname — that last one matters because the workflow normalises the address with
    rstrip("/"), and masking matches literal substrings, so the derived form is masked
    explicitly rather than relying on the runner. The Vault step's entire output is one
    line: Tenant credentials fetched and masked. Every occurrence of a credential
    variable elsewhere is *** or a bare variable name, including the pytest parametrize
    IDs, which print a setting's name and never its value.

  • The Vault step was exercised against a stand-in server for every response shape it can
    meet: KV v2, KV v1, an empty secret, one missing a field, and a 403 — confirming it
    authenticates in the first two and fails without writing a partial environment in the
    rest.

  • Private-key normalisation was verified against a generated RSA key: a pasted PEM, a
    pre-encoded value, and a double-normalised value all reduce to one line and decode to a
    key AppAuth accepts.

  • The stub test passes with all eleven settings present and fails naming only the
    variable when one is missing — verified that a sentinel password in the environment
    appears nowhere in a failing run's output.

  • scripts/check_agents_md.py passes; every run block passes bash -n.

Review focus

  • Credential handling is the constraint that shaped the rest. Values read from Vault
    are ::add-mask::ed in the step that reads them; the Vault response is parsed in
    memory and never written to a file, so no failure path can leave it on a runner whose
    disk outlives the job; steps use set -euo pipefail and never set -x.
  • Derived values are masked explicitly. Masking matches literal substrings, so a
    value the workflow transforms — the base64 of the private key, the normalised Vault
    address — is not covered by the runner's automatic masking of the secret it came from.
  • The private key is accepted as pasted. A PEM goes into the secret exactly as GitHub
    issues it; the workflow normalises it to one line, because the pytest environment
    template is line-oriented and a multi-line value would break the env file.
  • pytest.fail rather than assert in the stub test is deliberate: pytest's
    assertion rewriting introspects the expression and would dump the whole of
    os.environ into the failure output.
  • All infrastructure detail is a secret, endpoints and project/network/domain names
    included. The tradeoff is that GitHub masks secret values by literal substring, so a
    common value redacts wherever it appears in a log.

Breaking changes

None. The workflow is workflow_dispatch-only and is not in any required status check,
so nothing runs automatically and existing CI is untouched.

New dependencies and workflow changes

  • One new workflow, not wired into a merge gate.
  • New charms/tests/e2e/ suite directory and tox -e garm-e2e environment, kept
    separate from charms/tests/integration/ so the end-to-end test stays out of the PR
    matrix.

Checklist

  • I updated docs/changelog.md with user-relevant changes
  • I used AI to assist with preparing this PR
  • I added or updated tests as needed (unit and integration)
  • If integration test modules are used: I updated the workflow configuration
  • If this PR involves a Grafana dashboard: I added a screenshot of the dashboard
  • If this PR involves Terraform: terraform fmt passes and tflint reports no errors
  • If this PR involves Rockcraft: I updated the version
  • If this PR adds/removes a charm, or changes a charm's base class, conventions, tooling, or repo structure: I updated the relevant AGENTS.md
  • If this PR changes .copilot-collections.yaml or .github/instructions/: I re-checked whether the AGENTS.md "12-factor divergences" guidance still matches the upstream copilot-collections guidance

Notes on the unchecked items:

  • changelog — not needed: CI-only scaffolding with no user-facing change.
  • integration test modules — this repo drives integration tests through spread rather
    than a modules list, and the end-to-end suite is deliberately excluded from that
    matrix. spread.yaml is untouched here; it changes in the next PR in the stack.
  • Grafana, Terraform, Rockcraft and copilot-collections items do not apply.

@cbartz
cbartz force-pushed the feat/garm-e2e-driver-ISD-5876 branch from 8a12890 to 3da80f1 Compare August 21, 2026 05:48
@cbartz
cbartz force-pushed the feat/garm-e2e-driver-ISD-5876 branch 4 times, most recently from 760778f to 9fa3f87 Compare August 21, 2026 06:24
@cbartz
cbartz force-pushed the feat/garm-e2e-driver-ISD-5876 branch from 9fa3f87 to b46b19a Compare August 21, 2026 06:25
Base automatically changed from feat/garm-e2e-skeleton-ISD-5876 to main August 21, 2026 07:20
@cbartz
cbartz force-pushed the feat/garm-e2e-driver-ISD-5876 branch from 718503b to de0b4d4 Compare August 21, 2026 07:40
Resolves everything the end-to-end test needs to authenticate: the OpenStack
username and password from Vault via AppRole, the rest of the tenant settings and
a dedicated GitHub App from repository secrets. Runs on the private-endpoint
runner, triggered manually, and is not a merge gate. The stub test asserts only
that the credentials reach pytest; the suite that uses them follows.

Split out of the dispatch-target PR so that file could be reviewed on its own.
The content is unchanged from the revision verified green on the private-endpoint
runner in run 32379213143.
@cbartz
cbartz force-pushed the feat/garm-e2e-driver-ISD-5876 branch from de0b4d4 to b98d408 Compare August 21, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant