Skip to content

ci: route own-PR checks to an opt-in self-hosted runner - #6277

Open
pedrofrxncx wants to merge 1 commit into
mainfrom
ci/local-runner-routing
Open

ci: route own-PR checks to an opt-in self-hosted runner#6277
pedrofrxncx wants to merge 1 commit into
mainfrom
ci/local-runner-routing

Conversation

@pedrofrxncx

@pedrofrxncx pedrofrxncx commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Makes the five bun-only check jobs resolve their runner from repo variables, defaulting to the hosted runner used today. Both variables unset = no-op, so merging this changes nothing.

Complements #5828 rather than replacing it: that PR targets an ephemeral Linux ARC pool for the whole repo; this one routes one person's own PRs to a machine they own. Different trust model, hence the extra gate. If both land, the expressions want a trivial reconcile.

What moves

job runner why
format, lint, typecheck, test, build needs.changes.outputs.runner bun-only — no Docker, no service containers, no apt
changes stays hosted it elects the runner, so it can't depend on one
web-component-tests stays hosted apt-mirror-fallback deps step is Linux-only
everything in e2e.yml untouched see "Why not e2e" below

The choice is resolved once in changes and read as needs.changes.outputs.runner. Five copy-pasted runs-on: expressions would be five places for the gate to drift out of sync; one is auditable.

Why these jobs

Measured across 146 runs (the same dataset as #5828):

job exec (median) queue (p90)
format 0.4 min 4.9 min
lint 0.5 min 6.5 min
typecheck 0.9 min 4.7 min
test 1.1 min 6.3 min
build 1.8 min 5.2 min

Queue p50 across the repo is healthy (6s). The tail is not: 15% of runs exceed 20 min and ~59% of that wall clock is queue, p99 ~90 min. These jobs spend 3-10× longer waiting than working. A runner that is always idle turns that into ~0.

Why not e2e

e2e-shard needs Postgres + NATS + MinIO + 4 bun processes + 4 chromium workers per shard, and there are 2 shards. That fits a Linux VM at roughly hosted parity (ubuntu-latest is 4 vCPU / 16 GB), but two shards concurrently need ~24 GB, so on a 16 GB box they serialize:

  • hosted today: queue p50 3m56s + exec p50 5m25s ≈ 9m21s (shards parallel)
  • one 16 GB self-hosted box: ~0 queue + 2 × 5m25s ≈ 10m50s (shards serial)

A loss at the median, a win only in the tail. Not worth the Docker-in-VM build-out, so e2e stays hosted where it is free and genuinely parallel. Revisit if a runner with >32 GB shows up.

Security

The same-repo condition is a load guard, not a security boundary, and the comment in the workflow says so. pull_request evaluates the workflow file from the merge ref, so a fork PR can rewrite runs-on and target the runner regardless of what this file says. The gate exists to keep teammates' runs off a personal machine, not to stop an attacker.

The control that actually holds is the repo setting Settings → Actions → General → Fork pull request workflows from outside collaborators → "Require approval for all outside collaborators." That must be on before any variable here is set. With it on, the trust set is people with write access to this repo.

Two properties that reduce the blast radius either way:

  • Neither secrets.GITHUB_TOKEN nor any other secret is referenced by the five routed jobs — secrets.GITHUB_TOKEN appears only in changes, which stays hosted.
  • Runners should be registered --ephemeral, so nothing persists between jobs.

Toggling

on:  gh variable set CI_LOCAL_RUNNER --body <runner-label> --repo decocms/studio
     gh variable set CI_LOCAL_RUNNER_ACTOR --body <github-login> --repo decocms/studio
off: gh variable delete CI_LOCAL_RUNNER --repo decocms/studio

Effective on the next run. No PR, no merge queue.

Known trade-off while the variables are on

There is no fallback when a self-hosted runner is unavailable. timeout-minutes bounds execution, not queue wait, so a job pointed at an offline runner sits queued until GitHub's 24h timeout — on exactly the PRs the opted-in person cares most about. Same failure mode #5828 documents for spot reclaims, and the mitigation is the same: unset the variable. Only opt in with a machine that is genuinely always on.

Test notes

  • Workflow YAML parses; job graph and changes outputs verified.
  • The selection logic was checked against 7 cases: both vars unset, opted-in own same-repo PR, fork PR from the owner, a teammate's PR, each variable set without the other, and a push to main with no PR context. Only the second routes to the self-hosted label; every other case yields ubuntu-latest.
  • No behavior change is observable until a variable is set, so this merges dark.

Summary by cubic

Routes five bun‑only CI checks to an opt‑in self‑hosted runner for the PR author; defaults to ubuntu-latest so behavior is unchanged until enabled. This reduces queue time for format/lint/typecheck/test/build while leaving e2e and Linux-only steps on hosted runners.

Scope

  • Affected jobs: format, lint, typecheck, test, build now use needs.changes.outputs.runner.
  • changes stays hosted; web-component-tests and everything in e2e.yml are unchanged.
  • Runner selection is resolved once in changes and reused to keep policy in one place.

Rollout and risks

  • Opt in by setting repo variables: CI_LOCAL_RUNNER= and CI_LOCAL_RUNNER_ACTOR=; unset either to revert on the next run.
  • The same-repo-and-actor gate is a load guard only; keep “Require approval for all outside collaborators” enabled.
  • Routed jobs do not use secrets.GITHUB_TOKEN; prefer registering the runner with --ephemeral.
  • There is no fallback if the self-hosted runner is offline; affected jobs will stay queued until timeout.

Written for commit 40da68e. Summary will update on new commits.

Review in cubic

The five bun-only check jobs (format, lint, typecheck, test, build) execute
in 0.4-1.8 min but wait 4.9-6.5 min at queue p90 — measured across 146 runs,
the hosted concurrency ceiling rather than slow execution.

Resolve the runner once in the `changes` gate and read it as
`needs.changes.outputs.runner`, so the routing policy lives in one auditable
place instead of five copy-pasted `runs-on:` keys.

Both CI_LOCAL_RUNNER and CI_LOCAL_RUNNER_ACTOR unset => always ubuntu-latest,
so this is inert until someone opts in, and unsetting either falls back on the
next run with no PR.

`changes` stays hosted because it elects the runner; web-component-tests stays
hosted because its apt-mirror-fallback deps step is Linux-only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant