feat(skills): gate deploys on catalogue consistency and SkillSpector scan

[DevelopmentCats]

Summary

Adds a two-stage gate in front of every registry deploy so the agent-skills
catalogue cannot ship broken or unsafe content:

1. Catalogue consistency. Extends cmd/readmevalidation with a
   READMEVALIDATION_ONLINE=1 mode that shallow-clones every declared
   owner/repo@ref, verifies every sources[].skills slug exists upstream
   as skills/<slug>/SKILL.md, validates each SKILL.md frontmatter
   against the agentskills.io v0.2.0 specification (required name and
   description), and rejects duplicate slugs across sources within the
   same namespace. Strict gate, blocks on failure.
2. Security scan. Runs NVIDIA SkillSpector
   (static-only, --no-llm) over each unique upstream source repo and
   uploads SARIF to GitHub Code scanning. Warn-only gate for now, see
   "Initial scan policy" below.

Sequential gating: offline catalogue checks → online catalogue checks →
SkillSpector. If anything fails (including the SkillSpector gate once it
is flipped to strict), the deploy's gcloud builds triggers run step
never fires and a single rolling tracker issue (label skills-gate) is
opened or updated by JasonEtco/create-an-issue, so repeat failures
update the same issue rather than spamming.

Initial scan policy: warn-only

SkillSpector flags 13 error-level findings against coder/skills@main
on the first run, all in the intentionally credential-handling setup
skill:

• 5x PE3 Credential Access in skills/setup/scripts/github-device-fetch.sh
• 1x PE3 Credential Access in skills/setup/scripts/github-device-poll.sh
• 1x E2 Env Variable Harvesting in skills/setup/references/coder-agents.md
• 2x PE3 Credential Access in test/claude.sh, test/codex.sh
• 4 others on the same setup files

These are real pattern matches but expected behavior for an installer
skill that orchestrates a GitHub device-flow login. Rather than
rubber-stamping them or holding this PR for upstream cleanup, the
severity step is shipped as continue-on-error: true. Findings still
flow to Code scanning so they show up as PR annotations and Security
tab alerts. A follow-up PR can:

• Tune coder/skills (add SkillSpector ignore comments, move test
  scripts out of the scan path, restructure the device-flow helpers).
• Flip continue-on-error: false in both ci.yaml and
  deploy-registry.yaml to make the gate strict.

What this solves

The registry-server build pipeline currently drops catalogue overrides
for upstream slugs it cannot find, silently. Upstream rename, deletion,
or branch-flip is invisible until a contributor notices a 404. This PR
makes the gap loud and blocks the deploy on the consistency check.

What this does NOT solve

• Immutability. Sources are still @main. Pinning to SHAs or tags
  depends on coder/registry-server PR chore: remove it wrappers from required variables tests  #442 follow-up chore: add CI step for validating contributor READMEs #1 landing first
  (the cloneSkillSourceRepo branch-only bug).
• Race window. The gate scans upstream at CI time; the deploy clones
  upstream at deploy time. A future follow-up can move the scan inside
  the deploy job to close it further.

Files

Path	Change
cmd/readmevalidation/coderskills.go	New online verification path (validateAllCoderSkillsOnline, fetchSkillsSourceContent, verifyReadmeAgainstSources, validateSkillMarkdownFrontmatter). Factors validateAllCoderSkills to expose parsed readmes. Uses the system git binary via exec.Command to avoid pulling in a Go Git library.
cmd/readmevalidation/main.go	Runs the online check after the offline check when READMEVALIDATION_ONLINE=1.
cmd/readmevalidation/readmefiles.go	Adds validationPhaseUpstream.
scripts/scan-skill-sources.sh	Iterates skills READMEs, extracts unique repo@ref tuples, runs SkillSpector per source, writes SARIF. Treats missing-SARIF as the only true tool crash; severity gating lives in the workflow.
.github/workflows/ci.yaml	dorny/paths-filter on validate-readme-files + new scan-skills job, both skipped on PRs that do not touch skills paths. SkillSpector severity step is continue-on-error: true.
.github/workflows/deploy-registry.yaml	Refactored into verify → deploy + open-issue-on-failure. Same SkillSpector severity policy as ci.yaml.
.github/ISSUE_TEMPLATE/skills-failure.md	Single tracker template used by JasonEtco/create-an-issue with update_existing: true.

Verification

Local, before pushing, against current coder/skills@main:

$ go build ./cmd/readmevalidation
$ READMEVALIDATION_ONLINE=1 ./readmevalidation
...
[info]  verifying upstream skill sources  num_unique_sources=1
[info]  upstream skill source verification passed  num_files=1  num_sources=1

Failure path (injected nonexistent-slug into sources[].skills):

exit=1
[erro]  Error during "Upstream skill source verification" phase of README validation:
   - "registry/coder/skills/README.md": sources[0].skills["nonexistent-slug"]:
     slug not found upstream at coder/skills/skills/nonexistent-slug/SKILL.md
     (repo=coder/skills ref=main)

CI status on this PR: all seven checks green, including the new
Validate README files (online catalogue check), Scan skill sources with SkillSpector (warn-only gate, SARIF uploaded), and skillspector
(Code scanning ingest). Validate Terraform output flaked twice on
registry.terraform.io provider downloads (Error while installing coder/coder v2.18.0: ... failed to retrieve authentication checksums for provider); passed on the third retry.

Open follow-ups

• Move SkillSpector from a git-pinned commit to a tagged release when
  one is available. Currently pinned at NVIDIA/SkillSpector
  2eb844780ab163f01468ecf142c40a2ec0fcaec0; SkillSpector is not on
  PyPI yet.
• Triage the warn-only SkillSpector findings in coder/skills@main,
  then flip continue-on-error: false in both workflows.
• Once cloneSkillSourceRepo in coder/registry-server accepts tags
  and SHAs, switch sources from @main to immutable refs and add a
  Renovate-style bumper.

Implementation plan and decision log

The full plan that drove this PR (option space, the choice to shell out
to git rather than vendor go-git, why we extend cmd/readmevalidation
instead of adding a new binary, severity policy reasoning) lives in
plans/skills-scan-pr-plan.md and is available on request.

This PR was created with help from Coder Agents.

[github-advanced-security]

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

• The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
• Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
• You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

[bpmct]

how does this scanning help us if its at deploy time and there are dynamically changing repos?

[DevelopmentCats]

Member

Its probably not the most elegant solution, but the idea is that anytime we would deploy the registry it would run this skill verification on each skills repo and if it fails the deployment will just fail and create an issue. I was fiddling with some stuff while researching on skill scanning last night and how we would implement this, and this is a WIP currently