Skip to content

CI: static lint pipeline + strict JSON Schemas for state.json and sibling artifacts #55

@arnaudlh

Description

@arnaudlh

Background

Today the PR pipeline only checks docs staleness, workflow lint (actionlint), and plugin version drift. It does not validate that:

  • Bash scripts (.github/scripts/, .github/skills/**/scripts/) are shellcheck-clean
  • YAML files conform to a consistent style
  • Markdown is consistent
  • The seven JSON artifact types Git-Ape emits per deployment match a documented shape:
    • state.json, metadata.json, requirements.json, cost-estimate.json, security-gate.json, policy-recommendations.json, parameters.json

Real-world drift exists. Inspecting an existing artifact corpus (e.g. Azure/git-ape-private) shows two distinct, incompatible shapes for security-gate.json shipping under the same claimed schemaVersion: "1.0":

  • Boolean form (older): criticalPassed: true, highPassed: true
  • Count form (newer): criticalTotal: 5, criticalPassed: 5, highTotal: 5, highPassed: 5

Same field name, different type. Only state.json carries schemaVersion at all today.

Goal

Land a tiered static-validation pipeline so every PR catches the broadest set of regressions before merge — without requiring a sandbox subscription. This issue covers Phase 1 (static lint), Phase 1A (schema introduction), and a reduced Phase 2 (fixture-driven schema validation).

Phase 1 — Static lint pipeline

New .github/workflows/git-ape-ci.yml running in parallel jobs:

  • shellcheck on every .sh (strict mode)
  • yamllint on every YAML
  • markdownlint on every .md (excluding generated website/build)
  • check-jsonschema validation of every committed JSON artifact (see Phase 1A)
  • bats tests against the fixture corpus

Reuse (do not duplicate):

  • git-ape-actionlint.yml — already covers workflow lint
  • git-ape-docs-check.yml — already covers docs staleness
  • git-ape-plugin-version-check.yml — already covers plugin.json version drift

Phase 1A — Schema introduction & versioning

Author strict JSON Schemas (draft 2020-12) under schemas/git-ape/<artifact>/v1.json:

  • _defs/v1.json — shared types (deploymentId, azureSubscription, armResourceId, iso8601DateTime, cafAbbreviation, environmentTier, deploymentStatus, softDeletableType)
  • state/v1.json — matches website/docs/deployment/state.md
  • metadata/v1.json, security-gate/v1.json, requirements/v1.json, cost-estimate/v1.json, policy-recommendations/v1.json
  • plugin/v1.json — for plugin.json

Locked-in design decisions:

  1. Versioning model: per-artifact schemaVersion, shared $defs versioned together as a release train
  2. Strictness: additionalProperties: false on top-level objects only; nested objects allow extras during transition
  3. Pre-1.0 corpus: migrate-first, quarantine fallback (migration tooling shipped in this PR)
  4. security-gate.json: adopts count form only; boolean form is rejected at v1.0
  5. $schema injection: emitters write a relative path (no network coupling)
  6. Doc generation: hybrid — narrative hand-written, field tables generated in a follow-up

Phase 2 — Reduced (parity deferred)

main does not yet have a state.json emitter; those scripts arrive in #44. Phase 2 in this PR therefore covers:

  • Fixture corpus under tests/fixtures/ with valid + invalid samples per artifact type
  • bats tests asserting good fixtures pass schema validation and bad fixtures fail
  • Negative test for security-gate.json boolean form (must be rejected by v1.0 schema)

Bash↔PowerShell parity testing is deferred to a follow-up issue once #44 lands.

Out of scope (tracked separately)

  • Real-Azure sandbox E2E (/e2e label trigger) — Phase 4
  • Agent behavioral snapshot tests — Phase 5
  • PR feedback aggregation — Phase 6
  • bash↔PowerShell parity (depends on Leverage Deployment Stacks for idempotent operations #44)
  • ARM template static checks (Checkov, ARM-TTK, PSRule for Azure, MSDO templateanalyzer)
  • Auto-generated field-reference docs

Acceptance criteria

  • git-ape-ci.yml runs all jobs in parallel and gates the PR
  • At least 7 schema files committed under schemas/git-ape/
  • Shared $defs referenced from per-artifact schemas (no duplication)
  • scripts/validate-schemas.sh invokable locally and from CI
  • Negative-test fixtures (e.g., security-gate.json boolean form, state.json missing required field) cause CI to fail when run against the strict schemas
  • plugin.json itself validates against schemas/git-ape/plugin/v1.json
  • All tools pinned to versions (no floating)
  • tests/fixtures/README.md documents how to add a new fixture

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions