diff --git a/.agents/skills/README.md b/.agents/skills/README.md new file mode 100644 index 00000000..87968b37 --- /dev/null +++ b/.agents/skills/README.md @@ -0,0 +1,58 @@ +# Agent Skills + +This directory contains agent skill descriptors for the SSVC repository. +Skills are Markdown files that describe capabilities an AI coding agent can +invoke to perform structured tasks. + +## Two-tier taxonomy + +| Tier | Path | Audience | Purpose | +|------|------|----------|---------| +| **dev** | `.agents/skills/dev/` | Agents working on the SSVC codebase | Support SSVC development workflows (spec ingestion, codebase study, concern tracking, etc.) | +| **ssvc** | `.agents/skills/ssvc/` | Agents helping practitioners apply SSVC | SSVC domain work (evaluating decision points, building custom tables, etc.) | + +## Canonical SKILL.md format + +Every skill directory must contain a `SKILL.md` file. Required front-matter: + +```yaml +--- +name: "skill-name" # unique, kebab-case +description: > + One or more sentences describing what the skill does and when to invoke it. +--- +``` + +Optional front-matter fields (add when stable): +`id`, `version`, `tags`, `runtime`, `capabilities`, `prerequisites`, `env`, +`usage_examples`. + +After the front-matter, every `SKILL.md` must contain: + +1. `## Purpose` — one paragraph explaining what the skill is for +2. At least one procedural section (`## Procedure` or `## Workflow`) + +## Adding a new skill + +1. Choose the correct tier (`dev/` or `ssvc/`). +2. Create `//SKILL.md` with required front-matter, a + `## Purpose` section, and at least one procedural section. +3. Validate locally: + ```bash + python -c " + import pathlib, yaml, sys + errs = [] + for p in pathlib.Path('.agents/skills').rglob('SKILL.md'): + fm = yaml.safe_load(p.read_text().split('---', 2)[1]) + if not fm or not fm.get('name') or not fm.get('description'): + errs.append(str(p)) + print('FAIL:', errs) if errs else print('OK') + " + ``` +4. Open a PR — the `validate_skills` CI job runs automatically. + +## CI validation + +The workflow `.github/workflows/validate_skills.yml` runs on every PR and +push to `main` when any `SKILL.md` changes. It checks that every skill has +valid YAML front-matter with non-empty `name` and `description` fields. diff --git a/.agents/skills/dev/ingest-idea/SKILL.md b/.agents/skills/dev/ingest-idea/SKILL.md new file mode 100644 index 00000000..fbf53927 --- /dev/null +++ b/.agents/skills/dev/ingest-idea/SKILL.md @@ -0,0 +1,296 @@ +--- +name: ingest-idea +description: > + Process a raw design idea from a GitHub Idea-type issue into formal specs + and implementation notes, then close the idea issue, open a docs-only PR, + and create a GitHub implementation Issue as a sub-issue of the idea issue. + Runs a structured interview (grill-me), writes specs/.yaml and + notes/.md, archives the idea to plan/history/, opens a docs-only PR + with the specs-notes label, and creates a GitHub Issue tagged + group:unscheduled. Use when the user says "ingest idea", references a GitHub + Idea issue number, or wants to convert an idea into spec and notes files. +--- + +# Skill: Ingest Idea + +Convert a GitHub Idea-type issue into durable specs and notes, then close the +idea issue and open a docs-only PR. This is the full end-to-end workflow: +interview → write → archive → PR → implementation issue. + +## Workflow + +### 1. Identify the target idea + +If the user specified a GitHub issue number (e.g., `#42` or just `42`), +skip to step 2. + +Otherwise, query GitHub for open Idea-type issues and present them to the +user as a multiple-choice list using `ask_user`. Include a **"Create a new +idea"** option at the end. + +```bash +gh issue list --repo CERTCC/SSVC \ + --limit 200 \ + --json number,title,issueType \ + --jq '.[] | select(.issueType.name == "Idea") | "#\(.number): \(.title)"' +``` + +Build a `choices` array from the results (e.g. `["#42: Exploitation signal", +"#51: Decision point refactor", "Create a new idea"]`). Wait for the user's +selection before continuing. + +#### 1a. Creating a new idea (if selected) + +Ask the user to describe the idea in freeform text (using `ask_user`). +Synthesize a short, descriptive title from the description, then create +a GitHub Idea-type issue: + +```bash +IDEA_BODY="" +IDEA_TITLE="" +REPO_NODE_ID="MDEwOlJlcG9zaXRvcnkyMzU4MDkzNTU=" +IDEA_TYPE_ID="IT_kwDOAjf0s84B_EoA" + +TITLE_JSON=$(printf '%s' "${IDEA_TITLE}" \ + | python3 -c "import sys,json; print(json.dumps(sys.stdin.read()))") +BODY_JSON=$(printf '%s' "${IDEA_BODY}" \ + | python3 -c "import sys,json; print(json.dumps(sys.stdin.read()))") + +IDEA_NUMBER=$(gh api graphql -f query=" +mutation { + createIssue(input: { + repositoryId: \"${REPO_NODE_ID}\" + title: ${TITLE_JSON} + body: ${BODY_JSON} + issueTypeId: \"${IDEA_TYPE_ID}\" + }) { + issue { number } + } +}" --jq '.data.createIssue.issue.number') +echo "Created idea issue #${IDEA_NUMBER}" +``` + +Continue with step 2 using `IDEA_NUMBER`. + +### 2. Read the idea + +Fetch the idea from GitHub: + +```bash +gh issue view "${IDEA_NUMBER}" --repo CERTCC/SSVC --json number,title,body +``` + +Use the issue title and body as the idea content for all subsequent steps. + +### 3. Explore the codebase + +Invoke the `study-project-docs` skill. It loads all specs, reads plan/, +docs/adr/, notes/, and docs/reference/code/, and scans src/ssvc/ and +src/test/. + +Answer questions from exploration rather than asking the user where possible. + +### 4. Interview with grill-me + +Invoke the `grill-me` skill. Follow its instructions to walk every design +decision branch one at a time using `ask_user`, providing a recommendation +for each question. Reach shared understanding before writing anything. + +### 5. Write the spec file + +Create or modify `specs/.yaml` following `specs/meta-specifications.yaml` +conventions and the ID scheme already in use: + +- Use a `FILE_PREFIX-SECTION_#-###` ID scheme (e.g., `EX-01-001`) +- Define requirements as YAML structures with RFC 2119 keywords +- Include an overview section with source reference and scope note +- Organize by category with clear section headings + +### 6. Write the notes file + +Create or modify `notes/.md` with implementation guidance: + +- Decision table (question → decision → rationale) +- Key design patterns and code examples +- Call-site migration guide if replacing existing patterns +- Testing pattern examples +- Layer / import rules relevant to the change + +### 7. Update specs/README.md + +Add the new spec to both: + +- The **Spec Files** table (file → ID → Kind → Topic) +- The **ID Prefix Convention** section if a new prefix is introduced + +### 8. Lint markdown + +Invoke the `format-markdown` skill on all new/modified markdown files. Fix +any errors before proceeding. + +### 9. Archive the idea + +Append a record to `plan/history/ideas.md` (create the file if it doesn't +exist). Include the full original idea text with a `**Processed**` line: + +```bash +DATE=$(date +%Y-%m-%d) +cat >> plan/history/ideas.md < + + + +**Processed**: ${DATE} — design decisions captured in +\`specs/.yaml\` and \`notes/.md\`. +ENDOFENTRY +``` + +### 10. Open a docs-only PR + +Create a branch, commit all spec/notes/README changes, and open a PR +carrying the `specs-notes` label. Reference the originating idea issue in +the PR body so GitHub auto-links them: + +```bash +git switch -c ingest/idea-- +git add specs/.yaml notes/.md specs/README.md plan/history/ideas.md +git commit -m "specs: ingest idea # — + +- Add specs/.yaml (ID-01 through ID-NN) +- Add notes/.md with implementation guidance +- Archive idea # to plan/history/ideas.md + +Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>" +git push -u origin ingest/idea-- + +gh pr create --repo CERTCC/SSVC \ + --title "specs: ingest idea # — " \ + --body "Docs-only PR: adds spec and notes for idea #. + +Ref # + +No .py files changed." \ + --label "specs-notes" +``` + +### 11. Create a GitHub Issue for implementation + +After opening the docs-only PR, create a GitHub Issue to track implementation. +Wire the idea issue as the **parent** of the new implementation issue: + +```bash +IMPL_ISSUE_NUMBER=$(gh api graphql -f query=" +mutation { + createIssue(input: { + repositoryId: \"MDEwOlJlcG9zaXRvcnkyMzU4MDkzNTU=\" + title: \"\" + body: \"## Summary\n\n\n\n## Acceptance Criteria\n\n- [ ] AC-1: \n- [ ] AC-2: \n\n## Reference\n\nSpec: \`specs/.yaml\`\nNotes: \`notes/.md\`\" + issueTypeId: \"IT_kwDOAjf0s84AcFLs\" + labelIds: [\"\", \"\"] + }) { + issue { number } + } +}" --jq '.data.createIssue.issue.number') +``` + +If the GraphQL label approach is cumbersome, fall back to `gh issue create` +and then set the parent relationship and labels separately: + +```bash +IMPL_ISSUE_NUMBER=$(gh issue create --repo CERTCC/SSVC \ + --title "" \ + --body "## Summary + + + +## Acceptance Criteria + +- [ ] AC-1: +- [ ] AC-2: + +## Reference + +Spec: \`specs/.yaml\` +Notes: \`notes/.md\`" \ + --label "group:unscheduled,size:" \ + | grep -oE '[0-9]+$') + +# Wire idea as parent of implementation issue +gh api graphql -f query=" +mutation { + updateIssue(input: { + id: \"\" + parentId: \"\" + }) { issue { number } } +}" +``` + +Set the `size:` label based on AC checkbox count: +1–2 ACs → `size:S`; 3–6 ACs → `size:M`; 7+ ACs → `size:L`. + +The implementation Issue sits in `group:unscheduled` until a human runs +`review-priorities` to slot it into `plan/PRIORITIES.md`. + +After creating the implementation issue, post the closing comment on the +idea issue and close it: + +```bash +gh issue comment "${IDEA_NUMBER}" --repo CERTCC/SSVC \ + --body "✅ Ingested. + +- Docs PR: +- Implementation issue: #${IMPL_ISSUE_NUMBER} + +Design decisions captured in \`specs/.yaml\` and \`notes/.md\`." + +gh issue close "${IDEA_NUMBER}" --repo CERTCC/SSVC +``` + +## Checklist + +- [ ] Target idea issue confirmed (specified by user, selected from list, or + created inline) +- [ ] Idea content fetched from GitHub issue +- [ ] Codebase explored via `study-project-docs` before grilling +- [ ] All design decision branches resolved via grill-me +- [ ] `specs/.yaml` created with correct ID scheme +- [ ] `notes/.md` created with decision table + examples +- [ ] `specs/README.md` updated (Spec Files table + ID Prefix section if new) +- [ ] Markdown lint clean +- [ ] Idea archived to `plan/history/ideas.md` +- [ ] Docs-only PR opened with `specs-notes` label and `Ref #` + in body +- [ ] Implementation GitHub Issue created with `group:unscheduled` and + `size:` labels; idea issue wired as parent +- [ ] Idea issue commented with links to PR and implementation issue, then + closed + +## Conventions + +- **Spec file names**: use the topic name, lowercase hyphenated with `.yaml` + extension (e.g., `exploitation.yaml`, `decision-points.yaml`) +- **ID prefix**: derive from the topic abbreviation (e.g., `EX`, `DP`) +- **Notes file name**: same slug as spec file with `.md` extension, in + `notes/` (e.g., `exploitation.md`, `decision-points.md`) +- **History archive**: append to `plan/history/ideas.md` using source ID + `IDEA-` + +## Label Naming Rules + +All new Issues use `group:unscheduled` by default. If assigning a specific +`group:` label: + +- **Never include a priority number** in the label name. + Use `group:architecture-hardening`, **not** `group:473-architecture-hardening`. +- **Derive the slug** from the priority group title in kebab-case. +- **Check for label existence** before assigning. Create it if missing: + + ```bash + gh label create "group:" \ + --repo CERTCC/SSVC \ + --description "" \ + --color "#1d76db" + ``` diff --git a/.agents/skills/dev/load-specs/SKILL.md b/.agents/skills/dev/load-specs/SKILL.md new file mode 100644 index 00000000..9962cb72 --- /dev/null +++ b/.agents/skills/dev/load-specs/SKILL.md @@ -0,0 +1,139 @@ +--- +name: load-specs +description: > + Export all project specifications as flat, inheritance-resolved JSON for + coding agents. Run this at the start of any implementation or design task. +tags: + - specs + - requirements + - agent-context +--- + +# Skill: Load All Specs as LLM-Optimized JSON + +## Purpose + +Export all project specifications as a flat, inheritance-resolved JSON +structure optimized for coding agents. This is the **required** way to load +specs before any implementation or design task. + +**Do not read raw `specs/*.yaml` files directly.** Those files are for +authoring and linting only. The JSON export resolves inheritance, flattens +group nesting, and provides a consistent structure for agent consumption. + +## Procedure + +From the repository root, run: + +```bash +uv run ssvc-spec-dump +``` + +This produces compact JSON on stdout. Capture or pipe it as needed. + +## Output Structure + +The JSON has three top-level arrays: + +```json +{ + "topics": [...], + "requirements": [...], + "edges": [...] +} +``` + +### `topics` + +One entry per spec topic (source file). Fields: `id`, `title`, `version`, `kind`. +Use this lookup table to resolve the short `topic` field on each requirement +to a human-readable title. **Do not open the source YAML files** — all +requirement content is already in the `requirements` array. + +### `requirements` + +One entry per requirement. Key fields: + +| Field | Meaning | +|---|---| +| `id` | Unique requirement ID (e.g. `AP-01-001`) | +| `topic` | Parent spec topic ID (e.g. `AP`) | +| `group` | Group within the file (e.g. `AP-01`) | +| `group_title` | Human-readable group name | +| `priority` | `MUST`, `MUST_NOT`, `SHOULD`, `SHOULD_NOT`, or `MAY` | +| `statement` | The normative requirement text | +| `kind` | `domain`, `general`, `implementation`, `language`, or `pattern` | +| `scope` | List: `prototype`, `production`, or both | +| `type` | `behavioral` or `statement` | +| `relationships` | Optional inline list of `{rel_type, spec_id, note?}` edges | +| `rationale` | Optional explanatory text | + +### `edges` + +Centralized list of all relationships across all files: + +```json +{"from": "AP-01-001", "rel_type": "depends_on", "to": "RG-01-001"} +``` + +Edge `rel_type` values: `constrains`, `depends_on`, `derives_from`, +`extends`, `implements`, `part_of`, `refines`, `supersedes`, `verifies`. + +## Cross-Cutting Constraints + +When implementing any feature, always pay attention to requirements from +these cross-cutting spec files regardless of the primary topic: + +- `SR` — spec registry conventions (applies to all spec authoring) +- `CI` — CI/CD integration requirements +- `TS` — testing standards (coverage, test structure) + +These apply to all code changes even when working on a specific feature area. + +## Examples + +```bash +# Full dump (default — always prefer this) +uv run ssvc-spec-dump + +# Write to file for reference +uv run ssvc-spec-dump > /tmp/specs.json + +# Count requirements +uv run ssvc-spec-dump | python -c "import json,sys; r=json.load(sys.stdin); print(len(r['requirements']))" + +# Filter MUST requirements in Python after loading +uv run ssvc-spec-dump | python -c " +import json, sys +data = json.load(sys.stdin) +musts = [r for r in data['requirements'] if r['priority'] == 'MUST'] +print(f'{len(musts)} MUST requirements') +" + +# Filter by topic +uv run ssvc-spec-dump | python -c " +import json, sys +data = json.load(sys.stdin) +ap = [r for r in data['requirements'] if r['topic'] == 'AP'] +print(f'{len(ap)} API requirements') +" +``` + +## Rationale + +The `specs/*.yaml` source files use an authoring-optimized format with +inheritance (kind/scope inherited from file→group→spec), optional rationale, +and group nesting. This structure is good for authors but burdens coding agents +with mental inheritance resolution and navigation of nested structures. + +The LLM JSON export: + +1. Resolves all inheritance so every requirement has explicit `kind` and `scope` +2. Flattens group nesting so requirements are a simple array +3. Centralizes edges for graph-based dependency planning +4. Omits empty/null fields to minimize token usage +5. Uses readable field names (no abbreviations) + +Running this export ensures agents always see the complete, consistent, +authoritative requirement set rather than a subset from browsing individual +files. diff --git a/.agents/skills/dev/process-concerns/SKILL.md b/.agents/skills/dev/process-concerns/SKILL.md new file mode 100644 index 00000000..18631e77 --- /dev/null +++ b/.agents/skills/dev/process-concerns/SKILL.md @@ -0,0 +1,260 @@ +--- +name: process-concerns +description: > + Batch-process docs/reference/codebase/CONCERNS.md into GitHub Concern-type + issues. Optionally runs a focused acquire-codebase-knowledge scan first. + Deduplicates against existing open Concern issues — updating the body and + appending a refresh comment on matches, creating new issues otherwise. + Does not write to specs/, notes/, or open a PR. + Use when you want to turn a codebase scan into a set of tracked GitHub issues. +--- + +# Skill: Process Concerns + +Convert `docs/reference/codebase/CONCERNS.md` into GitHub `type:Concern` +issues. One issue per concern item. Deduplicates against open Concern issues +before creating anything. + +## Constants + +```text +REPO = CERTCC/SSVC +REPO_NODE_ID = MDEwOlJlcG9zaXRvcnkyMzU4MDkzNTU= +CONCERN_TYPE = IT_kwDOAjf0s84B_2VT +``` + +--- + +## Workflow + +### Phase 0 — Decide on Scan Freshness + +Ask the user (via `ask_user`): + +> Should I run a fresh `acquire-codebase-knowledge` scan (focused on +> CONCERNS.md) to get the latest snapshot, or use the existing +> `docs/reference/codebase/CONCERNS.md` as-is? + +Choices: + +1. **Run a fresh focused scan (Recommended)** +2. **Use the existing CONCERNS.md** + +If the user chooses a fresh scan, invoke `acquire-codebase-knowledge` with +focus area `CONCERNS.md` only (do not regenerate the other six documents). +After the scan completes, proceed with the newly generated file. + +### Phase 1 — Load Situation Awareness + +List **all open Concern-type issues** in the repository using the GraphQL API +(the `gh issue list --json issueType` field is not supported by the CLI for +this repo — use GraphQL instead): + +```bash +gh api graphql -f query=' +query { + repository(owner: "CERTCC", name: "SSVC") { + issues(first: 200, states: [OPEN], filterBy: { issueTypeId: "IT_kwDOAjf0s84B_2VT" }) { + nodes { number title } + } + } +}' --jq '.data.repository.issues.nodes[] | "#\(.number): \(.title)"' +``` + +Store the resulting list (numbers + titles) for deduplication in Phase 3. + +### Phase 2 — Parse CONCERNS.md into topics + +Read `docs/reference/codebase/CONCERNS.md`. **Do not assume a fixed format.** +The file is generated and its structure may differ across runs (tables, prose, +numbered lists, bullet points, or a mix). Instead, apply semantic extraction: + +1. **Read the whole file** and identify every distinct concern, risk, debt + item, or fragile area it describes — regardless of how it is formatted. +2. For each topic, extract whatever the file provides: + - **Title**: the clearest short label for the concern (heading text, + bold phrase, bullet key, or a synthesized summary if no label is present) + - **Narrative**: any explanatory text, implications, or context + - **Evidence**: file paths, module names, or line references (backtick + strings, paths ending in `.py` / `.yml` / `.md`, etc.) + - **Severity signal**: any explicit severity language ("high", "critical", + "medium", "low") or implicit priority cues (section placement, wording) + - **Category signal**: any language indicating the type of concern (risk, + debt, security, performance, fragility, etc.) +3. **Infer severity** when no explicit label is given: + - Items in sections labelled "high", "top", "critical" → `high` + - Items in sections labelled "medium" → `medium` + - Items in sections labelled "low", "minor", "constraint" → `low` + - Default to `medium` when no signal is present +4. **Skip sections that are clearly not concerns** — e.g. sections titled + "Safe operating advice", "Recommendations", "How to use this document", + or similar guidance/meta sections that contain no trackable risk items. + +### Phase 3 — Create or Update Issues + +For each concern item from Phase 2: + +#### 3a — Deduplication Check + +Compare the concern's title text against the titles of existing open Concern +issues (loaded in Phase 1). Use semantic similarity — titles do not have to +be identical, but the subject matter must clearly match. + +- **Match found** → proceed to **3b (Update)**. +- **No match** → proceed to **3c (Create)**. + +#### 3b — Update Existing Issue + +1. Synthesize a current, descriptive title from the item data (if notably + different from the existing title, update it; otherwise leave it). +2. Rebuild the issue body from the concern template (see **Issue Body Format** + below) using the latest scan data. +3. Edit the issue body: + + ```bash + gh issue edit "${ISSUE_NUMBER}" \ + --repo CERTCC/SSVC \ + --body "$(cat body.md)" + ``` + +4. Append a refresh comment: + + ```bash + gh issue comment "${ISSUE_NUMBER}" \ + --repo CERTCC/SSVC \ + --body "♻️ Refreshed from codebase scan — $(date +%Y-%m-%d)." + ``` + +#### 3c — Create New Issue + +Synthesize a short, descriptive title from the item data. Build a body +following the **Issue Body Format** below. Ensure labels exist, create the +issue, then apply labels: + +```bash +TITLE_JSON=$(printf '%s' "${TITLE}" \ + | python3 -c "import sys,json; print(json.dumps(sys.stdin.read()))") +BODY_JSON=$(printf '%s' "${BODY}" \ + | python3 -c "import sys,json; print(json.dumps(sys.stdin.read()))") + +# Ensure labels exist before applying them +gh label create "group:unscheduled" \ + --repo CERTCC/SSVC \ + --description "Not yet scheduled in PRIORITIES.md" \ + --color "#e4e669" 2>/dev/null || true + +gh label create "concern" \ + --repo CERTCC/SSVC \ + --description "Technical risk, debt, or fragile area" \ + --color "#d93f0b" 2>/dev/null || true + +ISSUE_NUMBER=$(gh api graphql -f query=" +mutation { + createIssue(input: { + repositoryId: \"${REPO_NODE_ID}\" + title: ${TITLE_JSON} + body: ${BODY_JSON} + issueTypeId: \"${CONCERN_TYPE}\" + }) { + issue { number url } + } +}" --jq '.data.createIssue.issue.number') + +gh issue edit "${ISSUE_NUMBER}" \ + --repo CERTCC/SSVC \ + --add-label "group:unscheduled,concern" + +echo "Created concern issue #${ISSUE_NUMBER}" +``` + +### Phase 4 — Summary + +After processing all items, print a summary table: + +```text +| # | Title | Action | +|---|-------|--------| +| 42 | Global registry with import-time side effects | created | +| 17 | CSV path inconsistency | updated | +``` + +--- + +## Issue Body Format + +All Concern issues use the structure from Vultron's concern template (copy +this structure — SSVC does not yet have a `concern.md` issue template): + +```markdown +## Summary + + + +## Category + +- [x] +- [ ] Top risk +- [ ] Technical debt +- [ ] Security +- [ ] Performance / scaling +- [ ] Fragile / high-churn area +- [ ] Other + +## Severity + + + +## Evidence + + + +- `path/to/module.py` + +## Impact if Ignored + + + +## Suggested Action + + +``` + +Map concern language to Category checkboxes using best judgment: + +| Signal in title or narrative | Category checkbox | +|---|---| +| risk, critical, blocking, severe | Top risk | +| debt, cleanup, refactor, TODO, legacy, deprecated | Technical debt | +| auth, injection, secret, exposure, CVE, privilege | Security | +| slow, latency, memory, scale, throughput | Performance / scaling | +| churn, fragile, hot-spot, tightly coupled, brittle | Fragile / high-churn area | +| anything else | Other | + +--- + +## Constraints + +- Do **not** write to `specs/`, `notes/`, `AGENTS.md`, or open a PR. +- Do **not** delete entries from `CONCERNS.md` — it is a generated file. +- Do **not** assign a `size:` label. +- Do **not** add a parent issue or link issues to each other. +- Do **not** create issues for sections that are clearly guidance/meta (e.g. + "Safe operating advice"), not trackable concern items. +- Always check for existing open Concern issues before creating a new one. +- Use `ask_user` for all user-facing questions; never ask in plain text. +- Use the GraphQL API (not `gh issue list --json issueType`) for listing + Concern issues — the CLI JSON field is not available for this repo. +- Do **not** hard-code section names from a previous CONCERNS.md run — + always derive topics from the current file content. + +## Checklist + +- [ ] User chose scan freshness (fresh focused scan or use existing) +- [ ] All open Concern issues loaded via GraphQL for situation awareness +- [ ] CONCERNS.md read and semantically parsed into topics regardless of format +- [ ] Guidance/meta sections skipped (e.g. "Safe operating advice") +- [ ] Each item deduplicated against open issues (by title similarity) +- [ ] Matched items: body updated + refresh comment added +- [ ] New items: Concern issue created with `group:unscheduled` + `concern` labels +- [ ] Summary table printed diff --git a/.agents/skills/dev/study-project-docs/SKILL.md b/.agents/skills/dev/study-project-docs/SKILL.md new file mode 100644 index 00000000..4cfa7727 --- /dev/null +++ b/.agents/skills/dev/study-project-docs/SKILL.md @@ -0,0 +1,77 @@ +--- +name: study-project-docs +description: > + Load all specs and read key project context files so the agent has a + complete picture before doing any implementation, design, or documentation + work. Invokes load-specs and reads plan/, docs/adr/, notes/, + docs/reference/codebase/, and docs/reference/decision_points/. + Steps 1, 2, and 4 are always required. Run this at the start of every + workflow skill. +--- + +# Skill: Study Project Docs + +Load the full specification set and read key project context so the agent can +work accurately without guessing. This is the standard "orient yourself" step +that all workflow skills run before doing anything else. + +**Steps 1, 2, and 4 are mandatory for every task.** Step 3 may be narrowed +for tightly scoped work, but never skipped entirely. + +## Procedure + +### Step 1 — Load specs *(mandatory)* + +Invoke the `load-specs` skill (run `uv run ssvc-spec-dump`). Capture the JSON +output. Do **not** read raw `specs/*.yaml` files directly. + +### Step 2 — Read plan context *(mandatory)* + +Read the following in parallel (do **not** recurse into `plan/history/`): + +- `plan/PRIORITIES.md` — authoritative priority ordering +- `plan/BUILD_LEARNINGS.md` — ephemeral queue of build/bugfix observations; + read and act on any critical insights before they are lost + +> **Tasks live in GitHub Issues.** Query GitHub for open Issues when selecting +> work; `PRIORITIES.md` is the index, not the task list. +> +> **`plan/history/` is excluded from this step.** It is an archive of +> completed work. Read it only when specifically investigating historical +> changes. + +### Step 3 — Read design documentation *(skippable for narrow tasks)* + +Always read: + +- `specs/README.md` — spec file inventory and ID prefix conventions + +Read in parallel when the task involves design, architecture, or new code: + +- `docs/adr/index.md`, then each ADR in `docs/adr/` relevant to the task +- `notes/` — any `notes/*.md` files relevant to the current task +- `docs/reference/codebase/ARCHITECTURE.md` — system flow and layer rules +- `docs/reference/codebase/CONVENTIONS.md` — naming, formatting, import rules +- `docs/reference/codebase/STRUCTURE.md` — top-level module map and entry points + +Read the following **only when relevant to the task**: + +- `docs/reference/codebase/STACK.md` — when adding or evaluating dependencies +- `docs/reference/codebase/TESTING.md` — when writing or reviewing tests +- `docs/reference/codebase/CONCERNS.md` — when assessing risk or technical debt +- `docs/reference/codebase/INTEGRATIONS.md` — when working on external integrations +- `docs/reference/decision_points/` — domain prose for decision point definitions + and values; read when working on decision points or decision tables + +> **Note:** `docs/reference/codebase/` is generated by the +> `architecture-blueprint-generator` skill. If the directory does not exist, +> run that skill first. +> +> **Note:** `notes/` may be empty early in the project — skip it if no +> relevant files exist yet. + +### Step 4 — Scan the codebase *(mandatory)* + +Search `src/ssvc/` and `src/test/` to verify assumptions about what is +currently implemented. Do not assert missing functionality without evidence +from code search. diff --git a/.agents/skills/ssvc/evaluate-decision-point/SKILL.md b/.agents/skills/ssvc/evaluate-decision-point/SKILL.md new file mode 100644 index 00000000..7eb00cad --- /dev/null +++ b/.agents/skills/ssvc/evaluate-decision-point/SKILL.md @@ -0,0 +1,74 @@ +--- +name: evaluate-decision-point +description: > + Walk through evaluating a single SSVC decision point for a vulnerability. + Given a decision point (e.g., Exploitation, Automatable, Human Impact) and + a vulnerability description, guide the user to select the correct value by + working through the decision point's definition, value descriptions, and + relevant evidence. Use when a practitioner needs help applying one SSVC + decision point to a specific vulnerability. +tags: + - ssvc + - decision-points + - vulnerability-management +--- + +# Skill: Evaluate Decision Point + +> **⚠️ Work in progress.** This skill is a placeholder stub. +> The procedure below is a skeleton — steps are intentionally incomplete +> and will be expanded in a follow-up implementation issue. + +## Purpose + +Help a practitioner correctly evaluate a single SSVC decision point for a +specific vulnerability. The skill loads the decision point definition from +the SSVC registry, presents the possible values with their descriptions, asks +targeted questions to gather relevant evidence, and guides the practitioner +to a well-reasoned value selection. + +This skill covers one decision point at a time. To evaluate all decision +points for a vulnerability and reach a prioritisation outcome, invoke this +skill once per decision point, then consult the appropriate decision table. + +## Procedure + +> **TODO:** Expand each step with concrete instructions and example prompts. + +### Step 1 — Identify the decision point + +Ask the practitioner which decision point to evaluate, or accept it as a +parameter. Confirm the exact name matches an entry in the SSVC registry. + +```bash +# List available decision points +uv run python -c " +from ssvc.registry import registry +for dp in registry.decision_points(): + print(dp.namespace, dp.key, dp.version) +" +``` + +### Step 2 — Load the decision point definition + +Fetch the decision point from the registry and display: +- Its name and description +- Each possible value with its label and description + +### Step 3 — Gather evidence + +Ask targeted questions that help narrow down the correct value. Questions +should be grounded in the decision point's value descriptions, not in +general vulnerability knowledge. + +> **TODO:** Define question templates per decision point. + +### Step 4 — Confirm value selection + +Present the candidate value, summarise the supporting evidence, and ask the +practitioner to confirm or revise. Record the selected value and rationale. + +### Step 5 — Output + +Return the selected value in a format suitable for input to a decision table +or for recording in the practitioner's vulnerability tracking system. diff --git a/.github/workflows/validate_skills.yml b/.github/workflows/validate_skills.yml new file mode 100644 index 00000000..e5436c41 --- /dev/null +++ b/.github/workflows/validate_skills.yml @@ -0,0 +1,77 @@ +name: Validate Skills + +on: + push: + branches: [main] + paths: + - ".agents/skills/**/SKILL.md" + pull_request: + paths: + - ".agents/skills/**/SKILL.md" + +permissions: + contents: read + +jobs: + validate: + name: Validate SKILL.md files + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Check SKILL.md front-matter + run: | + python3 - <<'EOF' + import sys + import pathlib + + try: + import yaml + except ImportError: + import subprocess + subprocess.run( + [sys.executable, "-m", "pip", "install", "pyyaml", "-q"], + check=True, + ) + import yaml + + skills_root = pathlib.Path(".agents/skills") + skill_files = sorted(skills_root.rglob("SKILL.md")) + + if not skill_files: + print("No SKILL.md files found — nothing to validate.") + sys.exit(0) + + errors = [] + for path in skill_files: + text = path.read_text() + if not text.startswith("---"): + errors.append(f"{path}: missing front-matter block (file must start with ---)") + continue + parts = text.split("---", 2) + if len(parts) < 3: + errors.append(f"{path}: front-matter block not closed (missing closing ---)") + continue + try: + fm = yaml.safe_load(parts[1]) + except yaml.YAMLError as exc: + errors.append(f"{path}: invalid YAML front-matter — {exc}") + continue + if not isinstance(fm, dict): + errors.append(f"{path}: front-matter is not a YAML mapping") + continue + for field in ("name", "description"): + value = fm.get(field) + if not value or not str(value).strip(): + errors.append( + f"{path}: missing or empty required field '{field}'" + ) + + if errors: + for err in errors: + print(f"::error::{err}") + print(f"\n{len(errors)} error(s) found in {len(skill_files)} SKILL.md file(s).") + sys.exit(1) + + print(f"All {len(skill_files)} SKILL.md file(s) valid.") + EOF diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 00000000..0df98a77 --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,10 @@ +repos: + - repo: local + hooks: + - id: spec-lint + name: Validate SSVC spec registry + entry: ssvc-spec-lint + language: system + types: [yaml] + files: ^specs/ + pass_filenames: false diff --git a/docs/reference/codebase/ARCHITECTURE.md b/docs/reference/codebase/ARCHITECTURE.md new file mode 100644 index 00000000..ddf5594a --- /dev/null +++ b/docs/reference/codebase/ARCHITECTURE.md @@ -0,0 +1,114 @@ +# SSVC Codebase Architecture + +## System shape + +SSVC is a static domain-model library with three attached delivery layers: +- an in-memory object registry used as the runtime index; +- a thin FastAPI API that exposes registry contents; +- build-time tooling that emits JSON, CSV, schema, and documentation artifacts. + +The authoritative sources are: +- Python object definitions under `src/ssvc/` for decision points, outcomes, decision tables, examples, and schemas; +- YAML spec files under `specs/` for normative engineering requirements. + +Generated outputs under `data/`, `site/`, and `src/ssvc/utils/namespace_patterns.py` are derived artifacts. + +## Primary runtime flow + +### 1. Import-time registry population + +```text +import ssvc + -> src/ssvc/__init__.py imports IMPORTABLES + -> utils.importer.import_modules(..., include_children=True) + -> package walk loads decision_points/, outcomes/, decision_tables/, dp_groups/ + -> module-level Pydantic objects are instantiated + -> _Registered.model_post_init() calls notify_registration(self) + -> registry hook in ssvc.registry.__init__ inserts object into SsvcObjectRegistry +``` + +Notes: +- Registration is eager and happens as a side effect of importing the package. +- Object identity is effectively `namespace:key:version`. +- Registry structure is `type -> namespace -> key -> version`. + +### 2. API query flow + +```text +HTTP request + -> FastAPI app in src/ssvc/api/main.py + -> router in src/ssvc/api/v1/routers/ + -> registry lookup helper (lookup_objtype/lookup_namespace/lookup_key/lookup_version) + -> Pydantic model returned as JSON +``` + +The API does not persist data. `POST` routes under `/examples` validate payloads against models and echo them back. + +### 3. Artifact generation flow + +```text +make regenerate_json + -> ssvc.doctools + -> import/register all objects + -> dump JSON examples, CSV tables, registry JSON, and JSON Schemas under data/ +``` + +Related build paths: +- `run_doctools.yml` regenerates `data/` on PRs touching `src/ssvc/**`. +- MkDocs builds the website from `docs/` plus generated artifacts in `data/`. + +### 4. Spec tooling flow + +```text +specs/*.yaml + -> metadata.specs.schema + metadata.specs.registry + -> ssvc-spec-lint validates authoring rules + -> ssvc-spec-dump exports flat JSON for agents +``` + +The spec subsystem is separate from the runtime object registry. It is build/tooling metadata, not API backing storage. + +## Layer rules + +### Domain definitions +- `decision_points/`: define ordered value sets and versioned decision points. +- `outcomes/`: define ordered outcome sets using the same object patterns. +- `decision_tables/`: compose decision points into validated mappings and tabular exports. +- `selection.py`: minimal evaluation-time representations derived from decision points. + +Must not own: +- HTTP routing; +- filesystem orchestration beyond local helpers; +- long-lived persistence. + +### Cross-cutting model infrastructure +- `_mixins.py` provides versioning, namespacing, schema versioning, timestamps, and auto-registration. +- `namespaces.py` and `utils/field_specs.py` enforce namespace and field constraints. +- `utils/toposort.py` and `csv_analyzer.py` encode ordering and feature-analysis helpers used by tables and policies. + +### Registry layer +- `registry/` owns lookup, registration, duplicate detection, latest-version resolution, and hierarchical storage. +- It is intentionally in-memory and process-local. +- It should remain ignorant of HTTP, MkDocs, and GitHub workflow behavior. + +### Delivery/tooling layers +- `api/` translates registry lookups into HTTP resources. +- `doctools.py`, `doc_helpers.py`, and `decision_tables/helpers.py` translate models into published artifacts. +- `metadata/specs/` validates and exports engineering specifications. + +## Architectural invariants from ADRs and code + +- Decision points are versioned using SemVer. +- Decision point groups are versioned but deprecated; `DecisionTable` is the preferred abstraction. +- Decision points and outcomes are ordered sets. +- Outcome sets are separate from decision point groups. +- “Decision table” is the preferred terminology over “decision tree”. +- Namespaces are constrained to official enum values or extension namespaces beginning with `x_`. +- JSON schemas use explicit `schemaVersion` and a published `$id` under `https://certcc.github.io/SSVC/data/schema/`. + +## Practical rules for agents + +- Do not hand-edit generated artifacts when the Python or spec source is authoritative. +- Assume `import ssvc` has side effects; isolate tests and scripts accordingly. +- Prefer `DecisionTable` for new modeling work; treat `dp_groups/` as compatibility surface. +- When documenting or implementing against specs, use `uv run ssvc-spec-dump` rather than reading raw YAML. diff --git a/docs/reference/codebase/CONCERNS.md b/docs/reference/codebase/CONCERNS.md new file mode 100644 index 00000000..710ee120 --- /dev/null +++ b/docs/reference/codebase/CONCERNS.md @@ -0,0 +1,77 @@ +# SSVC Known Concerns + +## High-priority concerns + +### 1. Global registry with import-time side effects + +`import ssvc` eagerly imports large parts of the package and auto-registers module-level objects into a process-global `SsvcObjectRegistry`. + +Implications: +- test isolation depends on manual resets or local registry injection; +- importing seemingly harmless modules can mutate runtime state; +- future parallelism or lazy loading is harder. + +### 2. Deprecated `DecisionPointGroup` is still active surface area + +`dp_groups/` is deprecated in code and warnings, but it is still: +- imported by the package initializer; +- used by `policy_generator.py`; +- present in examples, schemas, tests, and some collections. + +This creates a split model: preferred `DecisionTable` vs. still-supported legacy groups. + +### 3. Generated-artifact discipline is easy to violate + +Several important outputs are generated, committed, or both: +- `src/ssvc/utils/namespace_patterns.py` +- `data/json/` +- `data/schema/` +- CSV outputs under `data/csv/` and `data/csvs/` + +Drift risk is real because Python source is authoritative but downstream artifacts are also stored in the repo and consumed by docs/workflows. + +## Medium-priority concerns + +### 4. CSV path inconsistency + +The repository uses both `data/csv/` and `data/csvs/`: +- `doctools.py` writes to `data/csv/` +- `mkdocs.yml` table-reader plugin points at `data/csvs` +- `decision_tables/helpers.py` writes under `data/csvs` + +Both directories exist, so behavior works, but the split is a maintenance hazard. + +### 5. Pytest config mismatch + +`pyproject.toml` sets `testpaths = ["test"]`, while the actual suite lives in `src/test/`. +Pytest currently recovers by searching recursively, but baseline runs emit a config warning. + +### 6. Selection serialization warnings + +Baseline tests pass, but `Selection` and `SelectionList` paths emit Pydantic warnings because tuple-typed `values` fields sometimes serialize from list-shaped data. +That is not a failure today, but it signals model-shape friction. + +### 7. Spec tooling looks reusable but is still local copy + +`metadata/specs/schema.py` and `metadata/specs/registry.py` both carry TODOs about extracting the spec-registry subsystem into a shared package. +Until that happens, SSVC owns a local implementation that may diverge from sibling projects. + +## Lower-priority constraints + +- The API is unauthenticated and effectively suited to read-only or trusted/internal use. +- Main project dependencies include docs/build libraries, so the base environment is heavier than a minimal API/library runtime. +- `site/` exists as generated output inside the repo tree, which can confuse tooling that does not distinguish source from built artifacts. + +## Change-risk hotspots + +- `src/ssvc/__init__.py`, `registry/`, `_mixins.py` — affect nearly all runtime loading. +- `decision_tables/base.py` and `utils/toposort.py` — affect mapping generation and validation semantics. +- `doctools.py`, `mkdocs.yml`, `data/` paths — affect docs and generated artifacts together. +- `metadata/specs/` — cross-cutting requirements and agent/export behavior. + +## Safe operating advice + +- Prefer additive changes to versioned decision-point and decision-table modules. +- Treat generated files as outputs, not design sources. +- If touching registry behavior, rerun the full suite, not just targeted tests. +- If touching specs or docs-data generation, verify both pytest and relevant CI-style build paths. diff --git a/docs/reference/codebase/CONVENTIONS.md b/docs/reference/codebase/CONVENTIONS.md new file mode 100644 index 00000000..501ac914 --- /dev/null +++ b/docs/reference/codebase/CONVENTIONS.md @@ -0,0 +1,61 @@ +# SSVC Codebase Conventions + +## Naming + +- Python modules and packages use `snake_case`. +- Classes use `PascalCase`. +- Functions, methods, and variables use `snake_case`. +- Constants use `UPPER_SNAKE_CASE`. +- Internal mixins and helper models often use a leading underscore: `_Registered`, `_SchemaVersioned`, `_Namespace`. +- Decision-point and outcome modules usually expose versioned constants plus: + - `VERSIONS` — append-only ordered tuple of versions; + - `LATEST` — alias for the newest version. + +## Object identity and versioning + +- Runtime objects are identified by `namespace:key:version`. +- Decision point values are identified by `namespace:key:version:value_key`. +- Versions are semantic versions validated with `semver`. +- Namespaces must be official enum values or extension namespaces that begin with `x_` and satisfy the ABNF-derived regex. + +## Model style + +- Pydantic v2 models are the dominant implementation style. +- Cross-cutting behavior is added with mixins rather than deep inheritance hierarchies. +- Validation is pushed into `field_validator` and `model_validator` methods. +- `schemaVersion` is auto-populated by schema-aware models. +- Domain objects auto-register on `model_post_init()` unless created with `registered=False`. + +## Formatting and imports + +- Formatting is controlled by Black with `line-length = 79`. +- Use absolute imports from `ssvc`, not relative imports. +- Typical import grouping is stdlib, third-party, then internal `ssvc` imports. +- Be aware that `import ssvc` is not a benign import; it walks subpackages and mutates the global registry. + +## Domain-module patterns + +- Decision point modules usually define reusable `DecisionPointValue` constants first, then one or more versioned decision point objects. +- Decision table modules usually import named decision points/outcomes and instantiate a module-level `DecisionTable` constant. +- `DecisionTable.key` is normalized to a `DT_` prefix if not already present. +- New work should prefer `DecisionTable`; `DecisionPointGroup` is retained for backward compatibility and emits deprecation warnings. + +## Logging and error handling + +- Logging uses the stdlib `logging` module with per-module loggers. +- Validation failures generally raise `ValueError` or `TypeError` from model validation helpers. +- API routers translate missing registry results into `HTTPException(404)` via `_404_on_none`. +- Duplicate registration with mismatched attributes raises an error rather than silently overwriting. + +## Testing conventions that affect implementation + +- Tests run under pytest but commonly use `unittest.TestCase` classes. +- Test files live in `src/test/` and are named `test_.py`. +- API tests typically replace a router module's global `r` registry with a fresh `SsvcObjectRegistry`. +- Fixtures that should not mutate the global registry are often created with `registered=False`. + +## Generated-artifact discipline + +- Do not hand-edit `src/ssvc/utils/namespace_patterns.py`; regenerate it. +- Do not treat `data/json/`, `data/schema/`, or CSV outputs as authoritative when corresponding Python source exists. +- For agent/spec work, use `uv run ssvc-spec-dump`; raw `specs/*.yaml` is authoring format, not the preferred consumption format. diff --git a/docs/reference/codebase/INTEGRATIONS.md b/docs/reference/codebase/INTEGRATIONS.md new file mode 100644 index 00000000..eb5082fa --- /dev/null +++ b/docs/reference/codebase/INTEGRATIONS.md @@ -0,0 +1,84 @@ +# SSVC Integrations and Boundaries + +## Runtime integration boundaries + +### HTTP API + +The only application network boundary is the FastAPI service in `src/ssvc/api/`. + +Base prefix: +- `/ssvc/api/v1` + +Route families: +- `decision_point` / `decision_points` +- `decision_table` / `decision_tables` +- `objects` +- `types`, `namespaces`, `keys`, `versions` +- `examples` + +Behavior: +- `GET` routes read from the in-memory registry. +- `POST` routes in `/examples` validate payloads against Pydantic models and return them; they do not write state. +- There is no auth, database session, cache, or external service dependency behind these routes. + +### Filesystem boundary + +Important file-backed interfaces: +- `specs/*.yaml` — authored engineering requirements. +- `data/json/` — generated JSON examples and registry export. +- `data/schema/` — generated JSON Schemas published by the docs site. +- `data/csv/` and `data/csvs/` — generated decision-table artifacts consumed by docs/tooling. +- `docs/` — MkDocs source that reads generated artifacts and Markdown content. + +## CI/CD and repository automation + +GitHub Actions workflows are the main external automation surface: +- `python-app.yml` — installs deps, runs pytest, builds the package, uploads artifact. +- `docs_build_check.yml` — validates MkDocs build. +- `lint_md_changes.yml` — lints changed Markdown files. +- `link_checker.yml` — builds the site and checks links. +- `run_doctools.yml` — regenerates `data/` on PRs touching `src/ssvc/**`. +- `deploy_site.yml` — builds and deploys the MkDocs site to GitHub Pages from `publish` branch pushes. + +Related GitHub integrations: +- Dependabot updates `uv` dependencies and workflow actions. +- CODEOWNERS, issue templates, and PR templates define repository process boundaries. + +## Published/public endpoints + +- Documentation site: `https://certcc.github.io/SSVC/` +- Schema base URL used in generated `$id` fields: `https://certcc.github.io/SSVC/data/schema/` +- Repository/project links in package metadata point to GitHub (`CERTCC/SSVC`). + +The repo builds package artifacts in CI, but there is no checked-in workflow here that publishes releases to PyPI. + +## Documentation-site external assets + +`mkdocs.yml` pulls in or links to external browser-side resources: +- MathJax CDN +- `tablesort` CDN +- jQuery CDN +- D3 CDN +- Google Analytics + +These affect the published site, not the core Python runtime or API behavior. + +## Docker boundary + +Local container workflows are defined under `docker/`: +- `docker-compose.yml` provides docs/API/test services. +- `docker/env_example` shows the only documented env var: `COMPOSE_PROJECT_NAME=ssvc`. + +Docker is optional for development; local `uv` + `make` workflows are first-class. + +## What is not integrated + +The codebase does **not** contain a runtime integration with: +- relational or document databases; +- message queues; +- background job systems; +- external REST/SOAP clients; +- cloud SDKs; +- authentication/identity providers. + +That absence is intentional: the system is mostly static reference data, validation logic, docs generation, and registry-backed reads. diff --git a/docs/reference/codebase/STACK.md b/docs/reference/codebase/STACK.md new file mode 100644 index 00000000..15811fd2 --- /dev/null +++ b/docs/reference/codebase/STACK.md @@ -0,0 +1,86 @@ +# SSVC Technology Stack + +## Core runtime choices + +- **Python 3.12+** + - Entire codebase is Python; package metadata requires `>=3.12`. +- **setuptools + setuptools-scm** + - Standard packaging/build backend with version file generation under `src/ssvc/_version.py`. +- **uv** + - Used for dependency sync, script execution, locking, and CI installs. + +## Primary application libraries + +- **Pydantic v2** + - Foundation for domain models, validation, JSON serialization, and JSON Schema generation. +- **FastAPI** + - Thin REST wrapper around the registry; no persistence layer behind it. +- **semver** + - Validates and compares object versions. +- **PyYAML** + - Loads `specs/*.yaml` into the spec registry. +- **jsonschema** + - Supports schema validation workflows around generated artifacts. + +## Table, graph, and analysis libraries + +- **pandas** + - Converts decision tables to shortform/longform tabular exports and CSV output. +- **networkx** + - Powers graph creation, topological sorting, and spec relationship graphs. +- **scikit-learn** + - Used by `csv_analyzer.py` for feature-importance analysis. +- **scipy** + - Present for analysis/scientific workflows in the broader toolchain. +- **thefuzz** + - Fuzzy matching utility dependency. + +## Documentation stack + +- **MkDocs + Material** + - Builds the published documentation site. +- **mkdocstrings + mkdocstrings-python** + - Generates API reference pages from Python objects/docstrings. +- **mkdocs-include-markdown-plugin** + - Reuses Markdown content across docs. +- **markdown-exec** + - Executes code blocks inside Markdown pages. +- **mkdocs-bibtex** + - Supports citations. +- **mkdocs-table-reader-plugin** + - Reads CSV data into docs pages. +- **mkdocs-print-site-plugin** + - Adds printable/export-friendly output. +- **pymdown-extensions** + - Enables richer Markdown features. + +## Build and local-run tooling + +- **Make** + - Single command surface for dev setup, tests, docs, API, and regeneration. +- **Docker / docker-compose** + - Optional local container workflow for docs, API, and tests. +- **abnf-to-regexp** + - Codegen tool used to build `namespace_patterns.py` from ABNF. + +## Quality and automation tools + +- **pytest** + - Main test runner. +- **black** + - Python formatter. +- **markdownlint** + - Markdown linting/fixing. +- **linkchecker** + - Validates built-site links in CI. +- **GitHub Actions** + - Runs tests, docs build checks, markdown linting, link checking, doctools regeneration, and Pages deployment. +- **Dependabot** + - Updates `uv` dependencies and workflow actions. + +## Why this stack fits the codebase + +- The project is mostly static, schema-heavy domain data, so Pydantic models and generated artifacts are a natural fit. +- Registry-backed reads are sufficient because the API serves versioned reference objects rather than mutable business transactions. +- NetworkX and pandas match the problem domain of ordered combinations, mappings, and tabular policy outputs. +- MkDocs plus generated data keeps the public website tied closely to the Python source of truth. diff --git a/docs/reference/codebase/STRUCTURE.md b/docs/reference/codebase/STRUCTURE.md new file mode 100644 index 00000000..ec0ce9fe --- /dev/null +++ b/docs/reference/codebase/STRUCTURE.md @@ -0,0 +1,88 @@ +# SSVC Codebase Structure + +## Top-level repository map + +- `src/ssvc/` — primary Python package. +- `src/test/` — pytest suite; mirrors production package areas. +- `specs/` — normative requirement files for the engineering/spec system. +- `docs/` — MkDocs source for the published documentation site. +- `docs/adr/` — architectural decision records. +- `data/` — generated JSON, schema, and CSV/CSVS artifacts consumed by docs and tooling. +- `docker/` — Dockerfile, compose file, and example environment file. +- `.github/workflows/` — CI, docs build, link checking, data regeneration, and Pages deploy. +- `wip_notes/` — draft/internal notes; not authoritative output. +- `obsolete/`, `doc/`, `pdfs/` — historical/reference material; not active implementation targets. +- `site/` — built MkDocs site output. + +## Main package map: `src/ssvc/` + +### Runtime package entry points +- `__init__.py` — eager importer that walks subpackages and populates the registry. +- `api/main.py` — FastAPI `app`; served by `uvicorn ssvc.api.main:app`. +- `registry/__init__.py` — creates the global `SsvcObjectRegistry` and attaches the registration hook. + +### Core modeling packages +- `decision_points/` + - Namespace-specific decision points (`ssvc`, `cvss`, `cisa`, `nist`, `basic`, `example`). + - Common pattern inside a module: value constants, versioned object constants, `VERSIONS`, `LATEST`. +- `outcomes/` + - Outcome sets for SSVC, CVSS, CISA, basic examples, and extension namespaces. +- `decision_tables/` + - Concrete decision tables by namespace plus shared validation/export logic in `base.py` and `helpers.py`. +- `dp_groups/` + - Deprecated `DecisionPointGroup` compatibility layer plus old grouped collections. +- `selection.py` + - `Selection`, `SelectionList`, and minimal value/reference models for evaluation payloads. + +### Tooling and support modules +- `doctools.py` — writes JSON examples, CSV tables, registry exports, and schemas under `data/`. +- `csv_analyzer.py` — checks topological ordering and feature importance of CSV tables. +- `policy_generator.py` — older graph-based policy generator built around deprecated groups. +- `doc_helpers.py` / `md_gen.py` — helpers for documentation formatting. +- `examples.py` — sample objects returned by `/examples` API routes. +- `metadata/specs/` — spec-file schema, registry, linting, rendering, and LLM export. +- `utils/` — defaults, field specs, importer, namespace regex support, schema ordering, misc helpers, topological sort. +- `namespaces.py` — namespace validation and official namespace enum. + +## API surface layout + +- `api/v1/routers/decision_point.py` and `decision_table.py` — single-object lookup by `id` query string. +- `api/v1/routers/decision_points.py` and `decision_tables.py` — collection, namespace, key, version, and latest endpoints. +- `api/v1/routers/objects.py` — explicit typed object paths. +- `api/v1/routers/types.py`, `namespaces.py`, `keys.py`, `versions.py` — registry browsing endpoints. +- `api/v1/routers/examples.py` — sample-object GETs and model-validation POSTs. +- `api/v1/response_models/` — typed wrappers and aliases for API responses. + +Base API prefix: `/ssvc/api/v1`. + +## Tests layout + +Representative mirrors under `src/test/`: +- `api/routers/` ↔ `src/ssvc/api/v1/routers/` +- `decision_points/` ↔ `src/ssvc/decision_points/` +- `decision_tables/` ↔ `src/ssvc/decision_tables/` +- `registry/`, `utils/`, `metadata/specs/`, `outcomes/`, `dp_groups/` +- top-level tests for `csv_analyzer`, `doctools`, `mixins`, `namespaces`, `policy_generator`, `selection` + +## Command entry points + +Defined in `pyproject.toml`: +- `ssvc_csv_analyzer` → `ssvc.csv_analyzer:main` +- `ssvc_doctools` → `ssvc.doctools:main` +- `ssvc-spec-lint` → `ssvc.metadata.specs.lint:main` +- `ssvc-spec-dump` → `ssvc.metadata.specs.render:main_llm_json` + +Developer entry points in `Makefile`: +- `make dev`, `make test`, `make docker_test` +- `make docs_local`, `make docs` +- `make api_dev`, `make api` +- `make regenerate_json`, `make mdlint_fix` + +## Generated and derived paths to recognize + +- `src/ssvc/utils/namespace_patterns.py` — generated from ABNF; do not hand-edit. +- `data/json/` — generated decision point, decision table, and registry JSON. +- `data/schema/` — generated JSON Schemas. +- `data/csv/` and `data/csvs/` — generated table artifacts used by tooling/docs. +- `site/` — built documentation site. +- `docs/reference/code/` — mkdocstrings website material, not a good source of agent-oriented architecture context. diff --git a/docs/reference/codebase/TESTING.md b/docs/reference/codebase/TESTING.md new file mode 100644 index 00000000..adf76806 --- /dev/null +++ b/docs/reference/codebase/TESTING.md @@ -0,0 +1,66 @@ +# SSVC Testing Guide + +## What is tested + +The suite exercises: +- domain models (`DecisionPoint`, `DecisionTable`, outcomes, selections, mixins); +- registry behavior and duplicate/version handling; +- FastAPI routers via `TestClient`; +- documentation helpers and doctools artifact generation; +- CSV analysis and graph/topology helpers; +- spec registry schema and lint behavior under `metadata/specs/`. + +## Test layout + +Tests live in `src/test/` and largely mirror production structure: +- `api/` and `api/routers/` +- `decision_points/` +- `decision_tables/` with namespace subtrees +- `registry/`, `utils/`, `outcomes/`, `dp_groups/`, `metadata/specs/` +- top-level tests for `csv_analyzer`, `doctools`, `mixins`, `namespaces`, `policy_generator`, `selection` + +This layout matches the normative testing spec topic (`TS`), which requires tests under `src/test/` with mirrored structure and `test_.py` naming. + +## Execution commands + +- `make test` — preferred local command; also regenerates `namespace_patterns.py` prerequisite. +- `uv run pytest -v` — direct local run. +- `make docker_test` — containerized run. +- `uv run pytest src/test/...` — targeted subtree execution. + +Current baseline: `make test` passes locally in this repository state. + +## Test style and isolation rules + +- Pytest is the runner, but many tests are written as `unittest.TestCase` classes. +- API tests create a fresh `FastAPI` app and include only the router under test. +- Router tests commonly replace the router module's global registry variable (`decision_point.r = self.r`) with a fresh `SsvcObjectRegistry`. +- Registry-backed tests call `reset(force=True)` before loading fixtures. +- Fixture objects that must not auto-register are commonly instantiated with `registered=False`. +- Spec tests are more pytest-native and use fixtures such as `tmp_path` and local YAML directories. + +## Quality signals and gaps + +Strong signals: +- broad coverage across runtime models, API, doctools, and specs; +- CI runs pytest on pushes and pull requests to `main`; +- docs build and link checks are separate workflows. + +Known gaps: +- no coverage threshold or `pytest-cov` configuration; +- no end-to-end deployed API test; +- warnings are tolerated in baseline runs, especially around deprecated `DecisionPointGroup`, selection serialization, and config drift. + +## Warnings agents should recognize + +A clean pass may still emit warnings for: +- `DecisionPointGroup` deprecation; +- Pydantic serializer warnings in `Selection` / `SelectionList` paths; +- pytest falling back from `testpaths` because `pyproject.toml` points to `test` while the suite lives in `src/test/`. + +## Practical change checklist + +When changing runtime code: +- run `make test`; +- if namespace grammar or generated artifacts are affected, run `make dev` or `make regenerate_json` as appropriate; +- if docs content or data-backed pages change, consider the docs build workflows (`mkdocs build`, link check) as part of validation. diff --git a/notes/namespaces.md b/notes/namespaces.md new file mode 100644 index 00000000..8656ae0c --- /dev/null +++ b/notes/namespaces.md @@ -0,0 +1,97 @@ +# Namespaces + +SSVC uses **namespaces** to distinguish between objects managed by the SSVC +project and objects derived from external sources. This makes it possible to +include, for example, CVSS vector elements as SSVC decision point objects +without claiming that the SSVC project controls their definitions. + +## Namespace Categories + +### Registered namespaces + +Registered namespaces are defined in the `NameSpace` enum in +`src/ssvc/namespaces.py`. Only the SSVC project controls which values are +registered. + +Current registered namespaces include `ssvc` (for core SSVC objects) and +`cvss` (for CVSS vector element wrappers). Any new registered namespace +requires a code change and a PR to the SSVC repository. + +### Extension namespaces + +Third-party adopters who want to define custom decision points or refine +existing ones can use **unregistered extension namespaces** without +coordinating with the SSVC project. Extension namespace strings must: + +1. Begin with the prefix `x_`. +2. Follow the format `x_#`. + +*Examples:* + +| Use case | Example namespace | +|----------|-------------------| +| Internal agency use | `x_example.agency#internal` | +| Interagency sharing | `x_example.agency#interagency` | +| ISAO constituency refinement | `x_example.isao#constituency-1` | + +Reverse-domain notation delegates uniqueness to DNS ownership, so no central +registration is needed. + +### Namespace extensions (refinements) + +A **namespace extension** refines the semantics of an existing namespace for a +specific context, without introducing wholly new objects. The syntax uses a +double-slash separator: + +``` +//# +``` + +*Example:* An ISAO refining SSVC decision point values for its member +constituency might use `ssvc//.example.isao#constituency-1`. + +Extensions are not intended to introduce new decision points; they add nuance +to the *interpretation* of existing ones. + +--- + +## Validation + +Namespace values are validated at object construction time. Valid values are: + +- A member of the `NameSpace` enum (registered namespaces). +- A string that starts with `x_` and matches the extension pattern + (see `src/ssvc/utils/ssvc_namespace_pattern.abnf` for the ABNF grammar). + +Anything else raises a `ValueError` at construction. + +Reserved namespace strings (listed in `ssvc.namespaces`) cannot be used as the +base of any namespace, registered or extension. + +--- + +## Practical Guidance for Adopters + +**I want to create my own decision points for internal use:** +Use an extension namespace like `x_myorg.example#internal`. You can instantiate +`DecisionPoint` objects with this namespace and they will register in the +in-memory registry without conflicting with SSVC project objects. + +**I want to share my decision points with other organisations:** +Use a stable extension namespace based on a domain you control, e.g. +`x_myorg.example#shared`. If your decision points prove broadly useful, +consider proposing them to the SSVC project for registration. + +**I want to adapt existing SSVC decision point descriptions for my context:** +Use a namespace extension, e.g. `ssvc//.myorg.example#context`. Document +clearly which base SSVC object you are extending so consumers understand the +relationship. + +--- + +## See Also + +- ADR-0012: [SSVC Namespaces](../docs/adr/0012-ssvc-namespaces.md) +- Spec NS: `specs/namespaces.yaml` +- Implementation: `src/ssvc/namespaces.py` +- Pattern grammar: `src/ssvc/utils/ssvc_namespace_pattern.abnf` diff --git a/notes/skills.md b/notes/skills.md new file mode 100644 index 00000000..d281030a --- /dev/null +++ b/notes/skills.md @@ -0,0 +1,164 @@ +# Skills Infrastructure + +Implementation guidance for the SSVC agent skill system (spec `SK`). + +## Decision Table + +| Question | Decision | Rationale | +|----------|----------|-----------| +| Where do skills live? | `.agents/skills/dev/` for dev skills; `.agents/skills/ssvc/` for domain skills | Keeps machine-facing tooling separate from source and docs; two-tier split clarifies audience and lifecycle | +| What format does each skill use? | SKILL.md with YAML front-matter (`name` + `description` required) + `## Purpose` + at least one procedural section | Minimal required surface avoids over-engineering early skills; prose body is what agents actually execute | +| What should CI check? | Valid YAML front-matter, non-empty `name` and `description` fields | Automated enforcement keeps the skill surface reliable without burdening authors | +| Where does CI validation logic live? | Inline Python in `.github/workflows/validate_skills.yml` | Avoids proliferating auxiliary scripts; logic is short and clear in situ | +| How should the workflow trigger? | PRs and push-to-main filtered to `.agents/skills/**/SKILL.md` | Path filtering prevents wasted CI cycles on unrelated changes | + +## Directory Layout + +``` +.agents/ + skills/ + dev/ ← dev-workflow skills (support SSVC development) + ingest-idea/ + SKILL.md + load-specs/ + SKILL.md + process-concerns/ + SKILL.md + study-project-docs/ + SKILL.md + ssvc/ ← SSVC domain skills (help practitioners apply SSVC) + evaluate-decision-point/ + SKILL.md + README.md ← explains the two-tier split + SKILL.md format +``` + +## Canonical SKILL.md Format + +### Required front-matter fields + +```yaml +--- +name: "skill-name" # unique, kebab-case identifier +description: > + One or more sentences describing what this skill does and when to invoke it. + Agents use this text when deciding whether to call the skill. +--- +``` + +### Optional front-matter fields + +Any subset of the following may be added when the information is stable: + +```yaml +id: "unique-id" +version: "1.0.0" +tags: + - python + - testing +runtime: + language: "python" + package_manager: "uv" + framework: "pytest" +capabilities: + - "Description of what the skill can do" +prerequisites: + - "Tool or condition required before the skill can run" +env: + - name: GITHUB_TOKEN + description: "Required for GitHub API calls" +usage_examples: + - prompt: "example user prompt that triggers this skill" + command: "uv run some-command" +``` + +### Required body sections + +After the front-matter block every SKILL.md MUST contain: + +1. `## Purpose` — one-paragraph explanation of what the skill is for +2. At least one procedural section (`## Procedure`, `## Workflow`, etc.) — the + step-by-step instructions an agent executes + +### WIP stubs + +New skills that are not yet ready for use SHOULD carry a prominent WIP notice +at the top of the `## Purpose` section: + +```markdown +> **⚠️ Work in progress.** This skill is a placeholder stub. +> Procedure is intentionally incomplete. +``` + +## Migrating Existing Dev Skills + +The four existing dev skills in `.agents/skills/` must be moved to +`.agents/skills/dev/` and their front-matter normalised to the canonical +format. The normalisation rules are: + +1. Keep `name` and `description`. If `description` is missing or blank, add one. +2. Remove non-canonical fields that add noise without value (e.g., `author`, + `shell`, `commands`, `inputs`, `outputs` from `load-specs`). + **Exception:** keep any optional field that is accurate and genuinely useful + to agents (e.g., `tags`, `usage_examples`). +3. Do not alter any Markdown body content — only the front-matter changes. + +## CI Validation + +The workflow `.github/workflows/validate_skills.yml` discovers all +`.agents/skills/**/SKILL.md` files and validates each one: + +```python +import sys, pathlib +try: + import yaml +except ImportError: + import subprocess + subprocess.run([sys.executable, "-m", "pip", "install", "pyyaml", "-q"], check=True) + import yaml + +errors = [] +for path in sorted(pathlib.Path(".agents/skills").rglob("SKILL.md")): + text = path.read_text() + if not text.startswith("---"): + errors.append(f"{path}: missing front-matter block") + continue + try: + fm_text = text.split("---", 2)[1] + fm = yaml.safe_load(fm_text) + except yaml.YAMLError as e: + errors.append(f"{path}: invalid YAML front-matter — {e}") + continue + for field in ("name", "description"): + if not fm or not fm.get(field): + errors.append(f"{path}: missing or empty required field '{field}'") + +if errors: + for e in errors: + print(f"::error::{e}") + sys.exit(1) +print(f"All {len(list(pathlib.Path('.agents/skills').rglob('SKILL.md')))} SKILL.md files valid.") +``` + +## Testing Patterns + +Because skill validation is enforced in CI, no unit tests in `src/test/` are +needed for the validator itself. However, if the validator script grows beyond +~50 lines, extract it to `.github/scripts/validate_skills.py` and add a +`src/test/github/test_validate_skills.py` that exercises it with in-memory +fixtures. + +## Adding a New Skill + +1. Decide tier: dev workflow → `.agents/skills/dev/`; SSVC domain → `.agents/skills/ssvc/` +2. Create `//SKILL.md` with required front-matter + `## Purpose` + `## Procedure` +3. Run the CI validation locally: + ```bash + python -c " + import pathlib, yaml, sys + for p in pathlib.Path('.agents/skills').rglob('SKILL.md'): + fm = yaml.safe_load(p.read_text().split('---',2)[1]) + assert fm.get('name') and fm.get('description'), f'FAIL: {p}' + print('OK') + " + ``` +4. Open a PR — the `validate_skills` CI job will run automatically. diff --git a/notes/terminology.md b/notes/terminology.md new file mode 100644 index 00000000..50d15474 --- /dev/null +++ b/notes/terminology.md @@ -0,0 +1,63 @@ +# Terminology: Decision Table vs. Decision Tree + +## The Canonical Term + +**Use "decision table"** when referring to the SSVC data structure that maps +combinations of decision point values to recommended actions. + +**Avoid "decision tree"** as the primary term for this structure. + +This is formalised in ADR-0014 (accepted 2025-08-26, first applied in +SSVC v2025.9). + +--- + +## Why the Change? + +SSVC's data structure is a **table**: a set of rows, each pairing a combination +of decision point values with an outcome. The term "decision tree" caused +confusion with two established concepts in other fields: + +- **Machine Learning:** A "decision tree" in ML is a recursive splitting + algorithm used for classification and regression. Practitioners familiar + with this meaning were misled by our use of the term. +- **Operations Research:** OR uses "decision tree" for branching probability + diagrams to model expected-value decisions under uncertainty. Again, a + different structure from what SSVC does. + +The term **"decision policy"** was also in use but has overloaded meanings +beyond what we intend. + +**"Decision table"** accurately describes a tabular mapping of inputs to +outputs, matches the actual CSV and JSON data structures we use, and avoids +both established competing meanings. + +--- + +## Usage Guide + +| Context | Preferred term | Notes | +|---------|---------------|-------| +| The SSVC data structure (mapping of inputs → outcomes) | *decision table* | | +| The broader class of modeling approaches | *decision model* | "Decision table" is a kind of decision model | +| A tree diagram used to *illustrate* a decision table | *tree diagram* or *tree view* | Visual aid, not the data structure itself | +| The old term in archived/legacy content | *decision tree* | Acceptable when quoting or referencing prior versions | + +--- + +## Updating Existing References + +When editing documentation or code, replace "decision tree" with "decision +table" where the SSVC tabular structure is intended. References to tree +*diagrams* used purely as visual aids may retain "tree" wording when it aids +clarity, but should be distinguished from the decision table itself. + +The Python class `DecisionTable` (in `src/ssvc/decision_tables/`) already +reflects the canonical term. + +--- + +## See Also + +- ADR-0014: [Use "Decision Table" Instead of "Decision Tree"](../docs/adr/0014-use-decision-table-terminology.md) +- `src/ssvc/decision_tables/` — canonical Python implementation using the correct term diff --git a/notes/versioning-rules.md b/notes/versioning-rules.md new file mode 100644 index 00000000..4c5a598b --- /dev/null +++ b/notes/versioning-rules.md @@ -0,0 +1,169 @@ +# Versioning Rules + +SSVC uses a **layered versioning strategy**: different levels of the system +use different versioning schemes suited to their specific compatibility semantics. + +## Summary + +| Level | Scheme | Example | Governed by | +|-------|--------|---------|-------------| +| Decision points and groups | SemVer 2.0.0 | `2.1.0` | ADR-0002, ADR-0004, ADR-0006 | +| JSON schema files | SchemaVer | `1-2-0` | ADR-0015 | +| SSVC project / docs releases | CalVer | `2025.6.0` | ADR-0013 | + +--- + +## Decision Point Versioning (SemVer) + +Individual `DecisionPoint` objects are versioned with **Semantic Versioning 2.0.0** +(`MAJOR.MINOR.PATCH`). The semantics map as follows: + +### Create a new object when + +A *different or fundamentally new concept* is being represented — even if it +superficially resembles an existing decision point. New objects **should** get +new names and new keys. Do not reuse an existing name with a version bump when +the underlying concept has genuinely changed. + +### Increment MAJOR when + +- An existing value is **removed**. +- Value semantics change such that **older answers are no longer usable**. +- New values are added that **divide previous value semantics ambiguously**. + +> **Note:** The ability to map old-to-new semantics is encouraged but not required. + +### Increment MINOR when + +*(Existing value semantics are preserved.)* + +- New options/values are **added**. +- Value names or keys are **changed** (without semantic shift). +- The decision point **name** is changed. + +### Increment PATCH when + +*(Existing value count and semantics are preserved.)* + +- **Typo fixes** in option or decision point names. +- The **description** changes in a way that does not affect semantics. + +### Pre-support (v0.x) decision points + +A decision point at major version `0` is **pre-support**: all aspects +(key, labels, ordering, descriptions, semantics) are subject to change without +a major-version increment. The Minor version absorbs the full SemVer +major/minor distinction during this phase. + +The lowest *supported* version of a decision point is `1.0.0`. + +--- + +## Decision Point Group Versioning (SemVer) + +`DecisionPointGroup` objects are also versioned with SemVer. The *core identity* +of a group is the pairing of the **stakeholder role** and the **decision being +modeled**. + +### Create a new object when + +The stakeholder role and/or the decision being modeled changes. Even if the +constituent decision points remain the same, a shift in either dimension +represents a fork in version history. **New objects must have new names.** + +> Name changes alone (e.g. *Patch Applier* → *Deployer* for the same role and +> decision) are **not** a reason to create a new object; they are a patch-level +> event. + +### Increment MAJOR when + +- A decision point is **added to or removed from** the group, OR +- Any constituent decision point increments its **own major version**. + +### Increment MINOR when + +- Any constituent decision point increments its **own minor version**. + +### Increment PATCH when + +- Any constituent decision point increments its **own patch version**, OR +- The group **description** changes, OR +- The group **name** changes. + +--- + +## JSON Schema Versioning (SchemaVer) + +SSVC JSON schema files use **SchemaVer** (`MODEL-REVISION-ADDITION`, starting +at `1-0-0`) rather than SemVer. The distinction matters because "breaking" for +a schema means *existing stored data will no longer validate* — a different +concern from API-level breaking changes. + +| Component | When to increment | Data-compatibility implication | +|-----------|-------------------|-------------------------------| +| `MODEL` | Incompatible with **any** historical data | Existing data **will not** validate | +| `REVISION` | Incompatible with **some** historical data | Existing data **may not** validate | +| `ADDITION` | Compatible with **all** historical data | Existing data **will** continue to validate | + +### Increment MODEL when + +- A **required field is removed**. +- A field's **type or allowed values** change in a way that invalidates previously valid data. +- The **root schema structure** changes (e.g. from object to array). + +### Increment REVISION when + +- A previously **optional field becomes required**. +- An `enum` gains values that affect **round-trip processing** for some consumers. +- Constraints are **tightened** such that some existing data would fail. + +### Increment ADDITION when + +- A **new optional field** is added. +- A constraint is **relaxed**. +- **Documentation** (`description`, `$comment`) is updated without changing validation behavior. +- An `enum` gains new allowed values that **only expand** what is valid. + +### Relationship to SemVer on the same objects + +A `DecisionPoint` can be at SemVer `2.1.0` while its JSON schema is at +SchemaVer `1-2-3`. These are independent versioning axes: + +- SemVer on the object answers: *"Has the concept or its option space changed?"* +- SchemaVer on the schema answers: *"Will my stored data still validate?"* + +--- + +## Project / Documentation Versioning (CalVer) + +The overall SSVC project and documentation releases use **Calendar Versioning** +(`YYYY.M.patch`): + +- `YYYY` = four-digit release year. +- `M` = single-digit month (no leading zero). +- `patch` = incremented for subsequent updates within the same month. + +*Examples:* `2025.6.0` (first June 2025 release), `2025.6.1` (second), +`2025.11.0` (November 2025). + +CalVer suits SSVC as a *living framework* because: + +- The version immediately communicates **recency** without counting features. +- Documentation changes that do not alter any domain object can still produce + a new project version. +- It avoids unproductive debates about whether a change is "major" or "minor" + at the project level. + +Individual domain objects continue to use SemVer regardless of the project +CalVer; the two schemes are complementary. + +--- + +## See Also + +- ADR-0002: [Decision Points are Versioned using SemVer](../docs/adr/0002-ssvc-decision-points-are-versioned.md) +- ADR-0005: [Decision Point Group Versioning Rules](../docs/adr/0005-ssvc-decision-point-group-versioning.md) +- ADR-0006: [Decision Point Versioning Rules](../docs/adr/0006-ssvc-decision-point-versioning-rules.md) +- ADR-0013: [SSVC Project Versions Follow CalVer](../docs/adr/0013-ssvc-project-versions.md) +- ADR-0015: [SSVC JSON Schemas Use SchemaVer](../docs/adr/0015-ssvc-json-schemas-use-schemaver.md) +- Spec VR: `specs/versioning.yaml` diff --git a/plan/BUILD_LEARNINGS.md b/plan/BUILD_LEARNINGS.md new file mode 100644 index 00000000..b4a25794 --- /dev/null +++ b/plan/BUILD_LEARNINGS.md @@ -0,0 +1,15 @@ +# Build Learnings + +This file is an ephemeral queue of observations from build and bugfix sessions. + +**Agents:** Read this file during `study-project-docs` and act on any critical +insights. If an insight is worth keeping permanently, move it to the appropriate +`notes/.md`, `docs/adr/`, or a GitHub Issue before the session ends. +Do not let valuable observations accumulate here indefinitely. + +**Humans:** Add brief notes here when you notice recurring issues, surprises, +or non-obvious constraints during development. Keep entries short and dated. + +--- + + diff --git a/plan/PRIORITIES.md b/plan/PRIORITIES.md new file mode 100644 index 00000000..fe6bd976 --- /dev/null +++ b/plan/PRIORITIES.md @@ -0,0 +1,20 @@ +# Priorities + +## Important note about priority numbers + +Priority numbers are ascending, so lower numbers are higher priority. +The scale is not linear, it's just intended to provide a rough ordering and +allow for space between to add new priorities in the future if needed. The +priority numbers themselves do not have any inherent meaning beyond their +relative order. Completed priorities should be archived via `uv run append-history priority` +(writes to `plan/history/YYMM/priority/`) and then removed from this file to keep +`plan/PRIORITIES.md` focused on pending and in-progress work. + +Each priority group should have a corresponding GitHub Issue of type `Epic` +that tracks the overall work as child issues (which may in turn have their +own child issues, etc.) The list of child issues in GitHub is +authoritative regardless what is listed below, this file is a high-level +index and summary. + +## Priority 100 + diff --git a/plan/history/ideas.md b/plan/history/ideas.md new file mode 100644 index 00000000..d4d6885f --- /dev/null +++ b/plan/history/ideas.md @@ -0,0 +1,45 @@ +# Idea History + +This file archives processed GitHub Idea-type issues. Each entry records the +original idea and a reference to where the design decisions were captured. + +--- + +## IDEA-1143: Add versioning rules for DecisionTable objects + +Add versioning rules for DecisionTable objects + +Currently, SSVC's `DecisionTable` objects lack versioning rules. This is a +significant omission — without versioning conventions, it is unclear how to +track changes to decision tables, how to communicate compatibility, or how to +deprecate old versions. We should define versioning semantics (e.g., semantic +versioning), rules for when a version bump is required, and how versions +should be represented in the Python model and JSON/CSV data files. + +**Processed**: 2026-05-18 — design decisions captured in +`specs/versioning.yaml` (new VR-05 group, VR-05-001 through VR-05-007). +Also: VR-01-006 downgraded from SHOULD to MAY; VR-02 tightened to cover +only DecisionPointGroup (DecisionTable reference removed). + +--- + +## IDEA-1151: Establish two-tier skill directory structure and canonical skill interface + +Bootstrap the skills infrastructure in the SSVC repository by establishing a +two-tier directory structure (`.agents/skills/dev/` and `.agents/skills/ssvc/`) +and defining a canonical SKILL.md format. Replaces the flat `.agents/skills/` +layout and lays the groundwork for SSVC domain skills. + +Key deliverables: restructure skill directories; add placeholder +`.agents/skills/ssvc/evaluate-decision-point/SKILL.md` (WIP stub); define +canonical SKILL.md frontmatter (required: `name` + `description` only); update +existing dev skills to conform; add `.agents/skills/README.md`; add +`specs/skills.yaml` (prefix SK); add CI validation workflow +`.github/workflows/validate_skills.yml`. + +Design decisions: skills stay under `.agents/`; required front-matter is +`name` + `description` only; CI uses a new standalone workflow with inline +Python filtered to `.agents/skills/**/SKILL.md`. + +**Processed**: 2026-05-20 — design decisions captured in +`specs/skills.yaml` and `notes/skills.md`. diff --git a/pyproject.toml b/pyproject.toml index f5104461..06f72945 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -49,12 +49,15 @@ dependencies = [ "pydantic>=2.11.7", "semver>=3.0.4", "fastapi[all,standard]>=0.116.1", + "pyyaml>=6.0", ] dynamic = ["version",] [project.scripts] ssvc_csv_analyzer="ssvc.csv_analyzer:main" ssvc_doctools="ssvc.doctools:main" +ssvc-spec-lint="ssvc.metadata.specs.lint:main" +ssvc-spec-dump="ssvc.metadata.specs.render:main_llm_json" [project.urls] "Homepage" = "https://certcc.github.io/SSVC" @@ -62,7 +65,7 @@ ssvc_doctools="ssvc.doctools:main" "Bug Tracker" = "https://github.com/CERTCC/SSVC/issues" [tool.setuptools.packages.find] -where = ["."] # list of folders that contain the packages (["."] by default) +where = ["src"] # packages live under src/ include = ["ssvc*"] # package names should match these glob patterns (["*"] by default) exclude = ["test*"] # exclude packages matching these glob patterns (empty by default) #namespaces = false # to disable scanning PEP 420 namespaces (true by default) diff --git a/specs/README.md b/specs/README.md new file mode 100644 index 00000000..7a932a32 --- /dev/null +++ b/specs/README.md @@ -0,0 +1,132 @@ +# Specifications + +This directory contains the normative requirement specifications for the SSVC +project, stored as structured YAML files and managed by the +`ssvc.metadata.specs` toolchain. + +## Purpose + +Spec files replace ad-hoc requirement lists with validated, machine-readable +YAML that can be linted, rendered to Markdown, exported to JSON, and consumed +by coding agents. The Python source in `src/ssvc/metadata/specs/` implements +the schema, registry, linter, and exporters. + +## Spec Files + +| File | ID | Kind | Topic | +|------|----|------|-------| +| `api.yaml` | AP | implementation | FastAPI REST API structure, response format, and error handling | +| `ci.yaml` | CI | general | Continuous integration gates and deployment automation | +| `codegen.yaml` | GN | implementation | Code generation source authority and artifact management | +| `domain-model.yaml` | DM | domain | SSVC object model: decision points, tables, outcomes, mixins | +| `registry.yaml` | RG | implementation | In-memory object registry structure, registration, and lookup | +| `spec-registry.yaml` | SR | general | Requirements for the spec file schema and toolchain itself | +| `testing.yaml` | TS | language | Test organisation, isolation, and quality standards | +| `versioning.yaml` | VR | domain | Versioning rules for decision points (SemVer), JSON schemas (SchemaVer), and the project (CalVer) | +| `namespaces.yaml` | NS | domain | Registered and extension namespace rules for SSVC domain objects | +| `skills.yaml` | SK | general | Agent skill infrastructure: directory layout, SKILL.md interface, two-tier taxonomy, and CI enforcement | + +### ID Prefix Convention + +Each file owns a two-to-eight letter uppercase prefix. All group IDs and spec +IDs within a file must start with that prefix: + +``` + e.g. DM +- e.g. DM-01 (group) +-- e.g. DM-01-001 (spec) +``` + +### SpecKind Portability Tiers + +| Kind | Applies to | +|------|-----------| +| `general` | Any project, any language | +| `pattern` | Architectural approach; language-agnostic | +| `domain` | SSVC problem domain; language-agnostic | +| `language` | Python ecosystem | +| `implementation` | This specific codebase | + +## Tooling + +### Lint + +Validate all spec files in this directory: + +```bash +uv run ssvc-spec-lint specs/ +# or +uv run python -m ssvc.metadata.specs.lint specs/ +``` + +Exit code `0` = no hard errors. Hard errors (duplicate IDs, dangling +cross-references, prefix mismatches) exit with code `1`. Advisory warnings +are printed to stdout but do not affect the exit code. + +### Render + +Render all specs as Markdown (human-readable): + +```bash +uv run python -m ssvc.metadata.specs.render --format md specs/ +``` + +Export as JSON (filterable): + +```bash +uv run python -m ssvc.metadata.specs.render --format json specs/ +``` + +### LLM Export + +Dump a flat, inheritance-resolved JSON projection for coding agents: + +```bash +uv run ssvc-spec-dump +# or filter to a single topic +uv run python -m ssvc.metadata.specs.render --format llm-json --topic DM specs/ +``` + +## Adding a New Spec File + +1. Choose a unique uppercase prefix (two to eight letters, e.g. `MY`). +2. Create `specs/my-topic.yaml` following the structure below. +3. Run `uv run ssvc-spec-lint specs/` — fix any reported errors. +4. Add a row to the table in this README. + +Minimal skeleton: + +```yaml +id: MY +title: My Topic +description: > + One-paragraph description of the topic covered by this file. +version: "0.1.0" +kind: implementation # general | pattern | domain | language | implementation +scope: [production] # prototype | production + +groups: + - id: MY-01 + title: First Group + description: What this group covers. + specs: + - id: MY-01-001 + priority: MUST # MUST | MUST_NOT | SHOULD | SHOULD_NOT | MAY + statement: > + MY-01-001 The system MUST do the thing. + rationale: > + One sentence explaining why this requirement exists. + testable: true + tags: [tooling] # ci-cd | code-style | documentation | + # performance | security | testing | tooling +``` + +### Requirement Authoring Checklist + +- [ ] Statement is **atomic**: covers exactly one verifiable behaviour. +- [ ] Statement **includes the spec ID** at the start (e.g. `MY-01-001 MUST …`). +- [ ] `rationale` explains *why*, not just *what*. +- [ ] `testable: true` unless the requirement can only be verified by human + inspection; if `false`, add `lint_suppress: [testable_without_steps]`. +- [ ] At least one `tag` is set. +- [ ] No requirement text is duplicated in another spec file. diff --git a/specs/api.yaml b/specs/api.yaml new file mode 100644 index 00000000..46b05464 --- /dev/null +++ b/specs/api.yaml @@ -0,0 +1,112 @@ +id: AP +title: REST API +description: > + Requirements for the FastAPI-based read-only REST API that exposes the + SSVC object registry over HTTP. The API serves decision points, decision + tables, outcomes, and decision point groups. +version: "0.1.0" +kind: implementation +scope: [production] + +groups: + - id: AP-01 + title: API Structure + description: Requirements for how the API is organised and routed. + specs: + - id: AP-01-001 + priority: MUST + statement: > + AP-01-001 The API MUST be implemented using FastAPI and expose all + routes under a versioned URL prefix (/v1). + rationale: > + A versioned prefix allows future breaking changes to be introduced + under a new version without disrupting existing clients. + testable: true + tags: [tooling] + + - id: AP-01-002 + priority: MUST + statement: > + AP-01-002 Each resource type (decision_points, decision_tables, + outcomes, dp_groups) MUST be served by a dedicated router module + under ssvc.api.v1.routers. + rationale: > + Separating resource routers keeps each module focused on one + resource, simplifies testing, and avoids a single large router + file. + testable: true + tags: [tooling] + + - id: AP-01-003 + priority: MUST + statement: > + AP-01-003 All currently exposed API endpoints MUST be read-only + (HTTP GET). + rationale: > + The registry is populated from Python source at startup; there is + no runtime write path, so all mutations must be made in source. + testable: true + tags: [security] + + - id: AP-02 + title: Response Format + description: Requirements for API response serialisation and typing. + specs: + - id: AP-02-001 + priority: MUST + statement: > + AP-02-001 All API responses MUST be serialised as JSON using + Pydantic response models. + rationale: > + Pydantic response models provide automatic validation of output + shape, OpenAPI schema generation, and consistent serialisation. + testable: true + tags: [tooling] + + - id: AP-02-002 + priority: MUST + statement: > + AP-02-002 Path parameters that identify SSVC objects (namespace, + key, version) MUST be declared as typed FastAPI path parameters. + rationale: > + Typed path parameters enable FastAPI to reject malformed requests + at the routing layer before they reach business logic. + testable: true + tags: [tooling] + + - id: AP-02-003 + priority: MUST + statement: > + AP-02-003 API response models MUST NOT expose internal registry + data structures (e.g. _NsType, _Key, _Version). + rationale: > + Leaking internal types couples clients to implementation details + and makes refactoring harder. + testable: true + tags: [tooling] + + - id: AP-03 + title: Error Handling + description: Requirements for how the API reports errors to clients. + specs: + - id: AP-03-001 + priority: MUST + statement: > + AP-03-001 A request for a non-existent SSVC object MUST return + HTTP 404 with a descriptive error body. + rationale: > + 404 is the correct HTTP semantic for "this resource does not + exist"; a descriptive body helps callers diagnose incorrect keys. + testable: true + tags: [tooling] + + - id: AP-03-002 + priority: MUST + statement: > + AP-03-002 A request with an invalid path parameter type MUST + return HTTP 422 (Unprocessable Entity). + rationale: > + FastAPI returns 422 for validation failures; this requirement + documents the expected contract so tests can assert on it. + testable: true + tags: [tooling] diff --git a/specs/ci.yaml b/specs/ci.yaml new file mode 100644 index 00000000..b339face --- /dev/null +++ b/specs/ci.yaml @@ -0,0 +1,91 @@ +id: CI +title: CI/CD Integration +description: > + Requirements for continuous integration, automated testing gates, and + deployment automation. The project uses GitHub Actions for CI, GitHub + Pages for documentation hosting, and Dependabot for dependency updates. +version: "0.1.0" +kind: general +scope: [production] + +groups: + - id: CI-01 + title: Continuous Integration + description: Requirements for automated checks on every pull request. + specs: + - id: CI-01-001 + priority: MUST + statement: > + CI-01-001 CI MUST run all pytest tests on every pull request. + rationale: > + Running the full test suite on every PR prevents regressions from + being merged and provides a fast feedback loop for contributors. + testable: false + lint_suppress: [testable_without_steps] + tags: [ci-cd, testing] + + - id: CI-01-002 + priority: MUST + statement: > + CI-01-002 CI MUST execute make dev (or equivalent) to generate + namespace_patterns.py before running tests. + rationale: > + namespace_patterns.py is a generated file required at test and + API startup; omitting the generation step causes CI failures that + do not reflect real code defects. + testable: false + lint_suppress: [testable_without_steps] + tags: [ci-cd, tooling] + + - id: CI-01-003 + priority: SHOULD + statement: > + CI-01-003 CI SHOULD run ssvc-spec-lint on every pull request that + modifies files under specs/. + rationale: > + Running the linter in CI enforces spec authoring rules automatically + and catches errors before they are merged. + testable: false + lint_suppress: [testable_without_steps] + tags: [ci-cd, tooling] + + - id: CI-01-004 + priority: SHOULD + statement: > + CI-01-004 CI SHOULD lint all Markdown files using markdownlint on + every pull request. + rationale: > + Automated Markdown linting keeps documentation consistent and + catches formatting errors that degrade rendered output. + testable: false + lint_suppress: [testable_without_steps] + tags: [ci-cd, documentation] + + - id: CI-02 + title: Deployment + description: Requirements for automated release and documentation deployment. + specs: + - id: CI-02-001 + priority: MUST + statement: > + CI-02-001 The MkDocs documentation site MUST be deployed to + GitHub Pages on every merge to the default branch. + rationale: > + Automatic deployment keeps the published documentation in sync + with the source without requiring manual intervention. + testable: false + lint_suppress: [testable_without_steps] + tags: [ci-cd, documentation] + + - id: CI-02-002 + priority: SHOULD + statement: > + CI-02-002 Python package releases SHOULD be published to PyPI + automatically via a GitHub Actions workflow triggered by a version + tag. + rationale: > + Automating package publication reduces the risk of manual release + errors and ensures a consistent release process. + testable: false + lint_suppress: [testable_without_steps] + tags: [ci-cd] diff --git a/specs/codegen.yaml b/specs/codegen.yaml new file mode 100644 index 00000000..0be6a2b9 --- /dev/null +++ b/specs/codegen.yaml @@ -0,0 +1,103 @@ +id: GN +title: Code Generation +description: > + Requirements governing the generated artifacts derived from the SSVC + Python source: JSON schema files, CSV decision tables, and the + namespace_patterns.py regex module. Python modules are the authoritative + source; all generated files are downstream artifacts. +version: "0.1.0" +kind: implementation +scope: [production] + +groups: + - id: GN-01 + title: Source Authority + description: > + Requirements establishing Python source as the single source of truth + for all SSVC data objects. + specs: + - id: GN-01-001 + priority: MUST + statement: > + GN-01-001 Python module definitions MUST be the authoritative + source for all SSVC domain objects; data/json/ and data/csv/ + artifacts MUST NOT be hand-edited. + rationale: > + Editing generated files by hand creates drift between source and + artifacts that is silently overwritten on the next regeneration, + losing the edits. + testable: false + lint_suppress: [testable_without_steps] + tags: [tooling] + + - id: GN-01-002 + priority: MUST + statement: > + GN-01-002 src/ssvc/utils/namespace_patterns.py MUST be generated + from the ABNF grammar file (ssvc_namespace_pattern.abnf) via + make dev and MUST NOT be hand-edited. + rationale: > + The regex is derived mechanically from the grammar; manual edits + diverge from the grammar and are overwritten by make dev. + testable: false + lint_suppress: [testable_without_steps] + tags: [tooling] + + - id: GN-02 + title: Artifact Generation + description: Requirements for how and when artifacts are regenerated. + specs: + - id: GN-02-001 + priority: MUST + statement: > + GN-02-001 make regenerate_json MUST regenerate all files in + data/json/ and data/csv/ from the current Python source without + requiring additional arguments. + rationale: > + A single no-argument command lowers the barrier to keeping + artifacts up to date and reduces the risk of contributors + forgetting required flags. + testable: true + tags: [tooling] + + - id: GN-02-002 + priority: MUST + statement: > + GN-02-002 Generated artifact files (data/json/, data/csv/, + namespace_patterns.py) MUST be committed to the repository so + they are available without running a build step. + rationale: > + Committing generated files allows the package and its schemas to + be consumed immediately after checkout without requiring a local + build environment. + testable: false + lint_suppress: [testable_without_steps] + tags: [tooling] + + - id: GN-02-003 + priority: SHOULD + statement: > + GN-02-003 CI SHOULD detect artifact drift by regenerating + data/json/, data/csv/, and namespace_patterns.py and comparing + the result against committed files. + rationale: > + Drift detection in CI catches cases where a source change was + committed without regenerating its downstream artifacts. + testable: false + lint_suppress: [testable_without_steps] + tags: [ci-cd, tooling] + + - id: GN-02-004 + priority: MUST + statement: > + GN-02-004 Generated JSON schema files MUST carry a SchemaVer + identifier in the schemaVersion field and in the $id URI, using + the MODEL-REVISION-ADDITION format (e.g. 1-0-0) rather than + SemVer dot notation. + rationale: > + SchemaVer communicates data-compatibility implications directly: + consumers can determine from the version string alone whether + their existing stored data will still validate, without consulting + a changelog. + testable: true + tags: [tooling] diff --git a/specs/domain-model.yaml b/specs/domain-model.yaml new file mode 100644 index 00000000..36d1d31e --- /dev/null +++ b/specs/domain-model.yaml @@ -0,0 +1,324 @@ +id: DM +title: Domain Model +description: > + Requirements for the SSVC domain object model: decision points, decision + tables, outcomes, and the shared mixin abstractions that compose them. + Python modules are the authoritative source; JSON and CSV artifacts are + generated from them. +version: "0.1.0" +kind: domain +scope: [production] + +groups: + - id: DM-01 + title: Object Identity + description: > + Rules that govern how SSVC domain objects are uniquely named and + versioned. + specs: + - id: DM-01-001 + priority: MUST + statement: > + DM-01-001 Each domain object MUST be uniquely identified by the + composite key namespace:key:version. + rationale: > + A stable three-part key allows objects from different namespaces + and sources to coexist in the same registry without collision. + testable: true + tags: [tooling] + + - id: DM-01-002 + priority: MUST + statement: > + DM-01-002 The namespace field of every domain object MUST hold a + value from the NameSpace enum defined in ssvc.namespaces. + rationale: > + Restricting namespaces to an enum prevents arbitrary strings from + polluting the registry and enables exhaustive validation. + testable: true + tags: [tooling] + + - id: DM-01-003 + priority: MUST + statement: > + DM-01-003 The version field of every domain object MUST be a valid + semantic version string (semver 2.0.0). + rationale: > + Semantic versioning enables ordered comparison, range filtering, + and clear communication about breaking changes between releases. + testable: true + tags: [tooling] + + - id: DM-02 + title: Decision Point Requirements + description: Requirements for DecisionPoint objects. + specs: + - id: DM-02-001 + priority: MUST + statement: > + DM-02-001 Each DecisionPoint MUST declare a non-empty list of + values. + rationale: > + A decision point with no values cannot be used as a table input + and is therefore invalid at construction time. + testable: true + tags: [tooling] + + - id: DM-02-002 + priority: MUST + statement: > + DM-02-002 All value key fields within a single DecisionPoint MUST + be unique. + rationale: > + Duplicate keys within a decision point make table row generation + ambiguous and produce incorrect cartesian products. + testable: true + tags: [tooling] + + - id: DM-02-003 + priority: MUST + statement: > + DM-02-003 Each DecisionPointValue MUST have a non-empty key and a + non-empty name. + rationale: > + Both fields are required for table generation and human-readable + output; absent values produce malformed artifacts. + testable: true + tags: [tooling] + + - id: DM-02-004 + priority: MUST + statement: > + DM-02-004 DecisionPoint definitions MUST reside in submodules + under ssvc.decision_points. + rationale: > + Centralising decision point definitions in one package subtree + ensures auto-import covers all instances and keeps the module + boundary clear. + testable: true + tags: [tooling] + + - id: DM-02-005 + priority: MUST_NOT + statement: > + DM-02-005 Description fields of DecisionPoint, DecisionPointValue, + OutcomeSet, and OutcomeValue objects MUST NOT embed worked examples + (e.g. "for example …", "an instance of this is …"). + rationale: > + Examples grow stale independently of the concept they illustrate, + causing unnecessary version increments when only an example + changes. Examples belong in surrounding documentation text. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: DM-02-006 + priority: SHOULD + statement: > + DM-02-006 Examples that would otherwise appear in object description + fields SHOULD be placed in the surrounding MkDocs documentation + pages or in the object's associated notes file. + rationale: > + Separating examples from definitions keeps object semantics stable + and allows examples to evolve independently without triggering + object version changes. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: DM-03 + title: Decision Table Requirements + description: Requirements for DecisionTable objects. + specs: + - id: DM-03-001 + priority: MUST + statement: > + DM-03-001 Each DecisionTable MUST reference at least one + DecisionPoint as an input. + rationale: > + A table with no inputs cannot generate rows and is not a valid + decision model. + testable: true + tags: [tooling] + + - id: DM-03-002 + priority: MUST + statement: > + DM-03-002 A DecisionTable MUST map every combination of its input + DecisionPoint values to an outcome (complete coverage). + rationale: > + Incomplete tables leave undefined outcomes for valid inputs, + which would cause silent failures during scoring. + testable: true + tags: [tooling] + + - id: DM-03-003 + priority: MUST + statement: > + DM-03-003 DecisionTable definitions MUST reside in submodules + under ssvc.decision_tables. + rationale: > + Centralising decision table definitions ensures auto-import covers + all instances and maintains a consistent module boundary. + testable: true + tags: [tooling] + + - id: DM-04 + title: Outcome Requirements + description: Requirements for outcome and action label objects. + specs: + - id: DM-04-001 + priority: MUST + statement: > + DM-04-001 Each outcome set MUST define at least one outcome value. + rationale: > + An empty outcome set cannot be referenced by a decision table + and is invalid at construction time. + testable: true + tags: [tooling] + + - id: DM-04-002 + priority: MUST + statement: > + DM-04-002 Outcome values within a set MUST have unique keys. + rationale: > + Duplicate outcome keys make table output ambiguous and prevent + deterministic scoring. + testable: true + tags: [tooling] + + - id: DM-04-003 + priority: MUST + statement: > + DM-04-003 Outcome definitions MUST reside in submodules under + ssvc.outcomes. + rationale: > + Co-locating all outcome definitions in one package subtree ensures + auto-import covers them and keeps the module boundary clear. + testable: true + tags: [tooling] + + - id: DM-05 + title: Mixin Composition + description: > + Cross-cutting requirements for how capabilities are added to domain + objects via the shared mixin classes in ssvc._mixins. + specs: + - id: DM-05-001 + priority: MUST + statement: > + DM-05-001 All domain model objects MUST compose the _Registered + mixin from ssvc._mixins to auto-register upon instantiation. + rationale: > + Auto-registration decouples object definition from registry + insertion: instantiating an object is sufficient to make it + available for lookup. + testable: true + tags: [tooling] + + - id: DM-05-002 + priority: MUST + statement: > + DM-05-002 All domain model objects MUST use Pydantic v2 BaseModel + as their base class. + rationale: > + Pydantic v2 provides construction-time validation, JSON schema + generation, and serialization, all of which the toolchain depends + on. + testable: true + tags: [tooling] + + - id: DM-05-003 + priority: MUST + statement: > + DM-05-003 Domain model invariants (e.g. unique value keys, valid + namespaces) MUST be enforced by Pydantic field_validator or + model_validator at construction time. + rationale: > + Failing fast at construction prevents invalid objects from ever + reaching the registry or being serialised to output artifacts. + testable: true + tags: [tooling] + + - id: DM-06 + title: Ordering Constraints + description: > + Requirements for the ordering of values within decision points and + outcome sets. Implements ADR-0008 and ADR-0009. + specs: + - id: DM-06-001 + priority: MUST + statement: > + DM-06-001 The values list of every DecisionPoint MUST be an + ordered sequence such that the index of each value reflects its + relative severity or urgency (lower index = lower severity/urgency). + rationale: > + An ordered value set enables partial-order reasoning across + combinations of decision point values, which in turn supports + automated policy generation and validation of monotonic policies. + testable: true + tags: [tooling] + + - id: DM-06-002 + priority: MUST + statement: > + DM-06-002 The values list of every OutcomeSet MUST be an ordered + sequence such that the index of each outcome reflects its relative + urgency of response (lower index = lower urgency). + rationale: > + Ordering outcomes enables the policy monotonicity invariant: a + higher-severity input combination must map to an outcome of equal + or greater urgency than any lower-severity input combination. + testable: true + tags: [tooling] + + - id: DM-06-003 + priority: MUST + statement: > + DM-06-003 The ordering of values within a DecisionPoint or + OutcomeSet MUST be stable across serialisation and + deserialisation (i.e. round-trip JSON/CSV encoding MUST preserve + the declared order). + rationale: > + If ordering is lost during serialisation, downstream consumers + may reconstruct the wrong partial order, silently producing + invalid policies. + testable: true + tags: [tooling] + + - id: DM-07 + title: Separation of Concerns + description: > + Requirements that enforce the separation between the structure of a + decision model and the policy that maps inputs to outcomes. Implements + ADR-0010. + specs: + - id: DM-07-001 + priority: MUST_NOT + statement: > + DM-07-001 A DecisionTable definition MUST NOT embed an OutcomeSet + as part of its structural identity; the outcome set is a separate, + stakeholder-specific concern. + rationale: > + Including the outcome set in the table definition would force + versioning events on the table whenever the outcome set changes, + even when the input structure is unchanged. Two organisations + sharing the same table structure but different outcome categories + would appear to use different tables. + testable: true + tags: [tooling] + + - id: DM-07-002 + priority: MUST_NOT + statement: > + DM-07-002 Policy mappings (the specific assignment of outcome + values to input-value combinations) MUST NOT be embedded in the + DecisionTable definition; policies are stakeholder-specific and + must be expressed separately. + rationale: > + Embedding a policy in the table definition conflates the + reusable decision structure with the organisation-specific + prioritisation choices, preventing table sharing and independent + policy versioning. + testable: true + tags: [tooling] diff --git a/specs/namespaces.yaml b/specs/namespaces.yaml new file mode 100644 index 00000000..1d1aea35 --- /dev/null +++ b/specs/namespaces.yaml @@ -0,0 +1,136 @@ +id: NS +title: Namespaces +description: > + Requirements for how SSVC objects are identified by namespace. Namespaces + distinguish objects managed by the SSVC project from objects derived from + external sources, and enable private or experimental extensions. Registered + namespaces are controlled by the SSVC project; unregistered extension + namespaces follow a prescribed format so that third parties can safely + extend SSVC without collision. Implements ADR-0012. +version: "0.1.0" +kind: domain +scope: [production] + +groups: + - id: NS-01 + title: Registered Namespaces + description: > + Requirements for objects that use namespaces managed by the SSVC project. + specs: + - id: NS-01-001 + priority: MUST + statement: > + NS-01-001 Every SSVC domain object MUST declare a namespace whose + value is either a member of the NameSpace enum in ssvc.namespaces + or a valid extension namespace string beginning with the x_ prefix. + rationale: > + Restricting the namespace to known values or the governed extension + prefix prevents arbitrary strings from polluting the registry and + enables exhaustive validation of object origin. + testable: true + tags: [tooling] + + - id: NS-01-002 + priority: MUST + statement: > + NS-01-002 The NameSpace enum in ssvc.namespaces MUST be the + authoritative list of registered namespace values; no registered + namespace value may be added by modifying only runtime data or + configuration files. + rationale: > + Embedding the canonical namespace list in a Python enum that is + version-controlled ensures traceability: every registered namespace + change goes through a code review process. + testable: true + tags: [tooling] + + - id: NS-01-003 + priority: MUST + statement: > + NS-01-003 Namespace validation MUST be enforced at object + construction time using the NameSpace.validate() method or + equivalent Pydantic field validator, rejecting any value that is + neither a NameSpace enum member nor a valid extension namespace. + rationale: > + Construction-time validation prevents invalid namespace strings + from reaching the registry or being serialised into output + artifacts. + testable: true + tags: [tooling] + + - id: NS-01-004 + priority: MUST_NOT + statement: > + NS-01-004 Reserved namespace strings listed in ssvc.namespaces + MUST NOT be used as the base of any namespace string (registered + or extension), as they are protected identifiers in the SSVC + specification. + rationale: > + Reserved strings prevent collision with future official namespaces + and protect the integrity of the specification against accidental + or deliberate hijacking. + testable: true + tags: [tooling] + + - id: NS-02 + title: Extension Namespaces + description: > + Requirements for unregistered (extension) namespaces used by + third-party adopters who wish to create custom SSVC objects or refine + existing ones without registering with the SSVC project. + specs: + - id: NS-02-001 + priority: MUST + statement: > + NS-02-001 All unregistered extension namespace strings MUST begin + with the prefix x_ (underscore immediately after the letter x). + rationale: > + The x_ prefix is the universal signal for an experimental or + private identifier; it prevents collisions with current and future + registered namespaces. + testable: true + tags: [tooling] + + - id: NS-02-002 + priority: MUST + statement: > + NS-02-002 Extension namespace strings MUST follow the format + x_# (e.g. x_example.agency#internal) + to ensure global uniqueness without central registration. + rationale: > + Reverse-domain notation delegates uniqueness to DNS ownership, + which is already managed globally; the fragment allows a single + domain owner to define multiple distinct namespaces. + testable: true + tags: [tooling] + + - id: NS-02-003 + priority: MAY + statement: > + NS-02-003 A namespace extension (a refinement of an existing + registered or extension namespace) MAY use the double-slash + extension separator in the form + //# to indicate that + the extension refines the semantics of the base namespace for a + specific constituency. + rationale: > + Namespace extensions allow ISAOs, government agencies, or other + constituencies to add nuance to existing decision point semantics + without creating a wholly independent namespace. + testable: true + tags: [tooling] + + - id: NS-02-004 + priority: SHOULD + statement: > + NS-02-004 Objects in an extension namespace SHOULD clearly + document which registered or extension namespace they extend or + are compatible with, so that consumers can determine the degree + of interoperability. + rationale: > + Without provenance documentation, consumers of extension-namespace + objects cannot determine whether they are compatible with objects + from the base SSVC namespace. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] diff --git a/specs/registry.yaml b/specs/registry.yaml new file mode 100644 index 00000000..695e95c6 --- /dev/null +++ b/specs/registry.yaml @@ -0,0 +1,118 @@ +id: RG +title: Object Registry +description: > + Requirements for the in-memory SsvcObjectRegistry singleton that stores + all domain objects and serves as the backing store for the REST API. +version: "0.1.0" +kind: implementation +scope: [production] + +groups: + - id: RG-01 + title: Registry Structure + description: > + Requirements for the shape and lifecycle of the registry singleton. + specs: + - id: RG-01-001 + priority: MUST + statement: > + RG-01-001 The package MUST maintain a single SsvcObjectRegistry + instance per Python process. + rationale: > + A singleton registry ensures that all code paths consult the same + object store and that there is exactly one authoritative lookup + point per process. + testable: true + tags: [tooling] + + - id: RG-01-002 + priority: MUST + statement: > + RG-01-002 The registry MUST organise objects in a four-level index: + type → namespace → key → version. + rationale: > + A structured index enables efficient lookup by any combination of + type, namespace, key, and version without scanning all objects. + testable: true + tags: [tooling] + + - id: RG-01-003 + priority: MUST + statement: > + RG-01-003 The registry MUST expose a reset(force=True) method that + clears all registered objects. + rationale: > + Test suites require a clean registry before each test case to + prevent cross-test contamination from module-level singletons. + testable: true + tags: [testing] + + - id: RG-02 + title: Registration Behavior + description: Requirements for how objects enter the registry. + specs: + - id: RG-02-001 + priority: MUST + statement: > + RG-02-001 Registering an object whose namespace:key:version + composite key already exists in the registry MUST raise ValueError. + rationale: > + Duplicate keys indicate conflicting object definitions; failing + loudly at startup surfaces the conflict before it causes silent + data errors. + testable: true + tags: [tooling] + + - id: RG-02-002 + priority: MUST + statement: > + RG-02-002 The _Registered.model_post_init hook MUST call + notify_registration to insert the object into the registry + immediately upon construction. + rationale: > + Registering in model_post_init ensures every valid domain object + is automatically available for lookup without requiring callers to + register manually. + testable: true + tags: [tooling] + + - id: RG-02-003 + priority: MUST + statement: > + RG-02-003 ssvc.__init__ MUST call import_modules to auto-import + all submodules under ssvc.decision_points, ssvc.outcomes, + ssvc.decision_tables, and ssvc.dp_groups. + rationale: > + Auto-import at package init guarantees that every domain object is + registered before any consumer queries the registry, without + requiring explicit per-module imports. + testable: true + tags: [tooling] + + - id: RG-03 + title: Lookup Behavior + description: Requirements for reading objects from the registry. + specs: + - id: RG-03-001 + priority: MUST + statement: > + RG-03-001 registry.lookup(type, namespace, key, version) MUST + return the matching registered object or raise a descriptive + exception when no match is found. + rationale: > + Callers must receive a clear error, not None or a silent default, + so that missing objects surface as explicit failures rather than + incorrect downstream results. + testable: true + tags: [tooling] + + - id: RG-03-002 + priority: MUST + statement: > + RG-03-002 The registry MUST expose a method to retrieve all + registered objects of a given type. + rationale: > + Bulk retrieval is required for the API list endpoints and for + doctools artifact generation. + testable: true + tags: [tooling] diff --git a/specs/skills.yaml b/specs/skills.yaml new file mode 100644 index 00000000..7dea39c0 --- /dev/null +++ b/specs/skills.yaml @@ -0,0 +1,190 @@ +id: SK +title: Skills +description: > + Requirements for the agent skill infrastructure: directory layout, canonical + SKILL.md interface, the two-tier (dev vs. domain) taxonomy, and CI + enforcement. Skills are Markdown-based capability descriptors consumed by + AI coding agents. Dev skills support SSVC project development workflows; + SSVC domain skills help practitioners perform SSVC analysis work. +version: "0.1.0" +kind: general +scope: [production] + +groups: + - id: SK-01 + title: Directory Layout + description: > + Requirements for where skill directories live and how they are organised + within the repository. + specs: + - id: SK-01-001 + priority: MUST + statement: > + SK-01-001 All agent skills MUST reside under `.agents/skills/` at + the repository root. + rationale: > + Keeping skills under `.agents/` makes the directory's machine-facing + purpose explicit and separates agent tooling from production source + code and documentation. + testable: true + tags: [tooling] + + - id: SK-01-002 + priority: MUST + statement: > + SK-01-002 Dev skills (those that support SSVC project development + workflows) MUST live under `.agents/skills/dev//`. + rationale: > + Separating dev skills from domain skills clarifies audience and + lifecycle: dev skills are used by agents working on the SSVC + codebase itself; domain skills are used by agents helping + practitioners apply SSVC. + testable: true + tags: [tooling] + + - id: SK-01-003 + priority: MUST + statement: > + SK-01-003 SSVC domain skills (those that help practitioners perform + SSVC analysis) MUST live under `.agents/skills/ssvc//`. + rationale: > + A dedicated `ssvc/` subtree makes domain skills discoverable and + distinguishable from development-workflow skills. + testable: true + tags: [tooling] + + - id: SK-01-004 + priority: MUST + statement: > + SK-01-004 Each skill MUST be contained in its own subdirectory + named using kebab-case (e.g., `evaluate-decision-point/`). + rationale: > + One skill per directory allows each skill to carry supplemental + assets (templates, examples, sub-scripts) without namespace + collision. + testable: true + tags: [tooling] + + - id: SK-02 + title: SKILL.md Interface + description: > + Requirements for the canonical skill descriptor file that every skill + directory must contain. + specs: + - id: SK-02-001 + priority: MUST + statement: > + SK-02-001 Every skill directory MUST contain a file named + `SKILL.md`. + rationale: > + A single well-known filename allows tooling, CI, and agents to + locate skill descriptors without directory scanning heuristics. + testable: true + tags: [tooling] + + - id: SK-02-002 + priority: MUST + statement: > + SK-02-002 Every `SKILL.md` MUST open with a YAML front-matter + block (delimited by `---`) containing at minimum the fields + `name` and `description`. + rationale: > + Machine-parseable front-matter enables CI validation, agent + discovery, and future registry tooling without parsing freeform + Markdown prose. + testable: true + tags: [tooling, ci-cd] + + - id: SK-02-003 + priority: MUST + statement: > + SK-02-003 The `name` field MUST be a non-empty string that + uniquely identifies the skill within the repository. + rationale: > + A unique name lets agents reference skills unambiguously and + prevents accidental duplication. + testable: true + tags: [tooling] + + - id: SK-02-004 + priority: MUST + statement: > + SK-02-004 The `description` field MUST be a non-empty string + of one or more sentences summarising what the skill does and when + to invoke it. + rationale: > + A clear description is the primary signal an agent uses when + deciding whether to invoke a skill; an absent or empty description + renders the skill undiscoverable. + testable: true + tags: [tooling] + + - id: SK-02-005 + priority: SHOULD + statement: > + SK-02-005 A `SKILL.md` SHOULD include optional front-matter + fields (`id`, `version`, `tags`, `runtime`, `capabilities`, + `prerequisites`, `env`, `usage_examples`) when those details + are known and stable. + rationale: > + Richer metadata improves agent selection accuracy and reduces + ambiguity for complex or narrowly-scoped skills, but must not be + mandated when the skill is in early or prototype state. + testable: false + lint_suppress: [testable_without_steps] + tags: [tooling] + + - id: SK-02-006 + priority: MUST + statement: > + SK-02-006 `SKILL.md` MUST contain a `## Purpose` section and at + least one procedural section (e.g., `## Procedure` or + `## Workflow`) after the front-matter block. + rationale: > + Prose procedure is the primary content an agent executes when + running the skill; a skill that contains only front-matter + provides no actionable guidance. + testable: true + tags: [tooling] + + - id: SK-03 + title: CI Enforcement + description: > + Requirements for automated validation of skill files in continuous + integration. + specs: + - id: SK-03-001 + priority: MUST + statement: > + SK-03-001 CI MUST validate every `.agents/skills/**/SKILL.md` + file for the presence of valid YAML front-matter and the required + `name` and `description` fields. + rationale: > + Automated enforcement prevents malformed or incomplete skill files + from being merged, keeping the skill surface reliable for agents. + testable: true + tags: [ci-cd, tooling] + + - id: SK-03-002 + priority: MUST + statement: > + SK-03-002 The skill-validation CI job MUST run on pull requests + and on pushes to the main branch, filtered to paths matching + `.agents/skills/**/SKILL.md`. + rationale: > + Path filtering avoids unnecessary CI runs on unrelated changes + while ensuring every SKILL.md change is validated before merge. + testable: true + tags: [ci-cd] + + - id: SK-03-003 + priority: MUST + statement: > + SK-03-003 The skill-validation CI job MUST exit with a non-zero + status code if any SKILL.md file fails validation, causing the + GitHub Actions check to fail. + rationale: > + A failing check blocks the PR merge, ensuring non-conforming + skill files cannot reach the main branch undetected. + testable: true + tags: [ci-cd] diff --git a/specs/spec-registry.yaml b/specs/spec-registry.yaml new file mode 100644 index 00000000..31067651 --- /dev/null +++ b/specs/spec-registry.yaml @@ -0,0 +1,139 @@ +id: SR +title: Spec Registry +description: > + Requirements for the structured YAML specification registry system + (ssvc.metadata.specs). The registry is the authoritative source for + normative requirements in SSVC. It replaces ad-hoc markdown requirement + lists with structured, validated, machine-readable YAML files. +version: "0.1.0" +kind: general +scope: [production] + +groups: + - id: SR-01 + title: Schema Requirements + description: Requirements for the spec file format and Pydantic validation. + specs: + - id: SR-01-001 + priority: MUST + statement: > + SR-01-001 Each spec file MUST be a valid YAML document that + conforms to the SpecFile schema (id, title, description, version, + kind, scope, groups). + rationale: > + Structural validation at load time catches authoring errors early + and ensures all downstream tooling (linter, renderer, LLM exporter) + can rely on the schema contract. + testable: true + tags: [tooling, documentation] + + - id: SR-01-002 + priority: MUST + statement: > + SR-01-002 Each spec file MUST contain at least one group. + rationale: > + A file with no groups contributes no requirements and adds noise + to the registry; it is rejected at load time. + testable: true + tags: [tooling] + + - id: SR-01-003 + priority: MUST + statement: > + SR-01-003 Each spec group MUST contain at least one spec. + rationale: > + An empty group contributes no requirements and adds noise to the + registry; it is rejected at load time. + testable: true + tags: [tooling] + + - id: SR-01-004 + priority: MUST + statement: > + SR-01-004 All spec IDs MUST be unique across all spec files loaded + into a single registry instance. + rationale: > + Duplicate IDs make cross-references ambiguous and break tooling + that relies on ID-based lookup. + testable: true + tags: [tooling] + + - id: SR-01-005 + priority: MUST + statement: > + SR-01-005 Each group ID MUST share the same alphabetic prefix as + its containing file ID (e.g. group SR-01 must live in file SR). + rationale: > + Consistent prefixes allow tracing any requirement back to its + topic file without consulting the registry. + testable: true + tags: [tooling] + + - id: SR-01-006 + priority: MUST + statement: > + SR-01-006 Each spec ID MUST share the same prefix as its containing + group ID (e.g. spec SR-01-001 must live in group SR-01). + rationale: > + Consistent prefixes enforce a three-level hierarchy (file, group, + spec) and prevent specs from being placed in the wrong group. + testable: true + tags: [tooling] + + - id: SR-02 + title: Linting Requirements + description: Requirements for the spec registry linter (ssvc-spec-lint). + specs: + - id: SR-02-001 + priority: MUST + statement: > + SR-02-001 The linter MUST exit with code 1 when hard errors are + found (duplicate IDs, dangling cross-references, prefix + mismatches). + rationale: > + Non-zero exit codes allow CI gates to block merges on malformed + spec files. + testable: true + tags: [tooling, ci-cd] + + - id: SR-02-002 + priority: SHOULD + statement: > + SR-02-002 The linter SHOULD emit advisory warnings for specs that + have no tags, rationale exceeding 500 characters, or testable=false + with no behavioral steps. + rationale: > + Warnings surface common authoring oversights without blocking CI, + keeping signal-to-noise ratio high. + testable: true + tags: [tooling] + + - id: SR-02-003 + priority: MAY + statement: > + SR-02-003 Individual specs MAY suppress specific advisory warnings + via the lint_suppress field when the warning condition is + intentional and documented. + rationale: > + Some specs are legitimately untestable via automated tests (e.g. + process requirements verified by human review). Suppression with + explicit acknowledgement is preferable to reducing warning + sensitivity globally. + testable: true + tags: [tooling] + + - id: SR-03 + title: Pre-commit and CI Integration + description: Requirements for automation hooks. + specs: + - id: SR-03-001 + priority: SHOULD + statement: > + SR-03-001 Projects SHOULD run ssvc-spec-lint as a pre-commit hook + triggered by any change to files under specs/. + rationale: > + Pre-commit enforcement catches spec errors before they reach CI, + shortening the feedback loop for authors. + testable: false + lint_suppress: [testable_without_steps] + tags: [tooling, ci-cd] diff --git a/specs/testing.yaml b/specs/testing.yaml new file mode 100644 index 00000000..7154a97a --- /dev/null +++ b/specs/testing.yaml @@ -0,0 +1,116 @@ +id: TS +title: Testing Standards +description: > + Requirements for test organisation, isolation, and quality within the + SSVC Python package. Tests use pytest with a unittest.TestCase hybrid + style and a manually-injected registry for isolation. +version: "0.1.0" +kind: language +scope: [production] + +groups: + - id: TS-01 + title: Test Organisation + description: Requirements for how test files and directories are structured. + specs: + - id: TS-01-001 + priority: MUST + statement: > + TS-01-001 All tests MUST reside in src/test/ with a directory + layout that mirrors src/ssvc/. + rationale: > + Mirroring the source layout makes it trivial to locate the test + for any module and ensures pytest autodiscovery covers every test. + testable: true + tags: [testing] + + - id: TS-01-002 + priority: MUST + statement: > + TS-01-002 Test files MUST be named test_.py. + rationale: > + Consistent naming enables pytest autodiscovery without extra + configuration and makes the correspondence between source and test + explicit. + testable: true + tags: [testing] + + - id: TS-01-003 + priority: MUST + statement: > + TS-01-003 Test classes MUST subclass unittest.TestCase and be + runnable under pytest without additional plugins. + rationale: > + The existing test suite uses unittest.TestCase; new tests MUST + follow the same style to keep the suite homogeneous until a + deliberate migration to native pytest fixtures is undertaken. + testable: true + tags: [testing] + + - id: TS-02 + title: Test Isolation + description: > + Requirements for preventing shared state from leaking between tests. + specs: + - id: TS-02-001 + priority: MUST + statement: > + TS-02-001 Each test class setUp MUST call registry.reset(force=True) + before populating test fixtures. + rationale: > + The global SsvcObjectRegistry is mutated by module-level + instantiation; resetting it in setUp prevents fixtures from one + test contaminating another. + testable: true + tags: [testing] + + - id: TS-02-002 + priority: MUST + statement: > + TS-02-002 API router tests MUST inject an isolated test registry + by directly replacing the router module's r variable + (e.g. router.r = self.r). + rationale: > + Direct attribute replacement avoids patching at the module level + and keeps the test registry fully under the test's control. + testable: true + tags: [testing] + + - id: TS-02-003 + priority: MUST + statement: > + TS-02-003 Tests MUST NOT assume a particular module import order + or rely on global registry state left by prior tests. + rationale: > + Import-order assumptions produce fragile tests that pass in one + configuration and fail in another, making CI results unreliable. + testable: true + tags: [testing] + + - id: TS-03 + title: Test Quality + description: > + Requirements for breadth and depth of test coverage. + specs: + - id: TS-03-001 + priority: SHOULD + statement: > + TS-03-001 Each new Python module added to src/ssvc/ SHOULD have a + corresponding test file in src/test/. + rationale: > + Paired test files prevent coverage blind spots from accumulating + as the codebase grows. + testable: false + lint_suppress: [testable_without_steps] + tags: [testing] + + - id: TS-03-002 + priority: SHOULD + statement: > + TS-03-002 Decision point and decision table modules SHOULD include + tests that verify JSON schema round-trip serialisation. + rationale: > + Round-trip tests catch regressions where a model change breaks the + generated JSON schema without triggering other test failures. + testable: true + tags: [testing] diff --git a/specs/versioning.yaml b/specs/versioning.yaml new file mode 100644 index 00000000..29577e23 --- /dev/null +++ b/specs/versioning.yaml @@ -0,0 +1,383 @@ +id: VR +title: Versioning Rules +description: > + Requirements for how SSVC objects, schemas, and the overall project are + versioned. SSVC uses a layered versioning strategy: individual domain + objects (decision points, decision point groups, decision tables) use + Semantic Versioning (SemVer 2.0.0); JSON schema files use SchemaVer; and + the overall SSVC project uses Calendar Versioning (CalVer). These rules + implement ADR-0002, ADR-0005, ADR-0006, ADR-0013, and ADR-0015. +version: "0.1.0" +kind: domain +scope: [production] + +groups: + - id: VR-01 + title: Decision Point SemVer Rules + description: > + Rules governing how Semantic Version numbers are incremented for + individual DecisionPoint objects. Implements ADR-0006. + specs: + - id: VR-01-001 + priority: MUST + statement: > + VR-01-001 A new DecisionPoint object (with a new name and key) + MUST be created when a different or fundamentally new concept is + being represented, even if it superficially resembles an existing + decision point. + rationale: > + Creating a new object for conceptually distinct decision points + preserves the integrity of prior versions and prevents misleading + backward-compatibility claims. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-01-002 + priority: MUST + statement: > + VR-01-002 The major version of a DecisionPoint MUST be incremented + when existing values are removed, value semantics change such that + older answers are no longer usable, or new values are added that + divide previous value semantics ambiguously. + rationale: > + A major bump signals to consumers that existing answers recorded + against the prior version may not map correctly to the new version, + requiring manual review or re-evaluation. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-01-003 + priority: MUST + statement: > + VR-01-003 The minor version of a DecisionPoint MUST be incremented + (and major held constant) when new options are added while existing + value semantics are preserved, value names or keys are changed, or + the decision point name is changed. + rationale: > + Minor increments signal backward compatibility: existing answers + remain valid but the option space has been extended or renamed. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-01-004 + priority: MUST + statement: > + VR-01-004 The patch version of a DecisionPoint MUST be incremented + (and major/minor held constant) for typo fixes in option or + decision point names, or description changes that do not affect + semantics. + rationale: > + Patch increments communicate that no change in meaning occurred, + so consumers need not re-evaluate prior answers. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-01-005 + priority: MUST + statement: > + VR-01-005 A DecisionPoint whose major version is 0 (i.e., v0.x) + MUST be treated as pre-support: its key, labels, label count, + label ordering, descriptions, and semantics are all subject to + change without a major-version increment. + rationale: > + The v0.x convention provides an explicit pre-stability phase + for new decision points before they enter production use and + acquire backward-compatibility obligations. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-01-006 + priority: MAY + statement: > + VR-01-006 DecisionPoint objects SHOULD carry a status field + indicating lifecycle stage (e.g. active, deprecated) so that + consumers can distinguish supported decision points from those + that should no longer be used. + rationale: > + ADR-0006 identifies the need for lifecycle status tracking but + defers it to a future decision. This requirement captures the + intent until a dedicated ADR formalises the status vocabulary. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-02 + title: Decision Point Group SemVer Rules + description: > + Rules governing how Semantic Version numbers are incremented for + DecisionPointGroup objects. Implements ADR-0004 and ADR-0005. + DecisionTable versioning is covered separately in VR-05. + specs: + - id: VR-02-001 + priority: MUST + statement: > + VR-02-001 A new DecisionPointGroup object (with a new name) MUST be + created when the stakeholder role and/or the decision being modeled + changes, even if the constituent decision points remain the same. + rationale: > + Stakeholder role and modeled decision are the core identity of a + group. A change in either represents a fork in version history that + cannot be expressed as a version increment. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-02-002 + priority: MUST + statement: > + VR-02-002 The major version of a DecisionPointGroup MUST be + incremented when a decision point is added to or removed from the + group, or when any constituent decision point increments its own + major version. + rationale: > + Adding or removing a decision point changes the input space of the + table; a major constituent change breaks any downstream policies + that assumed the prior input structure. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-02-003 + priority: MUST + statement: > + VR-02-003 The minor version of a DecisionPointGroup MUST be + incremented (and major held constant) when any constituent decision + point increments its own minor version. + rationale: > + A constituent minor increment adds options without breaking existing + answers; a group minor increment mirrors that compatible expansion. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-02-004 + priority: MUST + statement: > + VR-02-004 The patch version of a DecisionPointGroup MUST be + incremented (and major/minor held constant) when any constituent + decision point increments its own patch version, or when the + group's name or description changes. + rationale: > + Patch-level changes carry no semantic shift; a group patch mirrors + the same signal for the group as a whole. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-03 + title: JSON Schema SchemaVer Rules + description: > + Rules governing how SSVC JSON schema files are versioned using + SchemaVer (MODEL-REVISION-ADDITION). Implements ADR-0015. + specs: + - id: VR-03-001 + priority: MUST + statement: > + VR-03-001 All SSVC JSON schema files MUST use SchemaVer + (MODEL-REVISION-ADDITION format, e.g. 1-0-0) rather than SemVer + for their schema version identifier. + rationale: > + SchemaVer was designed specifically for JSON schema evolution and + maps directly onto data-compatibility questions, unlike SemVer + which was designed for software API compatibility. + testable: true + tags: [tooling] + + - id: VR-03-002 + priority: MUST + statement: > + VR-03-002 The MODEL component of a schema's SchemaVer MUST be + incremented when a required field is removed, a field type or + allowed values change in a way that invalidates previously valid + data, or the root schema structure changes. + rationale: > + MODEL increments warn consumers that existing stored data will + no longer validate against the new schema. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-03-003 + priority: MUST + statement: > + VR-03-003 The REVISION component of a schema's SchemaVer MUST be + incremented when a previously optional field becomes required, an + enum gains values that affect round-trip processing for some + consumers, or constraints are tightened such that some existing + data would fail validation. + rationale: > + REVISION increments warn that some (but not necessarily all) + existing data may fail validation. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-03-004 + priority: MUST + statement: > + VR-03-004 The ADDITION component of a schema's SchemaVer MUST be + incremented when a new optional field is added, a constraint is + relaxed, documentation fields are updated, or an enum gains new + allowed values that only expand what is valid. + rationale: > + ADDITION increments signal that all existing data continues to + validate; consumers need no immediate migration action. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-04 + title: Project CalVer Rules + description: > + Rules governing how the overall SSVC project and documentation + releases are versioned using Calendar Versioning (CalVer). + Implements ADR-0013. + specs: + - id: VR-04-001 + priority: MUST + statement: > + VR-04-001 SSVC project releases MUST use CalVer in the format + YYYY.M.patch (e.g. 2025.6.0), where YYYY is the four-digit year, + M is the single-digit month (no leading zero), and patch is + incremented for subsequent updates in the same month. + rationale: > + CalVer embeds recency directly in the version string, which is + more informative than SemVer for a living framework whose docs + update frequently and independently of object-level changes. + testable: true + tags: [ci-cd] + + - id: VR-04-002 + priority: MUST + statement: > + VR-04-002 Individual SSVC domain objects (decision points, outcome + sets, decision tables) MUST continue to use SemVer independent + of the project-level CalVer version. + rationale: > + Object-level SemVer communicates compatibility semantics that + CalVer cannot; the two schemes are complementary and operate at + different granularities. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-05 + title: Decision Table SemVer Rules + description: > + Rules governing how Semantic Version numbers are incremented for + DecisionTable objects. A DecisionTable is a composite domain object + whose version reflects changes to its constituent decision points + (inputs and the designated outcome decision point). Implements + ADR-0004 and ADR-0005. Outcome sets are themselves decision points + and are versioned under VR-01. + specs: + - id: VR-05-001 + priority: MUST + statement: > + VR-05-001 A new DecisionTable object (with a new name and key) MUST + be created when the stakeholder role and/or the decision being + modeled changes, even if the constituent decision points remain the + same. + rationale: > + Stakeholder role and modeled decision are the core identity of a + table. A change in either represents a fork in version history that + cannot be expressed as a version increment. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-05-002 + priority: MUST + statement: > + VR-05-002 A new DecisionTable object (with a new name and key) MUST + be created when the designated outcome decision point is replaced + with a conceptually different decision point. + rationale: > + The outcome decision point defines what question the table answers. + Replacing it with a conceptually different output changes the purpose + of the table in a way that cannot be expressed as a version + increment; a version bump would misrepresent backward compatibility. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-05-003 + priority: MUST + statement: > + VR-05-003 The major version of a DecisionTable MUST be incremented + when an input decision point is added to or removed from the table, + or when any constituent decision point (input or outcome) increments + its own major version. + rationale: > + Adding or removing an input changes the combinatoric space of the + table; a major constituent change breaks any downstream policies + that assumed the prior input structure. Both events require consumers + to reassess previously recorded answers. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-05-004 + priority: MUST + statement: > + VR-05-004 The minor version of a DecisionTable MUST be incremented + (and major held constant) when any constituent decision point (input + or outcome) increments its own minor version. + rationale: > + A constituent minor increment adds options without breaking existing + answers; a table minor increment mirrors that compatible expansion + and signals that prior answers remain valid. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-05-005 + priority: MUST + statement: > + VR-05-005 The patch version of a DecisionTable MUST be incremented + (and major/minor held constant) when any constituent decision point + (input or outcome) increments its own patch version, or when the + table's name or description changes without altering its semantics. + rationale: > + Patch increments carry no semantic shift; a table patch mirrors the + same signal for the table as a whole, and name/description-only + changes equally carry no semantic impact. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-05-006 + priority: MUST + statement: > + VR-05-006 A DecisionTable whose major version is 0 (i.e., v0.x) + MUST be treated as pre-support: its constituent decision points, + designated outcome, and any row-level mappings are all subject to + change without a major-version increment. + rationale: > + The v0.x convention provides an explicit pre-stability phase for new + decision tables before they enter production use and acquire + backward-compatibility obligations. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] + + - id: VR-05-007 + priority: MAY + statement: > + VR-05-007 DecisionTable objects MAY carry a status field indicating + lifecycle stage (e.g. active, deprecated) so that consumers can + distinguish supported tables from those that should no longer be + used. + rationale: > + Lifecycle status tracking for DecisionTable is a future concern; no + current implementation provides this field. This requirement + captures the intent as a forward-looking allowance without imposing + an obligation. + testable: false + lint_suppress: [testable_without_steps] + tags: [documentation] diff --git a/src/ssvc/metadata/__init__.py b/src/ssvc/metadata/__init__.py new file mode 100644 index 00000000..b35a8559 --- /dev/null +++ b/src/ssvc/metadata/__init__.py @@ -0,0 +1,21 @@ + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +"""SSVC metadata tooling layer.""" diff --git a/src/ssvc/metadata/base.py b/src/ssvc/metadata/base.py new file mode 100644 index 00000000..528ce2b2 --- /dev/null +++ b/src/ssvc/metadata/base.py @@ -0,0 +1,27 @@ +"""Shared type aliases for the ssvc.metadata tooling layer.""" + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +from typing import Annotated + +from pydantic import StringConstraints + +NonEmptyStr = Annotated[str, StringConstraints(min_length=1)] +NonEmptyStrList = list[NonEmptyStr] diff --git a/src/ssvc/metadata/specs/__init__.py b/src/ssvc/metadata/specs/__init__.py new file mode 100644 index 00000000..8c98cda8 --- /dev/null +++ b/src/ssvc/metadata/specs/__init__.py @@ -0,0 +1,58 @@ +"""Spec registry for ``specs/*.yaml`` structured requirement files.""" + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +import warnings + +from ssvc.metadata.specs.registry import SpecRegistry, load_registry + + +class UnknownSpecIdWarning(UserWarning): + """Warning emitted when a test references a spec ID not in the registry. + + Emitted (non-blocking) by ``pytest_collection_modifyitems`` when a + ``@pytest.mark.spec`` marker references an ID that cannot be found in the + loaded :class:`SpecRegistry`. + """ + + +def warn_unknown_spec_id(spec_id: str, registry: SpecRegistry) -> None: + """Emit :class:`UnknownSpecIdWarning` if ``spec_id`` is not in registry. + + Args: + spec_id: The spec ID string to validate. + registry: The loaded :class:`SpecRegistry` to check against. + """ + try: + registry.get(spec_id) + except KeyError: + warnings.warn( + f"Unknown spec ID referenced in test marker: {spec_id!r}", + UnknownSpecIdWarning, + stacklevel=2, + ) + + +__all__ = [ + "SpecRegistry", + "UnknownSpecIdWarning", + "load_registry", + "warn_unknown_spec_id", +] diff --git a/src/ssvc/metadata/specs/lint.py b/src/ssvc/metadata/specs/lint.py new file mode 100644 index 00000000..ccb42775 --- /dev/null +++ b/src/ssvc/metadata/specs/lint.py @@ -0,0 +1,155 @@ +"""Spec registry linter. + +Usage:: + + ssvc-spec-lint specs/ + python -m ssvc.metadata.specs.lint specs/ +""" + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +from __future__ import annotations + +import sys +from pathlib import Path + +from pydantic import ValidationError + +from ssvc.metadata.specs.registry import SpecRegistry, load_registry +from ssvc.metadata.specs.schema import BehavioralSpec, LintWarningCode + +_RATIONALE_WARN_CHARS = 500 + + +def _check_prefix_consistency(registry: SpecRegistry) -> list[str]: + """Verify each group ID prefix matches its containing file prefix.""" + errors: list[str] = [] + for spec_file in registry.files: + file_prefix = spec_file.id + for group in spec_file.groups: + group_prefix = group.id.split("-")[0] + if group_prefix != file_prefix: + errors.append( + f"Group '{group.id}' prefix '{group_prefix}' does not " + f"match file prefix '{file_prefix}'" + ) + return errors + + +def _check_spec_id_prefix_consistency(registry: SpecRegistry) -> list[str]: + """Verify each spec ID prefix matches the group it lives in. + + A spec with ID ``HP-07-002`` MUST reside in group ``HP-07``. + """ + errors: list[str] = [] + for spec_file in registry.files: + for group in spec_file.groups: + expected_prefix = group.id + "-" + for spec in group.specs: + if not spec.id.startswith(expected_prefix): + errors.append( + f"Spec '{spec.id}' does not belong in group " + f"'{group.id}' (expected prefix '{expected_prefix}')" + ) + return errors + + +def lint(spec_dir: Path) -> int: + """Validate the spec registry in ``spec_dir``. + + Hard errors cause exit code 1. Advisory warnings are printed but do not + affect the exit code. + + Args: + spec_dir: Directory containing ``*.yaml`` spec files. + + Returns: + ``0`` if no hard errors, ``1`` if any hard errors found. + """ + hard_errors: list[str] = [] + warnings: list[str] = [] + + try: + registry = load_registry(spec_dir) + except (ValidationError, ValueError) as exc: + print(f"[FATAL] Registry load failed:\n{exc}", file=sys.stderr) + return 1 + + hard_errors.extend(registry.validate_cross_references()) + hard_errors.extend(_check_prefix_consistency(registry)) + hard_errors.extend(_check_spec_id_prefix_consistency(registry)) + + for spec_id, spec in registry.all_specs.items(): + suppressed = set(spec.lint_suppress or []) + + is_behavioral = isinstance(spec, BehavioralSpec) and bool(spec.steps) + + if ( + not spec.testable + and not is_behavioral + and LintWarningCode.TESTABLE_WITHOUT_STEPS not in suppressed + ): + warnings.append( + f"[WARN] {spec_id}: testable=false but no behavioral steps " + f"(suppress with lint_suppress: [testable_without_steps])" + ) + + if ( + spec.rationale + and len(spec.rationale) > _RATIONALE_WARN_CHARS + and LintWarningCode.RATIONALE_TOO_LONG not in suppressed + ): + warnings.append( + f"[WARN] {spec_id}: rationale exceeds " + f"{_RATIONALE_WARN_CHARS} characters" + ) + + tags = spec.tags or [] + if not tags and LintWarningCode.MISSING_TAGS not in suppressed: + warnings.append(f"[WARN] {spec_id}: no tags defined") + + for w in warnings: + print(w) + for e in hard_errors: + print(f"[ERROR] {e}", file=sys.stderr) + + return 0 if not hard_errors else 1 + + +def main() -> None: + """CLI entry point: ``ssvc-spec-lint`` or + ``python -m ssvc.metadata.specs.lint [spec_dir]``. + + ``spec_dir`` defaults to ``specs/`` relative to the current working + directory so that ``ssvc-spec-lint`` from the repository root behaves + identically to the pre-commit hook. + """ + spec_dir = Path(sys.argv[1]) if len(sys.argv) >= 2 else Path("specs") + if not spec_dir.is_dir(): + print( + f"[FATAL] spec_dir '{spec_dir}' not found or not a directory", + file=sys.stderr, + ) + sys.exit(2) + sys.exit(lint(spec_dir)) + + +if __name__ == "__main__": + main() diff --git a/src/ssvc/metadata/specs/llm_export.py b/src/ssvc/metadata/specs/llm_export.py new file mode 100644 index 00000000..f9eb1e9b --- /dev/null +++ b/src/ssvc/metadata/specs/llm_export.py @@ -0,0 +1,263 @@ +"""LLM-optimized export for the spec registry. + +Produces a flat, inheritance-resolved JSON projection designed for +coding agent consumption. Requirements become primary objects with +denormalized group/file provenance and resolved kind/scope/tags. + +Usage:: + + from ssvc.metadata.specs.registry import load_registry + from ssvc.metadata.specs.llm_export import to_llm_json + + registry = load_registry() + # All specs + print(to_llm_json(registry)) + # Single topic + print(to_llm_json(registry, topic="SR")) + # Specific IDs with transitive dependencies + print(to_llm_json(registry, spec_ids=["SR-01-001"], include_deps=True)) +""" + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +from __future__ import annotations + +import json +from typing import Any + +from ssvc.metadata.specs.registry import ( + SpecRegistry, + effective_kind, + effective_scope, + effective_tags, +) +from ssvc.metadata.specs.schema import ( + BehavioralSpec, + Spec, + SpecFile, + SpecGroup, +) + + +def _spec_record( + spec: Spec, + group: SpecGroup, + file: SpecFile, +) -> dict[str, Any]: + """Build a flat, inheritance-resolved dict for a single spec.""" + rec: dict[str, Any] = { + "id": spec.id, + "topic": file.id, + "group": group.id, + "group_title": group.title, + "type": ( + "behavioral" if isinstance(spec, BehavioralSpec) else "statement" + ), + "priority": spec.priority.value, + "statement": spec.statement, + "kind": effective_kind(spec, group, file).value, + "scope": [s.value for s in effective_scope(spec, group, file)], + } + + tags = effective_tags(spec) + if tags: + rec["tags"] = [t.value for t in tags] + + if spec.rationale is not None: + rec["rationale"] = spec.rationale + + if not spec.testable: + rec["testable"] = False + + if spec.relationships: + rec["relationships"] = [_rel_record(r) for r in spec.relationships] + + if isinstance(spec, BehavioralSpec): + if spec.preconditions: + rec["preconditions"] = [p.description for p in spec.preconditions] + if spec.steps: + rec["steps"] = [_step_record(s) for s in spec.steps] + if spec.postconditions: + rec["postconditions"] = [ + p.description for p in spec.postconditions + ] + + return rec + + +def _rel_record(r: object) -> dict[str, str]: + d: dict[str, str] = { + "rel_type": r.rel_type.value, # type: ignore[attr-defined] + "spec_id": r.spec_id, # type: ignore[attr-defined] + } + if r.note is not None: # type: ignore[attr-defined] + d["note"] = r.note # type: ignore[attr-defined] + return d + + +def _step_record(s: object) -> dict[str, Any]: + d: dict[str, Any] = { + "order": s.order, # type: ignore[attr-defined] + "actor": s.actor, # type: ignore[attr-defined] + "action": s.action, # type: ignore[attr-defined] + } + if s.expected is not None: # type: ignore[attr-defined] + d["expected"] = s.expected # type: ignore[attr-defined] + return d + + +def _topic_record(file: SpecFile) -> dict[str, str]: + return { + "id": file.id, + "title": file.title, + "version": file.version, + "kind": file.kind.value, + } + + +def _selected_spec_ids( + registry: SpecRegistry, + spec_ids: list[str] | None, + *, + include_deps: bool, +) -> set[str] | None: + if spec_ids is None: + return None + + selected_ids = set(spec_ids) + if include_deps: + for spec_id in list(selected_ids): + selected_ids |= registry.transitive_deps(spec_id) + return selected_ids + + +def _matches_filters( + spec: Spec, + group: SpecGroup, + file: SpecFile, + *, + topic: str | None, + selected_ids: set[str] | None, + spec_id: str, + kind: str | None, + scope: str | None, + tags: list[str] | None, + priority: str | None, +) -> bool: + if topic is not None and file.id != topic: + return False + if selected_ids is not None and spec_id not in selected_ids: + return False + + eff_kind = effective_kind(spec, group, file) + eff_scope_values = { + item.value for item in effective_scope(spec, group, file) + } + eff_tag_values = {item.value for item in effective_tags(spec)} + + if kind and eff_kind.value != kind: + return False + if scope and scope not in eff_scope_values: + return False + if tags and not set(tags).issubset(eff_tag_values): + return False + if priority and spec.priority.value != priority: + return False + return True + + +def _edge_record(spec_id: str, relationship: object) -> dict[str, str]: + edge: dict[str, str] = { + "from": spec_id, + "rel_type": relationship.rel_type.value, # type: ignore[attr-defined] + "to": relationship.spec_id, # type: ignore[attr-defined] + } + if relationship.note: # type: ignore[attr-defined] + edge["note"] = relationship.note # type: ignore[attr-defined] + return edge + + +def to_llm_json( + registry: SpecRegistry, + *, + topic: str | None = None, + spec_ids: list[str] | None = None, + include_deps: bool = False, + kind: str | None = None, + scope: str | None = None, + tags: list[str] | None = None, + priority: str | None = None, +) -> str: + """Produce a flat, inheritance-resolved JSON projection of the registry. + + Args: + registry: The loaded SpecRegistry. + topic: Filter to specs from the file with this ID prefix. + spec_ids: Filter to these specific spec IDs. + include_deps: When True and *spec_ids* is given, expand to include + transitive dependencies via the requirements graph. + kind: Filter to specs with this effective kind value. + scope: Filter to specs whose effective scope contains this value. + tags: Filter to specs that have ALL of the given tags. + priority: Filter to specs with this priority value. + + Returns: + Compact JSON string (no indentation). + """ + selected_ids = _selected_spec_ids( + registry, spec_ids, include_deps=include_deps + ) + requirements: list[dict[str, Any]] = [] + edges: list[dict[str, str]] = [] + topic_ids_seen: set[str] = set() + + for spec_id, spec in registry.all_specs.items(): + group, file = registry._spec_context[spec_id] + if not _matches_filters( + spec, + group, + file, + topic=topic, + selected_ids=selected_ids, + spec_id=spec_id, + kind=kind, + scope=scope, + tags=tags, + priority=priority, + ): + continue + + requirements.append(_spec_record(spec, group, file)) + topic_ids_seen.add(file.id) + edges.extend( + _edge_record(spec_id, relationship) + for relationship in spec.relationships or [] + ) + + result: dict[str, Any] = { + "topics": [ + _topic_record(file) + for file in registry.files + if file.id in topic_ids_seen + ], + "requirements": requirements, + "edges": edges, + } + return json.dumps(result, separators=(",", ":")) diff --git a/src/ssvc/metadata/specs/registry.py b/src/ssvc/metadata/specs/registry.py new file mode 100644 index 00000000..f4644841 --- /dev/null +++ b/src/ssvc/metadata/specs/registry.py @@ -0,0 +1,248 @@ +"""SpecRegistry and loader for ``specs/*.yaml`` files. + +TODO: Consider extracting this module into a standalone shared library +(e.g. ``certcc-spec-registry``) so that both SSVC and Vultron can depend +on it rather than maintaining parallel copies. +""" + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +from __future__ import annotations + +from pathlib import Path + +import networkx as nx +from pydantic import BaseModel, PrivateAttr + +from ssvc.metadata.specs.schema import ( + BehavioralSpec, + Scope, + Spec, + SpecFile, + SpecGroup, + SpecIdStr, + SpecKind, + SpecTag, +) + +try: + import yaml +except ImportError as exc: # pragma: no cover + raise ImportError( + "pyyaml is required for the spec registry loader. " + "Install it with: pip install pyyaml" + ) from exc + + +def effective_kind(spec: Spec, group: SpecGroup, file: SpecFile) -> SpecKind: + """Resolve the effective ``kind`` for *spec* via inheritance.""" + if spec.kind is not None: + return spec.kind + if group.kind is not None: + return group.kind + return file.kind + + +def effective_scope( + spec: Spec, group: SpecGroup, file: SpecFile +) -> list[Scope]: + """Resolve the effective ``scope`` for *spec* via inheritance.""" + if spec.scope is not None: + return spec.scope + if group.scope is not None: + return group.scope + return file.scope + + +def effective_tags(spec: Spec) -> list[SpecTag]: + """Return tags for *spec*, defaulting to empty list when absent.""" + return spec.tags if spec.tags is not None else [] + + +class SpecRegistry(BaseModel): + """Registry of all loaded spec files with ID-based lookup.""" + + files: list[SpecFile] + + _index: dict[SpecIdStr, Spec] = PrivateAttr(default_factory=dict) + _group_index: dict[SpecIdStr, SpecGroup] = PrivateAttr( + default_factory=dict + ) + _spec_context: dict[SpecIdStr, tuple[SpecGroup, SpecFile]] = PrivateAttr( + default_factory=dict + ) + _graph: nx.DiGraph = PrivateAttr(default_factory=nx.DiGraph) + + def model_post_init(self, __context: object) -> None: + for file in self.files: + for group in file.groups: + self._register_group(group) + for spec in group.specs: + self._register_spec(spec) + self._spec_context[spec.id] = (group, file) + + self._build_graph() + + def _build_graph(self) -> None: + """Populate ``_graph`` with spec nodes and relationship edges.""" + g = self._graph + for spec_id, spec in self._index.items(): + group, file = self._spec_context[spec_id] + spec_type = ( + "behavioral" + if isinstance(spec, BehavioralSpec) + else "statement" + ) + g.add_node( + spec_id, + priority=spec.priority.value, + kind=effective_kind(spec, group, file).value, + scope=[s.value for s in effective_scope(spec, group, file)], + file_id=file.id, + group_id=group.id, + type=spec_type, + statement=spec.statement, + ) + + for spec_id, spec in self._index.items(): + for rel in spec.relationships or []: + note = rel.note if rel.note else None + g.add_edge( + spec_id, + rel.spec_id, + rel_type=rel.rel_type.value, + note=note, + ) + + @property + def graph(self) -> nx.DiGraph: + """The requirements graph (specs as nodes, relationships as edges).""" + return self._graph + + def subgraph_for_topic(self, file_id: str) -> nx.DiGraph: + """Return the subgraph containing only specs from file *file_id*.""" + nodes = [ + n + for n, d in self._graph.nodes(data=True) + if d.get("file_id") == file_id + ] + return self._graph.subgraph(nodes).copy() + + def transitive_deps(self, spec_id: SpecIdStr) -> set[str]: + """Return all spec IDs reachable from *spec_id* via outgoing edges.""" + if spec_id not in self._graph: + return set() + return set(nx.descendants(self._graph, spec_id)) + + def _register_spec(self, spec: Spec) -> None: + if spec.id in self._index: + raise ValueError(f"Duplicate spec ID: {spec.id}") + self._index[spec.id] = spec + + def _register_group(self, group: SpecGroup) -> None: + if group.id in self._group_index: + raise ValueError(f"Duplicate group ID: {group.id}") + self._group_index[group.id] = group + + def get(self, spec_id: SpecIdStr) -> Spec: + """Return the spec for the given ID. + + Raises: + KeyError: If the spec ID is not found in the registry. + """ + if spec_id not in self._index: + raise KeyError(f"Unknown spec ID: {spec_id}") + return self._index[spec_id] + + def get_effective_kind(self, spec_id: SpecIdStr) -> SpecKind: + """Return the resolved ``kind`` for *spec_id* via inheritance.""" + spec = self.get(spec_id) + group, file = self._spec_context[spec_id] + return effective_kind(spec, group, file) + + def get_effective_scope(self, spec_id: SpecIdStr) -> list[Scope]: + """Return the resolved ``scope`` for *spec_id* via inheritance.""" + spec = self.get(spec_id) + group, file = self._spec_context[spec_id] + return effective_scope(spec, group, file) + + def validate_cross_references(self) -> list[str]: + """Return error strings for any dangling relationship targets.""" + errors = [] + for spec_id, spec in self._index.items(): + rels = spec.relationships or [] + for rel in rels: + if rel.spec_id not in self._index: + errors.append( + f"{spec_id}: relationship target " + f"'{rel.spec_id}' not found" + ) + return errors + + @property + def all_specs(self) -> dict[SpecIdStr, Spec]: + """Read-only view of the full spec index.""" + return dict(self._index) + + @property + def all_groups(self) -> dict[SpecIdStr, SpecGroup]: + """Read-only view of the full group index.""" + return dict(self._group_index) + + +def _find_repo_root(start: Path | None = None) -> Path: + """Return the repository root by searching upward for ``pyproject.toml``.""" + origin = start or Path.cwd() + for parent in [origin, *origin.parents]: + if (parent / "pyproject.toml").exists(): + return parent + raise FileNotFoundError( + f"Could not locate repository root (pyproject.toml) " + f"starting from {origin}" + ) + + +def load_registry( + spec_dir: Path | None = None, +) -> SpecRegistry: + """Discover and validate all ``*.yaml`` files in ``spec_dir``. + + Args: + spec_dir: Directory containing ``*.yaml`` spec files. When ``None`` + the repository root is resolved automatically and ``specs/`` is + used. + + Returns: + A fully-indexed :class:`SpecRegistry`. + + Raises: + ValueError: If any spec file fails validation or contains duplicate IDs. + FileNotFoundError: If the repository root cannot be resolved. + """ + if spec_dir is None: + root = _find_repo_root() + spec_dir = root / "specs" + + files = [] + for yaml_path in sorted(spec_dir.glob("*.yaml")): + raw = yaml.safe_load(yaml_path.read_text()) + files.append(SpecFile.model_validate(raw)) + + return SpecRegistry(files=files) diff --git a/src/ssvc/metadata/specs/render.py b/src/ssvc/metadata/specs/render.py new file mode 100644 index 00000000..1695f2d8 --- /dev/null +++ b/src/ssvc/metadata/specs/render.py @@ -0,0 +1,389 @@ +"""Context generation tool for the spec registry. + +Provides a markdown renderer, a JSON exporter, and a YAML exporter for +agent/human consumption. + +Usage:: + + from pathlib import Path + from ssvc.metadata.specs import load_registry + from ssvc.metadata.specs.render import render_markdown, export_json + + registry = load_registry(Path("specs/")) + md = render_markdown(registry.files[0]) + js = export_json(registry) +""" + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +from __future__ import annotations + +import json +from pathlib import Path + +from ssvc.metadata.specs.registry import ( + SpecRegistry, + effective_tags, + load_registry, +) +from ssvc.metadata.specs.schema import ( + BehavioralSpec, + Spec, + SpecFile, + SpecGroup, +) + +try: + import yaml +except ImportError as exc: # pragma: no cover + raise ImportError( + "pyyaml is required for YAML export. " + "Install it with: pip install pyyaml" + ) from exc + + +def _priority_line(spec: Spec) -> str: + return f"- `{spec.id}` {spec.statement}" + + +def _append_behavioral_markdown( + lines: list[str], spec: BehavioralSpec +) -> None: + for precondition in spec.preconditions or []: + lines.append(f" - *Precondition*: {precondition.description}") + + for step in spec.steps or []: + lines.append(f" - *Step {step.order}* [{step.actor}]: {step.action}") + if step.expected: + lines.append(f" - *Expected*: {step.expected}") + + for postcondition in spec.postconditions or []: + lines.append(f" - *Postcondition*: {postcondition.description}") + + +def _append_relationship_markdown(lines: list[str], spec: Spec) -> None: + for relationship in spec.relationships or []: + note = f" ({relationship.note})" if relationship.note else "" + lines.append( + f" - {spec.id} {relationship.rel_type.value} " + f"{relationship.spec_id}{note}" + ) + + +def _append_spec_markdown(lines: list[str], spec: Spec) -> None: + lines.append(_priority_line(spec)) + if spec.rationale: + lines.append(f" - *Rationale*: {spec.rationale}") + if isinstance(spec, BehavioralSpec): + _append_behavioral_markdown(lines, spec) + _append_relationship_markdown(lines, spec) + + +def _append_group_markdown(lines: list[str], group: SpecGroup) -> None: + lines.append(f"## {group.title}") + lines.append("") + if group.description: + lines.append(group.description) + lines.append("") + for spec in group.specs: + _append_spec_markdown(lines, spec) + lines.append("") + + +def render_markdown(spec_file: SpecFile) -> str: + """Render a single SpecFile as a Markdown string.""" + lines = [ + f"# {spec_file.title}", + "", + "## Overview", + "", + spec_file.description, + "", + f"**Version**: {spec_file.version}", + "", + "---", + "", + ] + for group in spec_file.groups: + _append_group_markdown(lines, group) + return "\n".join(lines) + + +def _add_behavioral_spec_fields(d: dict, spec: BehavioralSpec) -> None: + authored_sequences = { + "preconditions": [ + {"description": item.description} + for item in spec.preconditions or [] + ], + "steps": [_step_dict(item) for item in spec.steps or []], + "postconditions": [ + {"description": item.description} + for item in spec.postconditions or [] + ], + } + for field, value in authored_sequences.items(): + if value: + d[field] = value + + +def _spec_to_dict(spec: Spec, group: SpecGroup, file: SpecFile) -> dict: + """Serialize a spec to a dict with only authored (non-inherited) fields.""" + _ = group, file + d: dict = { + "id": spec.id, + "priority": spec.priority.value, + "statement": spec.statement, + } + authored_optional_fields = { + "rationale": spec.rationale, + "kind": spec.kind.value if spec.kind is not None else None, + "scope": ( + [s.value for s in spec.scope] if spec.scope is not None else None + ), + "tags": ( + [t.value for t in spec.tags] if spec.tags is not None else None + ), + "relationships": ( + [_rel_dict(relationship) for relationship in spec.relationships] + if spec.relationships + else None + ), + "lint_suppress": ( + [warning.value for warning in spec.lint_suppress] + if spec.lint_suppress + else None + ), + } + for field, value in authored_optional_fields.items(): + if value is not None: + d[field] = value + if not spec.testable: + d["testable"] = False + if isinstance(spec, BehavioralSpec): + _add_behavioral_spec_fields(d, spec) + return d + + +def _rel_dict(r: object) -> dict: + d = { + "rel_type": r.rel_type.value, # type: ignore[attr-defined] + "spec_id": r.spec_id, # type: ignore[attr-defined] + } + if r.note is not None: # type: ignore[attr-defined] + d["note"] = r.note # type: ignore[attr-defined] + return d + + +def _step_dict(s: object) -> dict: + d = { + "order": s.order, # type: ignore[attr-defined] + "actor": s.actor, # type: ignore[attr-defined] + "action": s.action, # type: ignore[attr-defined] + } + if s.expected is not None: # type: ignore[attr-defined] + d["expected"] = s.expected # type: ignore[attr-defined] + return d + + +def _group_to_dict(group: SpecGroup, file: SpecFile) -> dict: + """Serialize a group to a dict with only authored fields.""" + d: dict = {"id": group.id, "title": group.title} + if group.description is not None: + d["description"] = group.description + if group.kind is not None: + d["kind"] = group.kind.value + if group.scope is not None: + d["scope"] = [s.value for s in group.scope] + d["specs"] = [_spec_to_dict(s, group, file) for s in group.specs] + return d + + +def _file_to_dict(spec_file: SpecFile) -> dict: + """Serialize a SpecFile to a dict with only authored fields.""" + return { + "id": spec_file.id, + "title": spec_file.title, + "description": spec_file.description, + "version": spec_file.version, + "kind": spec_file.kind.value, + "scope": [s.value for s in spec_file.scope], + "groups": [_group_to_dict(g, spec_file) for g in spec_file.groups], + } + + +class _YamlDumper(yaml.SafeDumper): + """Custom YAML dumper with folded block scalars for long strings.""" + + pass + + +def _str_representer(dumper: yaml.SafeDumper, data: str) -> yaml.ScalarNode: + if "\n" in data or len(data) > 80: + return dumper.represent_scalar( + "tag:yaml.org,2002:str", data, style=">" + ) + return dumper.represent_scalar("tag:yaml.org,2002:str", data) + + +_YamlDumper.add_representer(str, _str_representer) + + +def export_yaml(spec_file: SpecFile) -> str: + """Serialize a single :class:`SpecFile` to authoritative YAML. + + Only authored fields are emitted — inherited defaults are omitted so + that the YAML remains the canonical source of truth. + """ + return yaml.dump( + _file_to_dict(spec_file), + Dumper=_YamlDumper, + default_flow_style=False, + sort_keys=False, + allow_unicode=True, + ) + + +def export_json( + registry: SpecRegistry, + *, + kind: str | None = None, + scope: str | None = None, + tags: list[str] | None = None, + priority: str | None = None, +) -> str: + """Serialize the registry (or a filtered subset) to JSON. + + Args: + registry: The loaded SpecRegistry. + kind: Filter to specs with this kind value (e.g. ``"general"``). + scope: Filter to specs whose scope list contains this value. + tags: Filter to specs that have ALL of the given tags. + priority: Filter to specs with this priority value (e.g. ``"MUST"``). + + Returns: + A JSON string of the filtered spec index. + """ + result = {} + for spec_id, spec in registry.all_specs.items(): + eff_kind = registry.get_effective_kind(spec_id) + eff_scope = registry.get_effective_scope(spec_id) + eff_tags = effective_tags(spec) + + if kind and eff_kind.value != kind: + continue + if scope and scope not in [s.value for s in eff_scope]: + continue + if tags and not all(t in [tg.value for tg in eff_tags] for t in tags): + continue + if priority and spec.priority.value != priority: + continue + result[spec_id] = spec.model_dump(mode="json") + + return json.dumps(result, indent=2) + + +def render_registry_markdown(registry: SpecRegistry) -> str: + """Render all spec files in the registry as concatenated Markdown.""" + parts = [render_markdown(f) for f in registry.files] + return "\n\n---\n\n".join(parts) + + +def main() -> None: + """CLI entry point for context generation. + + Usage:: + + python -m ssvc.metadata.specs.render --format md specs/ + python -m ssvc.metadata.specs.render --format json specs/ + python -m ssvc.metadata.specs.render --format yaml specs/ + python -m ssvc.metadata.specs.render --format llm-json specs/ + python -m ssvc.metadata.specs.render --format llm-json --topic SR specs/ + """ + import sys + + fmt = "md" + topic = None + args = sys.argv[1:] + if "--format" in args: + idx = args.index("--format") + fmt = args[idx + 1] + args = args[:idx] + args[idx + 2 :] + if "--topic" in args: + idx = args.index("--topic") + topic = args[idx + 1] + args = args[:idx] + args[idx + 2 :] + + if not args: + print( + f"Usage: {sys.argv[0]} [--format md|json|yaml|llm-json]" + " [--topic FILEID] ", + file=sys.stderr, + ) + sys.exit(2) + + spec_dir = Path(args[0]) + registry = load_registry(spec_dir) + + if fmt == "json": + print(export_json(registry)) + elif fmt == "yaml": + for sf in registry.files: + print(export_yaml(sf)) + print("---") + elif fmt == "llm-json": + from ssvc.metadata.specs.llm_export import to_llm_json + + print(to_llm_json(registry, topic=topic)) + else: + print(render_registry_markdown(registry)) + + +def main_llm_json() -> None: + """LLM-optimized spec dump entry point (``ssvc-spec-dump``). + + Exports all specs as flat, inheritance-resolved JSON for coding agents. + Defaults to the ``specs/`` directory relative to the current working + directory. + + Usage:: + + ssvc-spec-dump + ssvc-spec-dump specs/ + """ + import sys + + args = sys.argv[1:] + spec_dir = Path(args[0]) if args else Path("specs") + + if not spec_dir.is_dir(): + print( + f"Error: spec directory not found: {spec_dir}", + file=sys.stderr, + ) + sys.exit(2) + + from ssvc.metadata.specs.llm_export import to_llm_json + + registry = load_registry(spec_dir) + print(to_llm_json(registry)) + + +if __name__ == "__main__": + main() diff --git a/src/ssvc/metadata/specs/schema.py b/src/ssvc/metadata/specs/schema.py new file mode 100644 index 00000000..f20d50ed --- /dev/null +++ b/src/ssvc/metadata/specs/schema.py @@ -0,0 +1,281 @@ +"""Pydantic schema for ``specs/*.yaml`` structured requirement files. + +Design principle: YAML is the authoritative data source. The schema +validates what is present but does **not** silently inject defaults for +absent fields. Inheritable fields (``kind``, ``scope``) are required at +the file level and optional at group/spec level; effective values are +resolved by the registry loader, not by Pydantic defaults. + +TODO: Consider extracting this module (ssvc.metadata.specs) into a +standalone shared library (e.g. ``certcc-spec-registry``) so that both +SSVC and Vultron can depend on it rather than maintaining parallel copies. +""" + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +from __future__ import annotations + +from enum import StrEnum +from typing import Annotated, Union + +from pydantic import BaseModel, StringConstraints, field_validator + +from ssvc.metadata.base import NonEmptyStr + +SpecIdStr = Annotated[ + str, + StringConstraints(pattern=r"^[A-Z]{2,8}(-\d{2}(-\d{3})?)?$"), +] + + +class RFC2119Priority(StrEnum): + """RFC 2119 priority levels for requirements.""" + + MUST = "MUST" + MUST_NOT = "MUST_NOT" + SHOULD = "SHOULD" + SHOULD_NOT = "SHOULD_NOT" + MAY = "MAY" + + +class RelationType(StrEnum): + """Relationship types between spec requirements.""" + + IMPLEMENTS = "implements" + SUPERSEDES = "supersedes" + EXTENDS = "extends" + DEPENDS_ON = "depends_on" + CONFLICTS = "conflicts" + REFINES = "refines" + DERIVES_FROM = "derives_from" + VERIFIES = "verifies" + PART_OF = "part_of" + CONSTRAINS = "constrains" + + +class SpecKind(StrEnum): + """Portability tier for a spec requirement. + + The five tiers form a portability hierarchy. Use them to filter which + specs apply to your project: + + - ``general`` — Universal: any project, any language. + Examples: idempotency, linter discipline, CI + security. + - ``pattern`` — Architectural / framework approach: language-agnostic + and domain-independent. + Examples: hexagonal architecture, event-driven + dispatch, structured logging format. + - ``domain`` — Project domain: language-agnostic but specific to + the problem domain of this repository. + Examples (for SSVC): decision point logic, + scoring tree semantics, stakeholder outcome + definitions. + - ``language`` — Python ecosystem: any Python project. + Examples: pydantic conventions, pytest, FastAPI + patterns. + - ``implementation`` — This specific codebase. + Examples: file paths under ``src/ssvc/``, class + names, tooling configuration. + + Portability use cases + ~~~~~~~~~~~~~~~~~~~~~ + - Implementing the domain in Python → all five tiers + - Implementing the domain in another language → general + pattern + domain + - Different domain, same Python stack → general + pattern + language + - Architectural wisdom, any language → general + pattern + - Universal wisdom only → general + """ + + GENERAL = "general" + PATTERN = "pattern" + DOMAIN = "domain" + LANGUAGE = "language" + IMPLEMENTATION = "implementation" + + +class Scope(StrEnum): + """Deployment scope for a spec requirement.""" + + PROTOTYPE = "prototype" + PRODUCTION = "production" + + +class SpecTag(StrEnum): + """Controlled vocabulary of topic tags. + + This is an intentionally small generic set. Domain-specific tags + should be added here as the project's spec vocabulary grows. + + TODO: Extend this enum with SSVC-specific tags (e.g. decision-points, + scoring, stakeholder-roles) as specs are authored. + """ + + CI_CD = "ci-cd" + CODE_STYLE = "code-style" + DOCUMENTATION = "documentation" + PERFORMANCE = "performance" + SECURITY = "security" + TESTING = "testing" + TOOLING = "tooling" + + +class LintWarningCode(StrEnum): + """Named linter warnings that can be suppressed via ``lint_suppress``.""" + + TESTABLE_WITHOUT_STEPS = "testable_without_steps" + RATIONALE_TOO_LONG = "rationale_too_long" + MISSING_TAGS = "missing_tags" + + +class Relationship(BaseModel): + """Cross-spec traceability link.""" + + rel_type: RelationType + spec_id: SpecIdStr + note: str | None = None + + +class StatementSpec(BaseModel): + """A single normative statement requirement. + + Inheritable fields (``kind``, ``scope``) default to ``None``, meaning + "inherit from parent group or file." The registry loader resolves + effective values after loading. + """ + + id: SpecIdStr + priority: RFC2119Priority + statement: NonEmptyStr + rationale: NonEmptyStr | None = None + testable: bool = True + kind: SpecKind | None = None + scope: list[Scope] | None = None + tags: list[SpecTag] | None = None + relationships: list[Relationship] | None = None + lint_suppress: list[LintWarningCode] | None = None + + @field_validator("scope", "tags", "relationships", "lint_suppress") + @classmethod + def _nonempty_if_present(cls, v: list | None, info: object) -> list | None: + if v is not None and len(v) == 0: + field_name = getattr(info, "field_name", "list field") + raise ValueError(f"{field_name} must be non-empty if present") + return v + + +class Precondition(BaseModel): + """A precondition for a behavioral spec.""" + + description: str + + +class BehaviorStep(BaseModel): + """A single step in a behavioral spec sequence.""" + + order: int + actor: str + action: str + expected: str | None = None + + +class Postcondition(BaseModel): + """A postcondition for a behavioral spec.""" + + description: str + + +class BehavioralSpec(StatementSpec): + """A spec with structured pre/step/post conditions.""" + + preconditions: list[Precondition] | None = None + steps: list[BehaviorStep] | None = None + postconditions: list[Postcondition] | None = None + + @field_validator("preconditions", "steps", "postconditions") + @classmethod + def _behavioral_lists_nonempty(cls, v: list | None, info: object) -> list | None: + if v is not None and len(v) == 0: + field_name = getattr(info, "field_name", "list field") + raise ValueError(f"{field_name} must be non-empty if present") + return v + + +Spec = Union[BehavioralSpec, StatementSpec] + + +class SpecGroup(BaseModel): + """A logical grouping of specs within a file. + + ``kind`` and ``scope`` are optional overrides; when absent, values are + inherited from the containing :class:`SpecFile`. + """ + + id: SpecIdStr + title: NonEmptyStr + description: NonEmptyStr | None = None + kind: SpecKind | None = None + scope: list[Scope] | None = None + specs: list[Spec] + + @field_validator("scope") + @classmethod + def _nonempty_if_present(cls, v: list | None, info: object) -> list | None: + if v is not None and len(v) == 0: + field_name = getattr(info, "field_name", "list field") + raise ValueError(f"{field_name} must be non-empty if present") + return v + + @field_validator("specs") + @classmethod + def _specs_nonempty(cls, v: list) -> list: + if not v: + raise ValueError("specs must not be empty") + return v + + +class SpecFile(BaseModel): + """One YAML spec file with its groups and file-level metadata. + + ``kind`` and ``scope`` are required at the file level and serve as + defaults for groups and specs that do not override them. + """ + + id: str + title: NonEmptyStr + description: NonEmptyStr + version: NonEmptyStr + kind: SpecKind + scope: list[Scope] + groups: list[SpecGroup] + + @field_validator("scope") + @classmethod + def _scope_nonempty(cls, v: list) -> list: + if not v: + raise ValueError("scope must not be empty") + return v + + @field_validator("groups") + @classmethod + def _groups_nonempty(cls, v: list) -> list: + if not v: + raise ValueError("groups must not be empty") + return v diff --git a/src/test/metadata/__init__.py b/src/test/metadata/__init__.py new file mode 100644 index 00000000..739632b9 --- /dev/null +++ b/src/test/metadata/__init__.py @@ -0,0 +1,21 @@ + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +"""Tests for ssvc.metadata.specs.""" diff --git a/src/test/metadata/specs/__init__.py b/src/test/metadata/specs/__init__.py new file mode 100644 index 00000000..739632b9 --- /dev/null +++ b/src/test/metadata/specs/__init__.py @@ -0,0 +1,21 @@ + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +"""Tests for ssvc.metadata.specs.""" diff --git a/src/test/metadata/specs/conftest.py b/src/test/metadata/specs/conftest.py new file mode 100644 index 00000000..ee336fa5 --- /dev/null +++ b/src/test/metadata/specs/conftest.py @@ -0,0 +1,94 @@ +"""Shared fixtures for test/metadata/specs/ tests.""" + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +import pytest +import yaml + +from ssvc.metadata.specs.registry import load_registry + +MINIMAL_YAML = { + "id": "TST", + "title": "Test Spec File", + "description": "A spec file for unit testing", + "version": "0.1", + "kind": "general", + "scope": ["production"], + "groups": [ + { + "id": "TST-01", + "title": "Test Group One", + "specs": [ + { + "id": "TST-01-001", + "priority": "MUST", + "statement": "TST-01-001 MUST satisfy the test", + "rationale": "Required for test coverage", + "tags": ["testing"], + } + ], + } + ], +} + +SECOND_YAML = { + "id": "MOR", + "title": "More Test Specs", + "description": "Additional spec file for testing", + "version": "0.2", + "kind": "general", + "scope": ["production"], + "groups": [ + { + "id": "MOR-01", + "title": "More Group", + "specs": [ + { + "id": "MOR-01-001", + "priority": "SHOULD", + "statement": "MOR-01-001 SHOULD also work correctly", + "rationale": "Multi-file registry testing", + "tags": ["testing"], + } + ], + } + ], +} + + +@pytest.fixture +def spec_dir(tmp_path): + """Single-file spec directory with a minimal valid YAML spec.""" + (tmp_path / "test_specs.yaml").write_text(yaml.dump(MINIMAL_YAML)) + return tmp_path + + +@pytest.fixture +def multi_spec_dir(tmp_path): + """Multi-file spec directory with two distinct valid YAML specs.""" + (tmp_path / "test_specs.yaml").write_text(yaml.dump(MINIMAL_YAML)) + (tmp_path / "more_specs.yaml").write_text(yaml.dump(SECOND_YAML)) + return tmp_path + + +@pytest.fixture +def loaded_registry(spec_dir): + """Loaded SpecRegistry from the minimal single-file spec_dir.""" + return load_registry(spec_dir) diff --git a/src/test/metadata/specs/test_lint.py b/src/test/metadata/specs/test_lint.py new file mode 100644 index 00000000..7fd1e119 --- /dev/null +++ b/src/test/metadata/specs/test_lint.py @@ -0,0 +1,177 @@ +"""Tests for ssvc.metadata.specs.lint.""" + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +import yaml + +from ssvc.metadata.specs.lint import lint + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _write_yaml(path, data, filename="specs.yaml"): + (path / filename).write_text(yaml.dump(data)) + + +def _minimal_spec(spec_id="TST-01-001", priority="MUST", extra=None): + spec = { + "id": spec_id, + "priority": priority, + "statement": f"{spec_id} MUST do the thing", + "rationale": "Because testing", + "tags": ["testing"], + } + if extra: + spec.update(extra) + return { + "id": "TST", + "title": "Test File", + "description": "Test spec file", + "version": "0.1", + "kind": "general", + "scope": ["production"], + "groups": [ + { + "id": "TST-01", + "title": "Group", + "specs": [spec], + } + ], + } + + +# --------------------------------------------------------------------------- +# Clean cases +# --------------------------------------------------------------------------- + + +def test_lint_clean_dir(tmp_path, capsys): + _write_yaml(tmp_path, _minimal_spec()) + result = lint(tmp_path) + captured = capsys.readouterr() + assert result == 0 + assert "[ERROR]" not in captured.err + + +def test_lint_empty_dir(tmp_path): + result = lint(tmp_path) + assert result == 0 + + +# --------------------------------------------------------------------------- +# Hard errors +# --------------------------------------------------------------------------- + + +def test_lint_duplicate_spec_ids(tmp_path, capsys): + data = _minimal_spec("DUP-01-001") + data["id"] = "DUP" + data["groups"][0]["id"] = "DUP-01" + data["groups"][0]["specs"][0]["statement"] = "DUP-01-001 MUST be unique" + _write_yaml(tmp_path, data, "file1.yaml") + _write_yaml(tmp_path, data, "file2.yaml") + result = lint(tmp_path) + assert result == 1 + + +def test_lint_dangling_relationship(tmp_path, capsys): + data = _minimal_spec( + extra={ + "relationships": [ + {"rel_type": "depends_on", "spec_id": "XX-99-999"} + ] + } + ) + _write_yaml(tmp_path, data) + result = lint(tmp_path) + captured = capsys.readouterr() + assert result == 1 + assert "XX-99-999" in captured.err + + +def test_lint_group_prefix_mismatch(tmp_path, capsys): + data = _minimal_spec() + data["groups"][0]["id"] = "ZZZ-01" # wrong prefix + _write_yaml(tmp_path, data) + result = lint(tmp_path) + assert result == 1 + + +def test_lint_spec_id_prefix_mismatch(tmp_path, capsys): + data = _minimal_spec() + data["groups"][0]["specs"][0]["id"] = "TST-02-001" # wrong group + data["groups"][0]["specs"][0][ + "statement" + ] = "TST-02-001 MUST be in wrong group" + _write_yaml(tmp_path, data) + result = lint(tmp_path) + assert result == 1 + + +# --------------------------------------------------------------------------- +# Advisory warnings +# --------------------------------------------------------------------------- + + +def test_lint_warns_missing_tags(tmp_path, capsys): + data = _minimal_spec() + del data["groups"][0]["specs"][0]["tags"] + _write_yaml(tmp_path, data) + result = lint(tmp_path) + captured = capsys.readouterr() + assert result == 0 + assert "[WARN]" in captured.out + assert "no tags" in captured.out + + +def test_lint_warns_testable_false_no_steps(tmp_path, capsys): + data = _minimal_spec(extra={"testable": False}) + _write_yaml(tmp_path, data) + result = lint(tmp_path) + captured = capsys.readouterr() + assert result == 0 + assert "testable=false" in captured.out + + +def test_lint_suppress_testable_warning(tmp_path, capsys): + data = _minimal_spec( + extra={ + "testable": False, + "lint_suppress": ["testable_without_steps"], + } + ) + _write_yaml(tmp_path, data) + result = lint(tmp_path) + captured = capsys.readouterr() + assert result == 0 + assert "testable=false" not in captured.out + + +def test_lint_warns_rationale_too_long(tmp_path, capsys): + long_rationale = "x" * 501 + data = _minimal_spec(extra={"rationale": long_rationale}) + _write_yaml(tmp_path, data) + result = lint(tmp_path) + captured = capsys.readouterr() + assert result == 0 + assert "rationale exceeds" in captured.out diff --git a/src/test/metadata/specs/test_schema.py b/src/test/metadata/specs/test_schema.py new file mode 100644 index 00000000..db6a80ac --- /dev/null +++ b/src/test/metadata/specs/test_schema.py @@ -0,0 +1,606 @@ +"""Tests for ssvc.metadata.specs.schema.""" + +# Copyright (c) 2026 Carnegie Mellon University. +# NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE +# ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. +# CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, +# EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT +# NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR +# MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE +# OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE +# ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM +# PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. +# Licensed under a MIT (SEI)-style license, please see LICENSE or contact +# permission@sei.cmu.edu for full terms. +# [DISTRIBUTION STATEMENT A] This material has been approved for +# public release and unlimited distribution. Please see Copyright notice +# for non-US Government use and distribution. +# This Software includes and/or makes use of Third-Party Software each +# subject to its own license. +# DM24-0278 + +import pytest +import yaml +from pydantic import ValidationError + +from ssvc.metadata.specs.registry import load_registry +from ssvc.metadata.specs.schema import ( + BehaviorStep, + BehavioralSpec, + LintWarningCode, + Postcondition, + Precondition, + RFC2119Priority, + RelationType, + Relationship, + Scope, + SpecFile, + SpecGroup, + SpecKind, + SpecTag, + StatementSpec, +) + + +# --------------------------------------------------------------------------- +# SpecIdStr pattern +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "spec_id", + [ + "AB", + "ABCDEFGH", + "AB-01", + "AB-01-001", + "ABCD-99-999", + ], +) +def test_spec_id_str_valid(spec_id): + spec = StatementSpec( + id=spec_id, + priority=RFC2119Priority.MUST, + statement="MUST do something", + rationale="Because testing", + ) + assert spec.id == spec_id + + +@pytest.mark.parametrize( + "spec_id", + [ + "A", # too short + "ABCDEFGHI", # too long (9 chars) + "ab-01-001", # lowercase + "AB-1-001", # group with 1 digit + "AB-01-01", # spec number with 2 digits + "", # empty + "AB_01", # underscore + ], +) +def test_spec_id_str_invalid(spec_id): + with pytest.raises(ValidationError): + StatementSpec( + id=spec_id, + priority=RFC2119Priority.MUST, + statement="MUST do something", + rationale="Because testing", + ) + + +# --------------------------------------------------------------------------- +# StatementSpec — absent optional fields are None +# --------------------------------------------------------------------------- + + +def test_statement_spec_absent_fields(): + """Optional fields default to None when not provided.""" + spec = StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST satisfy this", + ) + assert spec.rationale is None + assert spec.testable is True + assert spec.kind is None + assert spec.scope is None + assert spec.tags is None + assert spec.relationships is None + assert spec.lint_suppress is None + + +def test_statement_spec_full(): + spec = StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.SHOULD, + statement="AB-01-001 SHOULD do the thing", + rationale="Because it helps", + testable=False, + kind=SpecKind.IMPLEMENTATION, + scope=[Scope.PROTOTYPE], + tags=[SpecTag.TESTING], + relationships=[ + Relationship( + rel_type=RelationType.DEPENDS_ON, + spec_id="AB-01-002", + note="needs this first", + ) + ], + lint_suppress=[LintWarningCode.TESTABLE_WITHOUT_STEPS], + ) + assert spec.testable is False + assert spec.kind == SpecKind.IMPLEMENTATION + assert spec.scope == [Scope.PROTOTYPE] + assert spec.tags is not None and len(spec.tags) == 1 + assert spec.relationships is not None and len(spec.relationships) == 1 + assert spec.lint_suppress is not None and len(spec.lint_suppress) == 1 + + +def test_statement_spec_empty_statement_rejected(): + with pytest.raises(ValidationError): + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="", + rationale="Because testing", + ) + + +def test_statement_spec_empty_rationale_rejected(): + with pytest.raises(ValidationError): + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST satisfy this", + rationale="", + ) + + +def test_statement_spec_rationale_none_allowed(): + spec = StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST satisfy this", + ) + assert spec.rationale is None + + +def test_statement_spec_rationale_omitted_allowed(): + spec = StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST satisfy this", + rationale=None, + ) + assert spec.rationale is None + + +# --------------------------------------------------------------------------- +# Non-empty-if-present list validation +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "field", ["scope", "tags", "relationships", "lint_suppress"] +) +def test_empty_list_rejected(field: str) -> None: + """Empty lists are rejected — use None for absent.""" + kwargs: dict = { + "id": "AB-01-001", + "priority": RFC2119Priority.MUST, + "statement": "AB-01-001 MUST pass", + field: [], + } + with pytest.raises(ValidationError, match="non-empty"): + StatementSpec(**kwargs) # type: ignore[arg-type] + + +# --------------------------------------------------------------------------- +# BehavioralSpec +# --------------------------------------------------------------------------- + + +def test_behavioral_spec_with_steps(): + spec = BehavioralSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST follow this workflow", + rationale="Protocol requirement", + preconditions=[Precondition(description="System is ready")], + steps=[ + BehaviorStep( + order=1, + actor="sender", + action="sends the message", + expected="message delivered", + ) + ], + postconditions=[Postcondition(description="State updated")], + ) + assert spec.steps is not None and len(spec.steps) == 1 + assert spec.steps[0].order == 1 + assert spec.preconditions is not None and len(spec.preconditions) == 1 + assert spec.postconditions is not None and len(spec.postconditions) == 1 + + +def test_behavioral_spec_absent_steps_valid(): + spec = BehavioralSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST do something", + rationale="Because testing", + ) + assert spec.steps is None + + +@pytest.mark.parametrize("field", ["preconditions", "steps", "postconditions"]) +def test_behavioral_spec_empty_list_rejected(field: str) -> None: + kwargs: dict = { + "id": "AB-01-001", + "priority": RFC2119Priority.MUST, + "statement": "AB-01-001 MUST pass", + field: [], + } + with pytest.raises(ValidationError, match="non-empty"): + BehavioralSpec(**kwargs) # type: ignore[arg-type] + + +@pytest.mark.parametrize( + "field", ["scope", "tags", "relationships", "lint_suppress"] +) +def test_behavioral_spec_inherited_empty_list_rejected(field: str) -> None: + """Inherited non-empty validators still fire on BehavioralSpec instances.""" + kwargs: dict = { + "id": "AB-01-001", + "priority": RFC2119Priority.MUST, + "statement": "AB-01-001 MUST pass", + field: [], + } + with pytest.raises(ValidationError, match="non-empty"): + BehavioralSpec(**kwargs) # type: ignore[arg-type] + + +# --------------------------------------------------------------------------- +# SpecGroup +# --------------------------------------------------------------------------- + + +def test_spec_group_valid(): + group = SpecGroup( + id="AB-01", + title="Test Group", + specs=[ + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST do the thing", + rationale="Rationale", + ) + ], + ) + assert group.id == "AB-01" + assert len(group.specs) == 1 + + +def test_spec_group_empty_title_rejected(): + with pytest.raises(ValidationError): + SpecGroup( + id="AB-01", + title="", + specs=[ + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST exist", + ) + ], + ) + + +def test_spec_group_empty_specs_rejected(): + with pytest.raises(ValidationError, match="must not be empty"): + SpecGroup(id="AB-01", title="Empty Group", specs=[]) + + +def test_spec_group_description_nonempty_if_present(): + with pytest.raises(ValidationError): + SpecGroup( + id="AB-01", + title="Group", + description="", + specs=[ + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST exist", + ) + ], + ) + + +def test_spec_group_description_none_allowed(): + group = SpecGroup( + id="AB-01", + title="Group", + specs=[ + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST exist", + ) + ], + ) + assert group.description is None + + +def test_spec_group_empty_scope_rejected(): + with pytest.raises(ValidationError, match="non-empty"): + SpecGroup( + id="AB-01", + title="Group", + scope=[], + specs=[ + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST exist", + ) + ], + ) + + +# --------------------------------------------------------------------------- +# SpecFile +# --------------------------------------------------------------------------- + + +def test_spec_file_valid(): + sf = SpecFile( + id="AB", + title="Test File", + description="A test file", + version="0.1", + kind=SpecKind.GENERAL, + scope=[Scope.PRODUCTION], + groups=[ + SpecGroup( + id="AB-01", + title="Group", + specs=[ + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST work", + rationale="Because", + ) + ], + ) + ], + ) + assert sf.id == "AB" + assert len(sf.groups) == 1 + + +def test_spec_file_requires_kind(): + with pytest.raises(ValidationError): + SpecFile( # type: ignore[call-arg] + id="AB", + title="Test File", + description="A test file", + version="0.1", + scope=[Scope.PRODUCTION], + groups=[ + SpecGroup( + id="AB-01", + title="Group", + specs=[ + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST work", + ) + ], + ) + ], + ) + + +def test_spec_file_requires_scope(): + with pytest.raises(ValidationError): + SpecFile( # type: ignore[call-arg] + id="AB", + title="Test File", + description="A test file", + version="0.1", + kind=SpecKind.GENERAL, + groups=[ + SpecGroup( + id="AB-01", + title="Group", + specs=[ + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST work", + ) + ], + ) + ], + ) + + +def test_spec_file_empty_scope_rejected(): + with pytest.raises(ValidationError, match="must not be empty"): + SpecFile( + id="AB", + title="Test File", + description="A test file", + version="0.1", + kind=SpecKind.GENERAL, + scope=[], + groups=[ + SpecGroup( + id="AB-01", + title="Group", + specs=[ + StatementSpec( + id="AB-01-001", + priority=RFC2119Priority.MUST, + statement="AB-01-001 MUST work", + ) + ], + ) + ], + ) + + +def test_spec_file_empty_groups_rejected(): + with pytest.raises(ValidationError, match="must not be empty"): + SpecFile( + id="AB", + title="Test File", + description="A test file", + version="0.1", + kind=SpecKind.GENERAL, + scope=[Scope.PRODUCTION], + groups=[], + ) + + +# --------------------------------------------------------------------------- +# SpecRegistry / load_registry +# --------------------------------------------------------------------------- + + +def test_registry_duplicate_spec_id_raises(tmp_path): + dup_data = { + "id": "DUP", + "title": "Dup File", + "description": "Duplicate spec IDs", + "version": "0.1", + "kind": "general", + "scope": ["production"], + "groups": [ + { + "id": "DUP-01", + "title": "Group", + "specs": [ + { + "id": "DUP-01-001", + "priority": "MUST", + "statement": "DUP-01-001 MUST be unique", + "rationale": "Uniqueness", + }, + { + "id": "DUP-01-001", # duplicate + "priority": "SHOULD", + "statement": "DUP-01-001 SHOULD also exist", + "rationale": "But is duplicate", + }, + ], + } + ], + } + (tmp_path / "dup.yaml").write_text(yaml.dump(dup_data)) + with pytest.raises(ValueError, match="Duplicate spec ID"): + load_registry(tmp_path) + + +def test_load_registry_round_trip(spec_dir): + registry = load_registry(spec_dir) + assert len(registry.files) == 1 + spec = registry.get("TST-01-001") + assert spec.priority == RFC2119Priority.MUST + + +def test_load_registry_empty_dir(tmp_path): + registry = load_registry(tmp_path) + assert registry.files == [] + + +def test_registry_get_unknown_raises(spec_dir): + registry = load_registry(spec_dir) + with pytest.raises(KeyError): + registry.get("XX-99-999") + + +def test_registry_all_specs(spec_dir): + registry = load_registry(spec_dir) + assert "TST-01-001" in registry.all_specs + + +def test_registry_validate_cross_references_clean(spec_dir): + registry = load_registry(spec_dir) + assert registry.validate_cross_references() == [] + + +# --------------------------------------------------------------------------- +# Inheritance resolution +# --------------------------------------------------------------------------- + + +def test_effective_kind_inherits_from_file(spec_dir): + registry = load_registry(spec_dir) + assert registry.get_effective_kind("TST-01-001") == SpecKind.GENERAL + + +def test_effective_scope_inherits_from_file(spec_dir): + registry = load_registry(spec_dir) + assert registry.get_effective_scope("TST-01-001") == [Scope.PRODUCTION] + + +def test_effective_kind_spec_override(tmp_path): + data = { + "id": "TST", + "title": "Test", + "description": "Test", + "version": "0.1", + "kind": "general", + "scope": ["production"], + "groups": [ + { + "id": "TST-01", + "title": "Group", + "specs": [ + { + "id": "TST-01-001", + "priority": "MUST", + "statement": "TST-01-001 MUST pass", + "kind": "implementation", + } + ], + } + ], + } + (tmp_path / "test.yaml").write_text(yaml.dump(data)) + registry = load_registry(tmp_path) + assert registry.get_effective_kind("TST-01-001") == SpecKind.IMPLEMENTATION + + +def test_effective_kind_group_override(tmp_path): + data = { + "id": "TST", + "title": "Test", + "description": "Test", + "version": "0.1", + "kind": "general", + "scope": ["production"], + "groups": [ + { + "id": "TST-01", + "title": "Group", + "kind": "implementation", + "specs": [ + { + "id": "TST-01-001", + "priority": "MUST", + "statement": "TST-01-001 MUST pass", + } + ], + } + ], + } + (tmp_path / "test.yaml").write_text(yaml.dump(data)) + registry = load_registry(tmp_path) + assert registry.get_effective_kind("TST-01-001") == SpecKind.IMPLEMENTATION diff --git a/uv.lock b/uv.lock index 003757d7..533354a3 100644 --- a/uv.lock +++ b/uv.lock @@ -187,6 +187,7 @@ dependencies = [ { name = "pandas" }, { name = "pydantic" }, { name = "pymdown-extensions" }, + { name = "pyyaml" }, { name = "scikit-learn" }, { name = "scipy" }, { name = "semver" }, @@ -221,6 +222,7 @@ requires-dist = [ { name = "pandas", specifier = ">=2.3.2" }, { name = "pydantic", specifier = ">=2.11.7" }, { name = "pymdown-extensions", specifier = ">=10.21.2" }, + { name = "pyyaml", specifier = ">=6.0" }, { name = "scikit-learn", specifier = ">=1.6.1" }, { name = "scipy", specifier = ">=1.16.1" }, { name = "semver", specifier = ">=3.0.4" },