GitHub Actions YAML extraction, ci-select, metadata.json contract doc

ARCHITECTURE.md

@@ -8,7 +8,7 @@ graphify is a Claude Code skill backed by a Python library. The skill orchestrat
 detect()  →  extract()  →  build()  →  cluster()  →  analyze helpers  →  report.generate()  →  export.to_*()
 ```
 
-Each stage lives in its own module and they communicate through plain Python dicts and NetworkX graphs - no shared state, no side effects outside `graphify-out/`. Most stages are a single function; `analyze.py` and `export.py` are sets of sibling functions rather than one entry point.
+Each stage lives in its own module; the public contract between them is plain Python dicts and NetworkX graphs, not shared in-process state. `extract()` does have process-level side effects of its own -- it raises the recursion limit, clears its own module-level caches on each call, and can emit warnings to stderr -- but nothing it does is visible to another stage except through the dict/graph it returns. Most stages are a single function; `analyze.py` and `export.py` are sets of sibling functions rather than one entry point.
 
 ## Module responsibilities
 

README.md

@@ -801,6 +801,7 @@ graphify label ./my-project --backend=openai --model gpt-4o   # force a specific
 - [How it works](docs/how-it-works.md) — the extraction pipeline, community detection, confidence scoring, benchmarks
 - [ARCHITECTURE.md](ARCHITECTURE.md) — module breakdown, how to add a language
 - [Optional integrations](docs/docker-mcp-sqlite.md) — Docker MCP Toolkit + SQLite
+- [Graph bundle metadata](docs/graph-bundle-metadata.md) — schema for `metadata.json` when a `graphify --update` output is published for other machines/CI to pull
 - [The Memory Layer](https://safishamsi.gumroad.com/l/qetvlo) — the book on the ideas behind graphify, the architecture end to end
 
 ---
@@ -870,7 +871,7 @@ is added to CI later. The Bandit and pip-audit CI steps currently use
 `continue-on-error`, so their findings are advisory rather than blocking.
 
 > macOS note: the test suite includes both `sample.f90` and `sample.F90` fixtures. These collide on case-insensitive HFS+ / APFS file systems. Run on Linux or in a Docker container if you need to test both Fortran variants simultaneously.
-
+>
 > Windows note: the native Windows test suite exercises symbolic links, long
 > paths, POSIX permissions, path separators, and UTF-8 filesystem behavior.
 > Enable Windows Developer Mode to allow unprivileged symbolic-link creation, or

docs/graph-bundle-metadata.md

@@ -0,0 +1,57 @@
+# graphify bundle metadata (`metadata.json`)
+
+When a `graphify --update` output is published for other machines/CI jobs to
+pull (rather than generated locally), it ships as one atomic bundle:
+`graph.json` + `GRAPH_REPORT.md` + `manifest.json` + a small `metadata.json`
+describing the bundle itself.
+
+`metadata.json` exists because the bundle has two independent consumers that
+must never drift apart on what fields to expect:
+
+- **Writer**: the CI job that runs `graphify --update` and publishes the
+  bundle (e.g. a scheduled graph-refresh workflow).
+- **Reader**: the fetch script that pulls the bundle down and validates it
+  before swapping it into a local `graphify-out/`, ahead of graphify's own
+  `CLAUDE.md` directive / `PreToolUse` hook consuming it.
+
+Both of those typically live in consuming repos outside this one, but both are built
+against graphify's own output format, so this repo is the natural single
+source of truth for the contract between them -- one documented schema
+instead of two independently-evolving assumptions.
+
+**Schema**: [`graph-bundle-metadata.schema.json`](./graph-bundle-metadata.schema.json)
+(JSON Schema, draft 2020-12).
+
+## Example
+
+```json
+{
+  "schema_version": 1,
+  "source_sha": "e4bfd2ad1a9393251023a4edef93e93dc798afc7",
+  "graphify_version": "0.9.41",
+  "generated_at": "2026-08-13T02:00:00Z",
+  "bundle": {
+    "graph": "graph.json",
+    "report": "GRAPH_REPORT.md",
+    "manifest": "manifest.json"
+  }
+}
+```
+
+## What each field is for
+
+- `source_sha` / `generated_at`: staleness. A consumer compares `source_sha`
+  against its local `HEAD` (ancestry check, not a race guard -- the bundle's
+  own publish path is already serialized by a CI `concurrency:` group); when
+  that comparison isn't possible, `generated_at` backs a TTL fallback.
+- `graphify_version`: a hard compatibility gate. A version mismatch against
+  the locally-installed `graphify --version` means the fetch script refuses
+  to load the bundle and prints the exact upgrade command, rather than
+  risking a schema-mismatched `graph.json` being consulted silently.
+- `bundle`: where the other three files live inside the archive, so the
+  reader doesn't hardcode filenames independently of what the writer chose.
+
+`manifest.json` rides along for the *writer's* own benefit (restoring
+incremental-extraction continuity across ephemeral CI runners between
+scheduled runs) -- ordinary consumers only need `graph.json` and
+`GRAPH_REPORT.md`.

docs/graph-bundle-metadata.schema.json

@@ -0,0 +1,50 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://github.com/Graphify-Labs/graphify/blob/v8/docs/graph-bundle-metadata.schema.json",
+  "title": "graphify knowledge-graph bundle metadata",
+  "description": "Schema for metadata.json, the small manifest bundled alongside graph.json/GRAPH_REPORT.md/manifest.json when a graphify --update output is published as a release/OCI artifact. Written by the CI job that runs `graphify --update` and publishes the bundle; read by the fetch script that pulls the bundle down before graphify's CLAUDE.md directive/PreToolUse hook consult it. This is the single canonical definition both sides validate against, so the writer and reader can't drift independently -- one typically lives in a scheduled graph-refresh workflow, the other in a SessionStart fetch script, both consuming repos using the same published schema.",
+  "type": "object",
+  "required": ["schema_version", "source_sha", "graphify_version", "generated_at", "bundle"],
+  "additionalProperties": false,
+  "properties": {
+    "schema_version": {
+      "type": "integer",
+      "const": 1,
+      "description": "Version of this metadata.json schema itself, not of graphify or the bundle contents. Bump on any breaking change to this file's shape so a reader can refuse an unrecognized version cleanly instead of guessing at missing/renamed fields."
+    },
+    "source_sha": {
+      "type": "string",
+      "pattern": "^[0-9a-f]{40}$",
+      "description": "Full git commit SHA of the source repository HEAD that graphify --update was run against. The staleness check compares this against a consumer's local HEAD (via `git merge-base --is-ancestor`, used purely as a freshness signal, not a publish-time race guard)."
+    },
+    "graphify_version": {
+      "type": "string",
+      "description": "Output of `graphify --version` for the graphify install that generated this bundle (e.g. \"0.9.41\"). The fetch script refuses to load a bundle whose graphify_version doesn't match the locally installed `graphify --version`, printing the exact upgrade command, rather than silently loading a graph shaped by a different schema/format version."
+    },
+    "generated_at": {
+      "type": "string",
+      "format": "date-time",
+      "description": "ISO 8601 UTC timestamp of when this bundle was published. Backs the TTL fallback staleness check (e.g. treat the bundle as stale if generated_at is >24h old) for the case where source_sha ancestry comparison isn't possible (e.g. consumer's local HEAD is on an unrelated branch/fork)."
+    },
+    "bundle": {
+      "type": "object",
+      "required": ["graph", "report", "manifest"],
+      "additionalProperties": false,
+      "description": "Paths, relative to the archive root, of the other files published alongside this metadata.json in the same atomic bundle (a single archive, so a consumer either gets the whole matched set or a clean 404/missing-bundle, never a mismatched pair from two different publishes).",
+      "properties": {
+        "graph": {
+          "type": "string",
+          "description": "Path to graph.json -- the queryable knowledge graph itself. What ordinary consumers (graphify's PreToolUse hook, ci-select) actually load."
+        },
+        "report": {
+          "type": "string",
+          "description": "Path to GRAPH_REPORT.md -- the human-readable summary of the graph."
+        },
+        "manifest": {
+          "type": "string",
+          "description": "Path to manifest.json -- graphify's own incremental-extraction state. Only needed by the generation workflow itself to restore continuity before its next `graphify --update` run (see the artifact-re-pull fallback for actions/cache eviction); ordinary consumers querying the graph never need to open it."
+        }
+      }
+    }
+  }
+}

graphify/__main__.py

@@ -622,6 +622,14 @@ def _run_cli() -> None:
         print("  global remove <tag>      remove a repo's nodes from the global graph")
         print("  global list              list repos in the global graph")
         print("  global path              print path to the global graph file")
+        print("  ci-select               graph-informed CI test selection from a diff")
+        print("    --repo <name>           repository name (required)")
+        print("    --diff-cmd <cmd>        shell command to produce a diff (e.g. 'git diff origin/main...HEAD')")
+        print("    --diff <path|->         read diff from file or stdin")
+        print("    --files <f1,f2,...>      comma-separated changed file paths")
+        print("    --graph <path>          path to graph.json (default graphify-out/graph.json)")
+        print("    --test-jobs <path>      path to test-jobs.yaml mapping (auto-detected if omitted)")
+        print("    --depth N               BFS traversal depth (default 3)")
         print("  benchmark [graph.json]  measure token reduction vs naive full-corpus approach")
         print("  export callflow-html    emit Mermaid-based architecture/call-flow HTML")
         print("  hook install            install post-commit/post-checkout git hooks (all platforms)")

graphify/cache.py

@@ -972,11 +972,15 @@ def load_cached(path: Path, root: Path = Path("."), kind: str = "ast",
     if entry.exists():
         try:
             result = json.loads(entry.read_text(encoding="utf-8"))
-        except json.JSONDecodeError:
+        except (json.JSONDecodeError, UnicodeDecodeError):
             # Corrupt entry, not a miss: a truncated write or a bad producer
             # (e.g. unescaped Windows backslashes in source_file) leaves JSON
             # that fails to parse on every future run, so the file is silently
             # re-extracted forever. Count it so the run can report it (#2405).
+            # UnicodeDecodeError included: read_text() can raise it before
+            # json.loads() ever runs, e.g. a truncated write that cuts off
+            # mid multi-byte UTF-8 character -- the same "corrupt, not a
+            # miss" case, just caught one call earlier.
             _corrupt_cache_entries += 1
             return None
         except OSError: