feat: integration train — Waves 0–5 dependency-ordered manifest + verifier

.github/workflows/integration-train-validate.yml

@@ -0,0 +1,61 @@
+name: Integration train validate
+
+# Validates the integration-train manifest (`meta/integration-train.json`)
+# is a well-formed instance of its declared schema, and that `verify-train.sh`
+# stays syntactically valid (so the wave-by-wave verifier never silently
+# breaks).
+#
+# Mirrors the OpenAPI-validate workflow pattern from PR #5 — fail fast on
+# any structural drift to the parity-proof artifact.
+
+on:
+  pull_request:
+    paths:
+      - 'meta/integration-train.json'
+      - 'docs/integration-train.md'
+      - 'verify-train.sh'
+      - '.github/workflows/integration-train-validate.yml'
+  push:
+    branches: [master]
+  workflow_dispatch:
+
+jobs:
+  validate-manifest:
+    name: Validate integration-train manifest + verifier
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Validate manifest is valid JSON
+        run: |
+          python3 -c "
+          import json, sys
+          d = json.load(open('meta/integration-train.json'))
+          assert 'schema' in d, 'missing schema field'
+          assert d['schema'].startswith('integration-train/'), \
+              f\"unexpected schema: {d['schema']}\"
+          assert 'waves' in d, 'missing waves field'
+          assert isinstance(d['waves'], list) and len(d['waves']) > 0, \
+              'waves must be a non-empty list'
+          for w in d['waves']:
+              assert 'id' in w, f'wave missing id: {w}'
+              assert 'name' in w, f\"wave {w['id']} missing name\"
+              assert 'status' in w, f\"wave {w['id']} missing status\"
+              assert w['status'] in {
+                  'PROVEN', 'IN_FLIGHT', 'BLOCKING_ALL',
+                  'GREEN_IN_ISOLATION', 'ANCHOR_DEFINED',
+              }, f\"wave {w['id']} bad status: {w['status']}\"
+          ids = [w['id'] for w in d['waves']]
+          assert ids == sorted(ids), f'wave ids not sorted: {ids}'
+          print(f'OK schema={d[\"schema\"]} waves={len(d[\"waves\"])} ids={ids}')
+          "
+
+      - name: Validate verify-train.sh is syntactically valid
+        run: bash -n verify-train.sh
+
+      - name: Validate verify-train.sh is executable
+        run: |
+          if [ ! -x verify-train.sh ]; then
+            echo "::error::verify-train.sh must be executable"
+            exit 1
+          fi

.github/workflows/openapi-validate.yml

@@ -0,0 +1,107 @@
+name: OpenAPI validate
+
+# Validates the generated meos-openapi.json against the OpenAPI 3.1 spec on
+# every PR that touches the parser, generator, or meta files — and on every
+# push to master. Fails the build if the projection is not a valid OpenAPI
+# document.
+#
+# Natural follow-up named in PR #5's body. Runs the same regenerate path a
+# downstream consumer would: clone MobilityDB master for headers, parse with
+# libclang, produce the enriched catalog, project to OpenAPI, validate.
+
+on:
+  pull_request:
+    paths:
+      - 'parser/**'
+      - 'generator/**'
+      - 'meta/**'
+      - 'generate_openapi.py'
+      - 'run.py'
+      - 'requirements.txt'
+      - '.github/workflows/openapi-validate.yml'
+  push:
+    branches: [master]
+  workflow_dispatch:
+
+jobs:
+  openapi-validate:
+    name: Regenerate + validate meos-openapi.json
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      # libclang (Python wheel) needs the system C headers MEOS depends on
+      # so types like `size_t`, `json_object *`, `GSERIALIZED *`, … resolve
+      # to their real names instead of degrading to `int` / `int *`. Mirror
+      # the same install set as MobilityAPI's vendor-drift workflow so the
+      # two regenerate paths produce byte-identical catalogs.
+      - name: Install dev headers for libclang sysroot
+        run: |
+          sudo apt-get update -qq
+          sudo apt-get install -y --no-install-recommends \
+            clang libclang-dev \
+            libjson-c-dev libgsl-dev libproj-dev libgeos-dev \
+            postgresql-server-dev-16
+
+      - name: Clone MobilityDB master (MEOS headers source)
+        run: |
+          git clone --depth 1 https://github.com/MobilityDB/MobilityDB \
+            "$RUNNER_TEMP/mobilitydb"
+          echo "MOBILITYDB_HEADERS=$RUNNER_TEMP/mobilitydb/meos/include" \
+            >> "$GITHUB_ENV"
+
+      - name: Install Python dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+          pip install openapi-spec-validator
+
+      # generate_openapi.py requires the enriched catalog (network fields).
+      # The enrichment pipeline lives on PR #4 (feat/service-enrichment),
+      # which on its branch rewrites run.py to do parse + reconcile + enrich
+      # in one step.  On any branch that does not yet include #4's content,
+      # check out the PR #4 tree (parser/ + run.py) so the regenerate step
+      # uses the enriched-pipeline run.py.  Tree-level checkout sidesteps
+      # the runner's lack of a default git identity.
+      - name: Fetch + apply enrichment pipeline from PR #4 if absent
+        run: |
+          if [ ! -f parser/enrich.py ]; then
+            echo "::notice::PR #4 enrichment pipeline not present on this branch; checking out PR #4 tree"
+            git fetch origin refs/pull/4/head:pr4
+            # PR #4 supplies: parser/enrich.py, parser/header_types.py, and
+            # a rewritten run.py that orchestrates parse + reconcile + enrich
+            # in one invocation.  Copy all three onto the working tree.
+            git checkout pr4 -- parser/ run.py 2>/dev/null || true
+            ls -la parser/enrich.py run.py
+          fi
+
+      - name: Regenerate the catalog (parse + reconcile + enrich in one step)
+        run: python3 run.py "$MOBILITYDB_HEADERS"
+
+      - name: Project to OpenAPI 3.1
+        run: python3 generate_openapi.py
+
+      - name: Validate meos-openapi.json against OpenAPI 3.1
+        run: |
+          python3 -c "
+          import json
+          from openapi_spec_validator import OpenAPIV31SpecValidator
+          spec = json.load(open('output/meos-openapi.json'))
+          # OpenAPIV31SpecValidator(spec).validate() raises on violation,
+          # returns None on success. Works with openapi-spec-validator 0.9.x.
+          OpenAPIV31SpecValidator(spec).validate()
+          print(f\"::notice::meos-openapi.json conforms to OpenAPI 3.1 — \"
+                f\"{len(spec.get('paths', {}))} paths, \"
+                f\"{len(spec.get('components', {}).get('schemas', {}))} schemas.\")
+          "
+
+      - name: Upload meos-openapi.json as artefact
+        uses: actions/upload-artifact@v4
+        with:
+          name: meos-openapi
+          path: output/meos-openapi.json
+          if-no-files-found: error

README.md

@@ -10,8 +10,10 @@ This catalog is the foundation for generating language bindings (Python, Java, G
 - [How it works](#how-it-works)
 - [Getting started](#getting-started)
 - [Output format](#output-format)
+- [Service-projection metadata](#service-projection-metadata)
 - [Adding metadata](#adding-metadata)
 - [Portable bare-name dialect](#portable-bare-name-dialect)
+- [OpenAPI generation](#openapi-generation)
 
 ## Ecosystem
 
@@ -60,10 +62,12 @@ service contracts generated from the same model:
 
 ## How it works
 
-The pipeline runs in two steps:
+The pipeline runs in four steps:
 
 1. **Parser** — scans the MEOS `.h` header files using libclang and extracts every function signature, struct, and enum into structured JSON.
-2. **Merger** — enriches the parser output with manual annotations from `meta/meos-meta.json`, such as documentation and memory ownership rules.
+2. **Reconcile** — restores opaque types the PostgreSQL stub headers `#define` to `int` (`Interval`, `text`, …) from the header source, so they are not mistaken for `int *` out-parameters.
+3. **Enrich** — derives the service-projection metadata (`category` / `typeEncodings` / `network` / `wire`).
+4. **Merger** — applies manual annotations from `meta/meos-meta.json` (documentation, ownership, overrides) on top.
 
 ## Getting started
 
@@ -127,9 +131,60 @@ A typical function entry looks like this:
 }
 ```
 
+## Service-projection metadata
+
+C headers describe *signatures*; they do not say what a function **is**, how an
+opaque type crosses the wire, or whether an operation can be served
+*statelessly*. A second pass (`parser/enrich.py`) derives that — the metadata a
+service generator (OpenAPI, MCP, gRPC, …) needs to project MEOS onto a network
+API. It runs **before** the merge, so every derived field is overridable from
+`meta/meos-meta.json`.
+
+Each function gains a `category`, a `network` verdict, and a `wire` mapping:
+
+```json
+{
+  "name": "temporal_eq",
+  "returnType": { "c": "bool", "canonical": "int" },
+  "params": [ { "name": "temp1", "canonical": "const struct Temporal *" },
+              { "name": "temp2", "canonical": "const struct Temporal *" } ],
+  "category": "predicate",
+  "network": { "exposable": true, "method": "POST", "reason": null },
+  "wire": {
+    "params": [
+      { "name": "temp1", "kind": "serialized", "cType": "const struct Temporal *",
+        "decode": "temporal_in", "encodings": ["mfjson","text","wkb"] },
+      { "name": "temp2", "kind": "serialized", "cType": "const struct Temporal *",
+        "decode": "temporal_in", "encodings": ["mfjson","text","wkb"] }
+    ],
+    "result": { "kind": "json", "json": "integer" }
+  }
+}
+```
+
+(MEOS predicates return `int`, and libclang emits canonical spellings such as
+`const struct Temporal *` — the enrichment matches those.)
+
+```text
+Live coverage (MobilityDB master): 2161 public + 511 internal functions.
+The service projects the public user API; internal (meos_internal*.h,
+Datum-generic) is policy-excluded.
+  1963 / 2161 = 91% of the public API stateless-exposable (verified).
+```
+
+The catalog also gains a top-level `typeEncodings` map (opaque type → its
+in/out functions) and an `enrichment` summary (category counts, exposable
+count) for coverage tracking. Non-exposable functions carry a precise
+`reason` (`array-or-out-param:…`, `no-encoder:…`, `lifecycle`, `index`, …) so
+generators can report exactly what they can and cannot emit.
+
+See [`docs/enrichment.md`](docs/enrichment.md) for the full contract and
+[`tests/test_enrich.py`](tests/test_enrich.py) for worked examples on real
+MEOS signatures (run: `python3 tests/test_enrich.py`).
+
 ## Adding metadata
 
-Manual annotations (ownership rules, additional documentation, deprecation flags, etc.) live in `meta/meos-meta.json`. The merger applies them on top of the libclang-parsed structure when generating the final catalog.
+Manual annotations (ownership rules, additional documentation, deprecation flags, etc.) live in `meta/meos-meta.json`. The merger applies them on top of the libclang-parsed structure when generating the final catalog — including any field derived by the service-projection pass (e.g. correcting a `category` or forcing `network.exposable`).
 
 ## Portable bare-name dialect
 
@@ -148,3 +203,30 @@ type-agnostic and applies to **every** temporal type family —
 must not be excluded from any parity headline. `python tools/portable_parity.py`
 audits it against the catalog — currently **29/29 = 100%** backed (verified,
 no guessing). See [`docs/portable-aliases.md`](docs/portable-aliases.md).
+
+## OpenAPI generation
+
+The enriched catalog (the `network` / `wire` / `typeEncodings` produced by the
+service-projection pass) can be projected onto an **OpenAPI 3.1** contract —
+this is the concrete "OpenAPI is a projection of MEOS-API" step:
+
+```bash
+python run.py                 # produce the enriched catalog
+python generate_openapi.py    # output/meos-idl.json -> output/meos-openapi.json
+```
+
+Every *stateless-exposable* MEOS function becomes one RPC-style
+`POST /{function}` operation (≈ an OGC API – Processes "process"); opaque
+values cross the wire as strings carried in their `typeEncodings`
+(text / MF-JSON / HexWKB), surfaced as reusable component schemas. `x-meos-*`
+extensions carry the decode/encode function names and category so a
+downstream server or MCP generator can consume the same document.
+
+Against the live MobilityDB `master` catalog this yields **1952 operations**
+(90% of the public API; internal `meos_internal*.h` policy-excluded),
+including array-of-string params for builders. The generator is pure
+`dict` → `dict` (no libclang,
+no MEOS runtime); see [`docs/openapi.md`](docs/openapi.md) for the projection
+rules, `x-meos-*` extensions, and roadmap (OGC API, MCP, runtime server), and
+[`tests/test_openapi.py`](tests/test_openapi.py) for worked examples
+(`python3 tests/test_openapi.py`).