Add CI workflows, /ci skill, and test fixes

.claude/skills/ci/SKILL.md

@@ -0,0 +1,96 @@
+---
+name: ci
+description: |
+  Monitor GitHub Actions CI runs for pgxntool and/or pgxntool-test after a push.
+  Reports which branches are under test, per-job pass/fail, and failure details.
+  Uses shell scripts for all heavy work to minimize context consumption.
+
+  Use when: "monitor CI", "watch CI", "check CI", "/ci"
+allowed-tools: Bash(bash .claude/skills/ci/scripts/*), Read
+---
+
+# CI Monitor Skill
+
+Monitor GitHub Actions CI across both repos after a push. Always run in background.
+
+## Usage
+
+- `/ci` — monitor the most recent CI run on both repos for the current branch
+- `/ci pgxntool-test` — monitor pgxntool-test only
+- `/ci pgxntool` — monitor pgxntool only
+- `/ci <branch> <pgxntool-sha> <pgxntool-test-sha>` — monitor specific push SHAs (most reliable)
+
+## Workflow
+
+### 1. Start Monitor (Background)
+
+After every `git push`, immediately launch:
+
+```bash
+bash .claude/skills/ci/scripts/monitor-ci.sh [repos] [branch] [sha1] [sha2]
+```
+
+Arguments:
+- `repos`: `both` (default), `pgxntool-test`, or `pgxntool`
+- `branch`: the branch just pushed (default: current git branch)
+- `sha1`: SHA pushed to pgxntool-test (optional but recommended)
+- `sha2`: SHA pushed to pgxntool (optional but recommended)
+
+When pushing to both repos, always pass the SHAs to avoid a race condition where
+`--branch` might pick up a different concurrent push on the same branch.
+
+> **Race condition note**: `gh run list --branch` returns the most recent run on
+> that branch — if two pushes happen close together (e.g. two sessions pushing
+> in parallel), it may pick up the wrong run. Passing `--commit SHA` targets the
+> exact push and avoids this. When SHA is unavailable, always verify the
+> `=== BRANCHES: ===` line in the output matches the code you pushed.
+
+**Always use `run_in_background: true`.**
+
+### 2. Read Results
+
+When the background task completes, read the output. The script emits:
+
+```
+[pgxntool-test] Run 12345678 found
+[pgxntool-test] === BRANCHES: pgxntool-test=feature/foo pgxntool=feature/foo ===
+[pgxntool-test] Polling... (running: 🐘 PostgreSQL 13, 🐘 PostgreSQL 15)
+[pgxntool-test] PASS  🐘 PostgreSQL 12
+[pgxntool-test] PASS  🐘 PostgreSQL 15
+[pgxntool-test] FAIL  🐘 PostgreSQL 13
+[pgxntool-test] Run completed: FAILURE
+[pgxntool-test] === FAILURE: 🐘 PostgreSQL 13 ===
+... failure log lines ...
+OVERALL: FAIL
+```
+
+The **last line is always `OVERALL: <STATUS>`**. Check this first:
+
+| OVERALL | Exit code | Meaning |
+|---------|-----------|---------|
+| `ALL_PASS` | 0 | All jobs green — safe to proceed |
+| `FAIL` | 1 | One or more jobs failed — stop and report |
+| `TIMEOUT` | 2 | Run(s) did not complete within timeout |
+
+**Always verify the `=== BRANCHES ===` line** matches the code you just pushed —
+this is your primary safeguard against the `--branch` race condition. If the
+branches don't match, cancel the run and re-trigger: `gh run cancel <id> --repo
+<repo>` then re-push or re-run via `gh run rerun`.
+
+### 3. Enforce Results
+
+**CRITICAL RULES:**
+
+1. Any CI failure must be **reported to the user immediately**. Do not continue with other work.
+2. Start diagnosis from the **first** `not ok` line to understand the root cause, but do not assume later failures are cascading or caused by it — treat each failure as likely real and needing its own investigation. Failures in separate test files are typically unrelated; even multiple failures within the same file may be independent.
+3. Failures in our workflow files (dependency installs, git config, etc.) are our problem to fix.
+4. Failures in test code (not ok from BATS) may be pre-existing — report to user and ask before touching test files.
+5. Never rationalize failures as "pre-existing" or "unrelated" without explicitly telling the user.
+6. If CI is taking longer than expected on pgxntool, it may be waiting up to 20 min for pgxntool-test CI to complete — that is normal.
+
+## Key rules
+
+1. **ALWAYS** monitor CI after every push — use this skill, never `gh run watch` directly
+2. When pushing to both repos, start two background monitors simultaneously (one per repo)
+3. Pass the exact push SHA when available — `--branch` has a race condition on rapid pushes
+4. The `=== BRANCHES ===` line in the output confirms which code is under test — always verify it matches your intent

.claude/skills/ci/scripts/monitor-ci.sh

@@ -0,0 +1,221 @@
+#!/usr/bin/env bash
+# monitor-ci.sh [repos] [branch] [sha_pgxntool_test] [sha_pgxntool]
+#
+# Monitor GitHub Actions CI runs for pgxntool-test and/or pgxntool.
+# Designed to be run in background by Claude after every git push.
+#
+# Arguments:
+#   repos            : "both" (default), "pgxntool-test", or "pgxntool"
+#   branch           : branch name (default: current git branch)
+#   sha_pgxntool_test: exact SHA pushed to pgxntool-test (optional)
+#   sha_pgxntool     : exact SHA pushed to pgxntool (optional)
+#
+# Exit codes:
+#   0 : ALL_PASS  — all jobs succeeded
+#   1 : FAIL      — one or more jobs failed
+#   2 : TIMEOUT   — run(s) did not complete within the timeout
+#   3 : NO_RUNS   — no CI run found for this branch after waiting
+#
+# Requires: gh CLI authenticated with repo access.
+
+set -euo pipefail
+
+REPOS="${1:-both}"
+BRANCH="${2:-$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "")}"
+SHA_TEST="${3:-}"
+SHA_PGXN="${4:-}"
+
+# Derive owner from the current repo (works for forks too)
+_current_repo=$(gh repo view --json nameWithOwner -q .nameWithOwner 2>/dev/null || true)
+_owner=$(echo "$_current_repo" | cut -d/ -f1)
+if [[ -z "$_owner" ]]; then
+  # fallback if gh can't determine the repo
+  _owner="Postgres-Extensions"
+fi
+REPO_TEST="${_owner}/pgxntool-test"
+REPO_PGXN="${_owner}/pgxntool"
+
+# pgxntool CI can wait up to 20 min for pgxntool-test CI to complete, then
+# runs tests itself (commit-with-no-tests case). Allow 35 min total.
+# pgxntool-test runs typically take 5-10 min (resolve + 6 PG matrix jobs).
+TIMEOUT_TEST=900    # 15 minutes
+TIMEOUT_PGXN=2100   # 35 minutes
+POLL_INTERVAL=10    # seconds between status polls
+
+# ─── Helper: wait for a run to appear, then poll until done ──────────────────
+monitor_one() {
+  local repo="$1"
+  local branch="$2"
+  local sha="$3"
+  local timeout="$4"
+  local label="[$repo]"
+  local elapsed=0
+
+  # Step 1: find the run ID.
+  # When a SHA is provided, wait up to 30s for GitHub to index that exact run
+  # before falling back to the branch lookup. Without this wait, rapid pushes
+  # cause the branch fallback to pick up the previous run instead of the new one.
+  local run_id=""
+  local sha_wait=0
+  local SHA_INDEX_WAIT=30  # seconds to wait for SHA indexing before branch fallback
+  echo "$label Waiting for CI run on branch '$branch'..."
+  while [[ -z "$run_id" ]]; do
+    if [[ -n "$sha" ]]; then
+      run_id=$(gh run list --repo "$repo" --commit "$sha" \
+        --json databaseId --jq '.[0].databaseId // empty' 2>/dev/null || true)
+    fi
+    if [[ -z "$run_id" && -n "$branch" && ( -z "$sha" || $sha_wait -ge $SHA_INDEX_WAIT ) ]]; then
+      # Only fall back to branch once the SHA wait window has elapsed (or no SHA given).
+      # NOTE: this can pick up a different run if two pushes happen rapidly.
+      run_id=$(gh run list --repo "$repo" --branch "$branch" \
+        --event pull_request --limit 1 \
+        --json databaseId --jq '.[0].databaseId // empty' 2>/dev/null || true)
+    fi
+    if [[ -z "$run_id" ]]; then
+      sleep 5
+      elapsed=$((elapsed + 5))
+      sha_wait=$((sha_wait + 5))
+      if [[ $elapsed -ge $timeout ]]; then
+        echo "$label ERROR: no CI run found after ${timeout}s" >&2
+        return 1
+      fi
+    fi
+  done
+  echo "$label Run $run_id found"
+
+  # Step 2: extract the BRANCHES line as soon as the first job starts.
+  # We use the direct jobs API (fast ~1s) rather than the zip-download log path
+  # (slow 3-10s). We only need one job — all jobs emit the same BRANCHES line.
+  local branches_line=""
+  local attempts=0
+  while [[ -z "$branches_line" && $elapsed -lt $timeout ]]; do
+    local first_job_id
+    first_job_id=$(gh run view "$run_id" --repo "$repo" \
+      --json jobs --jq '[.jobs[].databaseId][0] // empty' 2>/dev/null || true)
+
+    if [[ -n "$first_job_id" ]]; then
+      # grep may return non-zero if the line isn't present yet — that's fine.
+      branches_line=$(gh api "repos/${repo}/actions/jobs/${first_job_id}/logs" \
+        2>/dev/null | grep "^=== BRANCHES:" | tail -1 || true)
+    fi
+
+    if [[ -z "$branches_line" ]]; then
+      attempts=$((attempts + 1))
+      if [[ $attempts -ge 3 ]]; then
+        # Give up waiting for the BRANCHES line and move on to polling.
+        echo "$label (BRANCHES line not yet available; proceeding to poll)"
+        break
+      fi
+      sleep "$POLL_INTERVAL"
+      elapsed=$((elapsed + POLL_INTERVAL))
+    fi
+  done
+  if [[ -n "$branches_line" ]]; then
+    echo "$label $branches_line"
+  fi
+
+  # Step 3: poll until all jobs complete.
+  local status="in_progress"
+  local result=""
+  while [[ "$status" != "completed" && $elapsed -lt $timeout ]]; do
+    result=$(gh run view "$run_id" --repo "$repo" \
+      --json status,conclusion,jobs \
+      --jq '{status: .status, conclusion: .conclusion,
+             jobs: [.jobs[] | {name: .name, status: .status, conclusion: .conclusion}]}' \
+      2>/dev/null || true)
+
+    if [[ -z "$result" ]]; then
+      sleep "$POLL_INTERVAL"
+      elapsed=$((elapsed + POLL_INTERVAL))
+      continue
+    fi
+
+    status=$(echo "$result" | jq -r '.status')
+
+    if [[ "$status" != "completed" ]]; then
+      local running
+      running=$(echo "$result" | jq -r \
+        '[.jobs[] | select(.status == "in_progress") | .name] | join(", ")' || true)
+      if [[ -n "$running" ]]; then
+        echo "$label Polling... (running: $running)"
+      fi
+      sleep "$POLL_INTERVAL"
+      elapsed=$((elapsed + POLL_INTERVAL))
+    fi
+  done
+
+  if [[ $elapsed -ge $timeout ]]; then
+    echo "$label ERROR: timed out after ${timeout}s" >&2
+    return 2
+  fi
+
+  # Step 4: report per-job outcomes.
+  local conclusion
+  conclusion=$(echo "$result" | jq -r '.conclusion')
+  echo "$label Run $run_id completed: $(echo "$conclusion" | tr '[:lower:]' '[:upper:]')"
+  echo "$result" | jq -r '.jobs[] | "\(if .conclusion == "success" then "PASS" elif .conclusion == null then .status else .conclusion | ascii_upcase end)  \(.name)"' \
+    | sed "s|^|$label |"
+
+  # Step 5: for failed jobs, print the failure log (last 60 lines per job).
+  if [[ "$conclusion" != "success" ]]; then
+    local failed_job_ids
+    failed_job_ids=$(gh run view "$run_id" --repo "$repo" \
+      --json jobs \
+      --jq '[.jobs[] | select(.conclusion == "failure") | .databaseId] | .[]' \
+      2>/dev/null || true)
+
+    for job_id in $failed_job_ids; do
+      local job_name
+      job_name=$(gh run view "$run_id" --repo "$repo" \
+        --json jobs \
+        --jq --argjson id "$job_id" \
+        '[.jobs[] | select(.databaseId == $id) | .name] | .[0]' 2>/dev/null || true)
+      echo ""
+      echo "$label === FAILURE: ${job_name:-job $job_id} ==="
+      # Use --log-failed to get only the failed step output, keeping output compact.
+      gh run view --repo "$repo" --job "$job_id" --log-failed 2>&1 \
+        | grep -v "^$" | tail -60 || true
+    done
+
+    return 1
+  fi
+
+  return 0
+}
+
+# ─── Main: run monitors in parallel or series ─────────────────────────────────
+exit_code=0
+pid_test=""
+pid_pgxn=""
+
+case "$REPOS" in
+  pgxntool-test)
+    monitor_one "$REPO_TEST" "$BRANCH" "$SHA_TEST" "$TIMEOUT_TEST" || exit_code=1
+    ;;
+  pgxntool)
+    monitor_one "$REPO_PGXN" "$BRANCH" "$SHA_PGXN" "$TIMEOUT_PGXN" || exit_code=1
+    ;;
+  both|*)
+    # Run both in parallel. Each writes to stdout (interleaved but prefixed with
+    # the repo name for readability). Capture both PIDs and wait for both.
+    monitor_one "$REPO_TEST" "$BRANCH" "$SHA_TEST" "$TIMEOUT_TEST" &
+    pid_test=$!
+    monitor_one "$REPO_PGXN" "$BRANCH" "$SHA_PGXN" "$TIMEOUT_PGXN" &
+    pid_pgxn=$!
+
+    wait "$pid_test" || { r=$?; echo "[both] pgxntool-test CI FAILED"; [[ $r -gt $exit_code ]] && exit_code=$r; }
+    wait "$pid_pgxn" || { r=$?; echo "[both] pgxntool CI FAILED";      [[ $r -gt $exit_code ]] && exit_code=$r; }
+    ;;
+esac
+
+# Emit a parseable summary line. Claude should check this line rather than
+# parsing the full output. Convention matches the test skill's STATUS line.
+if [[ $exit_code -eq 0 ]]; then
+  echo "OVERALL: ALL_PASS"
+elif [[ $exit_code -eq 2 ]]; then
+  echo "OVERALL: TIMEOUT"
+else
+  echo "OVERALL: FAIL"
+fi
+
+exit $exit_code

.github/workflows/CLAUDE.md

@@ -0,0 +1,57 @@
+# .github/workflows — CI Architecture
+
+## Workflow files
+
+- **`ci.yml`** — main CI for pgxntool-test pull requests. Runs a `resolve` job
+  (determines which pgxntool branch to test against), then calls `run-tests.yml`.
+- **`run-tests.yml`** — reusable workflow (`workflow_call`). Single source of truth
+  for all test steps (PostgreSQL matrix, checkouts, git config, pgtap install, etc.).
+  Called by both `ci.yml` here and by `pgxntool/ci.yml` for the commit-with-no-tests path.
+
+## Cross-repo reusable workflow — tradeoffs and constraints
+
+`run-tests.yml` is referenced from **pgxntool** as:
+```yaml
+uses: Postgres-Extensions/pgxntool-test/.github/workflows/run-tests.yml@<ref>
+```
+
+GitHub Actions requires the `uses:` ref to be a **static string** — expressions are
+not supported. This creates an unavoidable structural constraint:
+
+### Merge order requirement
+
+**pgxntool-test MUST be merged before pgxntool** whenever both repos change in the
+same feature branch. Here's why:
+
+- pgxntool's `ci.yml` pins to `run-tests.yml@master`
+- While developing on a branch, pgxntool's `ci.yml` temporarily uses `@<branch>`
+- When it's time to merge, pgxntool-test must land on master first so that
+  `run-tests.yml@master` exists before pgxntool's CI tries to use it
+
+### What this means for `run-tests.yml` changes
+
+- Changes to `run-tests.yml` are always tested through **pgxntool-test's own CI**
+  (which uses `./.github/workflows/run-tests.yml` — a local ref that always sees
+  the current branch version).
+- pgxntool's CI (commit-with-no-tests path) uses `run-tests.yml@master`. Until
+  pgxntool-test merges, pgxntool CI will use the old master version.
+- These two scenarios are mutually exclusive in practice: the commit-with-no-tests
+  path only runs when there is NO paired test PR. If you're changing `run-tests.yml`,
+  you have a paired test PR, so pgxntool's test job is skipped anyway.
+
+### The @branch → @master transition
+
+While developing on a feature branch, pgxntool's `ci.yml` uses `@<branch>` so CI
+can find `run-tests.yml` before it lands on master. Before pgxntool can be merged,
+pgxntool-test must merge first and then pgxntool's ref must be updated to `@master`.
+
+**For Claude**: This transition is NOT automatic. You must get explicit user approval
+before leaving a `@<branch>` ref in pgxntool's `ci.yml`. Do not assume this will be
+handled at merge time — the user merges directly from the PR page with no manual steps.
+The safe approach is to coordinate the merge order explicitly with the user.
+
+## Expanding the matrix
+
+The PostgreSQL version matrix (`pg: [17, 16, 15, 14, 13, 12]`) is hardcoded in
+`run-tests.yml`. GitHub Actions does not support passing a matrix as a workflow_call
+input. To add or remove a PG version, edit `run-tests.yml` directly.