feat(llm): openai-cli backend — semantic extraction through the locally authenticated Codex CLI

CHANGELOG.md

@@ -2,6 +2,14 @@
 
 Full release notes with details on each version: [GitHub Releases](https://github.com/safishamsi/graphify/releases)
 
+## Unreleased
+
+- Feature: a new `openai-cli` backend (`--backend openai-cli`) runs semantic extraction and community labeling through the locally authenticated OpenAI Codex CLI (`codex exec`), so the work rides a ChatGPT OAuth subscription instead of a metered `OPENAI_API_KEY` — removing it would silently fall extraction back to metered API spend. The prompt travels via stdin (Linux caps a single argv entry at 128 KB; real chunks reach 240–306 KB), the sandbox is read-only so an agentic Codex cannot write to the corpus it reads, every configured MCP server is disabled per call through Codex's own per-server `enabled` override (a blanket `mcp_servers={}` deep-merges away; the measured cost was four ~152 MB servers per `codex exec`), token usage is read from the last `turn.completed` JSONL event without double-counting cached input, an empty-but-valid graph raises instead of triggering the hollow-response bisect (which once burned 87% of an hourly quota), and calls are forced serial unless `GRAPHIFY_OPENAI_CLI_PARALLEL=1`. `GRAPHIFY_OPENAI_CLI_MODEL` (default `gpt-5.6-sol`) and `GRAPHIFY_OPENAI_CLI_EFFORT` (default `ultra`) configure it; the credential gate accepts a present `codex` CLI in place of an API key, mirroring `claude-cli`.
+- Feature: `graphify extract` now holds the same per-repo rebuild lock the watcher and git-hook rebuilds take, for its whole pipeline, so two extracts (or an extract racing `graphify update`) on one `graphify-out/` serialize instead of interleaving cache saves and clobbering `graph.json`; a contended run names the holder's PID, waits up to `GRAPHIFY_LOCK_TIMEOUT` seconds (default 600), and exits with an error instead of hanging forever behind a wedged rebuild. The lock file's PID payload and unlink-on-release contracts are unchanged.
+- Feature: `graphify extract --fallback-backend <B>` (or `GRAPHIFY_FALLBACK_BACKEND`; the flag wins) retries the semantic pass once on a second backend when every chunk fails on the primary, so a missing SDK package, a bad key, or an outage no longer costs the whole build; the retry covers exactly the still-uncached files, `--model` stays with the primary backend (the fallback runs on its own default model), a typo'd fallback name is rejected before any API spend, and only a zero-success retry keeps the all-chunks-failed exit 1.
+- Feature: `graphify watch --semantic` runs LLM-backed semantic extraction automatically when doc/paper/image files change, instead of only writing the `needs_update` flag; the extract runs as a subprocess so it serializes on the per-repo rebuild lock (in-process re-entry would self-deadlock on `flock`), a failed extract still falls back to the flag + `/graphify --update` instruction, and a successful run clears the flag so no stale prompt is left behind. `--backend`/`--fallback-backend` are forwarded to the extract and rejected without `--semantic` rather than being a silent no-op.
+- Feature: `graphify export neo4j|falkordb --push` now sends nodes and edges in UNWIND batches (`--batch-size`, default 100) instead of one query per entry, so a remote push stops spending nearly all its time on round trips (~100x fewer for the default); rows are grouped by sanitized node label / relationship type first (those are baked into the Cypher text and cannot be parameters), the row payloads are exactly the old per-entry params, and UNWIND processes rows in order, so the MERGE/SET upsert semantics — including idempotent re-runs — are unchanged.
+
 ## 0.9.48 (2026-08-20)
 
 - Fix: a control character in a node label or id no longer aborts the whole export; the GraphML and Obsidian exporters scrub only the characters those formats forbid (tab, newline, and non-ASCII letters are preserved), and `graph.json` and its byte-identity round-trip are untouched (#2897, thanks @abhay-codes07).

graphify/__main__.py

@@ -534,6 +534,9 @@ def _run_cli() -> None:
         print("    --contributor \"Name\"    tag who added it to the corpus")
         print("    --dir <path>            target directory (default: ./raw)")
         print("  watch <path>            watch a folder and rebuild the graph on code changes")
+        print("    --semantic              also run LLM extraction automatically on doc/image changes")
+        print("    --backend <name>        extraction backend for --semantic")
+        print("    --fallback-backend <name>  fallback backend for --semantic")
         print("  update <path>           re-extract code files and update the graph (no LLM needed)")
         print("    --force                 overwrite graph.json even if the rebuild has fewer nodes")
         print("                            (also: GRAPHIFY_FORCE=1 env var; use after refactors that delete code)")
@@ -544,13 +547,14 @@ def _run_cli() -> None:
         print("    --no-label              keep 'Community N' placeholders (skip LLM community naming)")
         print("    --backend=<name>        backend to use for community naming (default: auto-detect)")
         print("    --model=<name>          model to use for community naming")
-        print("    --max-concurrency=N     parallel community-labeling LLM calls (default 4; forced to 1 for ollama/claude-cli)")
+        # PATCHED FOR TELB-COCKPIT: openai-cli is also forced serial in llm.py.
+        print("    --max-concurrency=N     parallel community-labeling LLM calls (default 4; forced to 1 for ollama/claude-cli/openai-cli)")
         print("    --batch-size=N          communities per labeling LLM call (default 100)")
         print("  label <path>            (re)name communities with the configured LLM backend, regenerate report")
         print("    --missing-only         keep existing labels and only name missing/placeholder communities")
         print("    --backend=<name>        backend to use (default: auto-detect from API keys)")
         print("    --model=<name>          model to use for community naming")
-        print("    --max-concurrency=N     parallel labeling LLM calls (default 4; forced to 1 for ollama/claude-cli)")
+        print("    --max-concurrency=N     parallel labeling LLM calls (default 4; forced to 1 for ollama/claude-cli/openai-cli)")
         print("    --batch-size=N          communities per labeling LLM call (default 100)")
         print("  query \"<question>\"       BFS traversal of graph.json for a question")
         print("    --dfs                   use depth-first instead of breadth-first")