Skip to content

feat(query): read-side multi-source — connectable pool, provenance, fail-loud client#68

Merged
colombod merged 1 commit into
mainfrom
feat/context-intelligence-read-side-multi-source
Jul 13, 2026
Merged

feat(query): read-side multi-source — connectable pool, provenance, fail-loud client#68
colombod merged 1 commit into
mainfrom
feat/context-intelligence-read-side-multi-source

Conversation

@colombod

Copy link
Copy Markdown
Collaborator

Summary

Delivers read-side multi-source support for the context-intelligence query tools (graph_query, blob_read): the graph-analyst can now select and connect to any endpoint in a merged connectable set (configured sources ∪ hook destinations), every result is provenance-tagged so the user always sees which source answered, and the client fails loud instead of silently swallowing failures.

This builds on the single-source selection foundation (fail-loud on unknown/ambiguous source) originally authored by @bkrabach in #67 — that commit (3287df4) is included here as the base. This PR carries the complete, proven, validated capability as one self-contained unit.

Hard scope guardrail: read-side only. The hook's write-side fan-out (logging / dispatch / fanout.py / _DestinationDispatcher) is untouched — verified zero hook fan-out files changed. Result fan-in (query-many + merge) is deliberately parked (cross-source data duplication risk); every query is single-hit and the default never blasts out.

What's delivered

Selection (base, from #67) — optional source=<name> selector; fail-loud on unknown source (enumerates valid names) and on ambiguity (2+ sources, no pointer); per-entry source validation.

Phase 0 — fail-loud clientAsyncCIClient.cypher() / fetch_blob() / list_blob_keys() now raise a classified CIClientError (connection_error | timeout | http_status | decode_error) instead of swallowing failures to [] / None / set(), plus a real per-source request_timeout (default 30s; non-positive → 0.1s floor). A down / 500 / hung source surfaces as success=False — never a silent empty result. Genuine empty-200 still succeeds. reconstruct/ callers surface failures instead of silent partial reconstruction.

Connectable set — merged pool = tool sources ∪ hook destinations. Explicit source=<name> selects across the whole pool (you can name a destination too); list_sources: true returns {"connectable_set":[{name,url,origin}]} (no api_key); source-wins on name collision. Default resolution: 1 source → it; 2+ sources + no pointer → fail-loud; 0 sources → first destination (established fallback, no fail-loud). Resolver consolidated into resolve_query_connection.

Source provenance — visible to the usergraph_query success returns {"source":{name,url,origin},"rows":[…]}; blob_read success adds a source block; failures carry source once an endpoint is resolved. The graph-analyst agent + SKILLs now state which source produced the answer.

  • Contract note: graph_query output changed from bare rows → {source, rows} to surface provenance. The tool is only visible to agents inside this bundle (no external consumers), and every in-repo consumer already speaks the new shape — no external contract to break.

list_blob_keys correctness (found via live testing) — parses the server's {"blobs":[…]} envelope and returns bare keys (was silently returning empty against the real server, and would have double-prefixed ci-blob:// on fetch_blob).

Docs / diagrams / toolingconfig-resolution.dot/.png rewritten (it is @-mentioned into the graph-analyst agent); README read-path contract section; corrected example return-shapes; ruff + pyright (1.1.411) dev deps; repo-wide uv.lock refresh (every package passes uv lock --check); regenerated bundle.dot/bundle.png.

Verification — end-to-end, real

Unit / integration (green): module tool-context-intelligence-query 192 passed, top-level 753 passed; ruff check + ruff format --check + pyright (1.1.411) clean. uv build succeeds for all 4 packages; uv lock --check clean repo-wide. Real end-to-end tests use live ThreadingHTTPServer sockets (not mocks).

Full bundle validation (scripts/validate-full.shvalidation_mode: full): PASS — all 10 bundles load via BundleRegistry as good; packaging, hygiene, context-sink, tool-placement, YAML-lint all pass. The lone mode ERROR is a documented false positive (name collision), confirmed against source per the repo's AGENTS.md.

Live DTU runs against production endpoints (read-only). Two distinct real DBs — private-home-server (1,072,740 nodes) and team-shared (541,677 nodes). Provenance verifiably names the endpoint that actually answered (node counts track the named source, never cross):

Config shape Proven live
Sources only default → source origin; explicit source; 2+ sources + no pointer → fail-loud (no query sent); list_sources all origin=source; no api_key
Merged (source + destination) list_sources = union (both origins); explicit source and explicit destination both reach real servers; default → source wins; collision → source wins
Destination only default → first destination; unreachable → connection_error fail-loud (not silent empty)
  • blob_read success + provenance — proven live: a real 476 KB server-minted blob fetched via BlobReadTool.execute(), provenance for both source and destination origins; 404 + down-endpoint fail-loud carry provenance.
  • list_blob_keysfetch_blob chain — proven live: 163 bare keys (sync == async); fetch_blob(session, bare_key) composes a clean /blobs/{sid}/{key} path (no doubled scheme); reconstruct metadata enrichment resolves live.

Review: six-lens /council ×2 (both surfaced blockers fixed) + a dedicated restless-old-brian ship-gate (GO). Two live-only bugs were caught by the live runs and fixed, each re-proven live.

Relationship to #67

This PR incorporates and completes the work in #67 (@bkrabach's selection commit is the base here). It exists as a separate PR so the full read-side capability can be reviewed and merged as one proven, self-contained unit. Coordinate with @bkrabach on whether to land this and close #67, or land #67 first.

Non-blocking follow-ups

  • Prove the in-tool DefaultAzureCredential acquisition path in a DTU with a managed identity (the real Azure APIM+graph round-trip was proven via a host-acquired bearer).
  • Malformed-but-HTTP-200 body still treated as empty-success (inherited from the selection base); every real transport/status/decode failure now fails loud.

Clears the BUNDLE_DOT_STALE warning from the full bundle validation
(scripts/validate-full.sh → validation_mode: full, overall PASS).

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@colombod colombod force-pushed the feat/context-intelligence-read-side-multi-source branch from b5eaccc to 2855ee1 Compare July 13, 2026 16:36
@colombod colombod merged commit c94c5d3 into main Jul 13, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant