feat(examples,adapters,docs): code-review bot + voice agent architectures + FastMCP CodeMode hooks (#87, #204, #205)#233
Merged
Conversation
…ures + FastMCP CodeMode hooks (#87, #204, #205) Lands the 3-issue "adoption demonstrations" group selected by the triage pass. Single combined PR (Mode B, owner-authorised). Same blast radius: two new reference architectures under examples/architectures/ + two new adapter callable factories in adapters/fastmcp.py + paired docs and tests. Zero changes to the context or routing core pipelines. #87 — FastMCP CodeMode discovery / context hooks - adapters/fastmcp.py grows two factories that return plain Callable[[str], list[dict]] and Callable[[str, str], str] respectively, matching the FastMCP CodeMode hook contract (PrefectHQ/fastmcp#3365) but framework- agnostic — neither captures any FastMCP reference at runtime. - make_discovery_tool(router, catalog, *, top_k=None) wraps Router + Catalog into a "given a query, return a shortlist of tools" callable; graph-only nodes are skipped silently. - make_context_hook(context_manager, *, firewall_threshold=2000) wraps the firewall as a "(query, raw_result) → summary" callable; raw bytes park in the artifact store, query is recorded as item.metadata for trace correlation. - examples/fastmcp_discovery_demo.py demos a 22-tool catalog → 3-tool shortlist with ~86% token reduction. - 12 unit tests under tests/test_adapters.py + 2 real-FastMCP integration tests in tests/test_adapters_fastmcp_discovery.py (spins up fastmcp.FastMCP in-memory server, round-trips through the hook). - fastmcp>=2.0 moved from [fastmcp] runtime extra into [dev] so the integration test runs on every CI matrix cell. #204 — Code-review bot reference architecture - examples/architectures/code_review_bot/ with main.py + catalog.yaml (24 tools across grep/git/lint/typecheck/test/review) + README.md + OUTPUT.md + __init__.py. - Six-step PR review walking a regression in payments/charge.py. The firewall is the load-bearing pattern: synthetic ~28 KB diff dump and ~2.5 KB grep result both compact to ~500-char summaries while raw bytes stay addressable. - 10 smoke tests in tests/test_architectures_code_review.py pin deterministic invariants (catalog size, intent matches, firewall fires=2/6, artifact count=6, fact keys). - docs/architectures/code_review_bot.md is the public docs page. #205 — Voice agent reference architecture (Pipecat) - examples/architectures/voice_agent/ with main.py + catalog.yaml (18 tools across support/orders/shipping/account/callback) + README.md + OUTPUT.md + __init__.py. - Canonical worked example for docs/integration_pipecat.md: every context build runs via asyncio.to_thread(mgr.build_sync, ...); ContextBudget(route=200, call=500, interpret=400, answer=1000) enforces sub-300 ms TTS-friendly answer prompts (max 200 tokens at five turns). - Pipecat optional: example runs end-to-end without pipecat-ai. pyproject.toml grows a [voice] extra (pipecat-ai>=0.0.50) for users who want the real FrameProcessor. - 10 smoke tests in tests/test_architectures_voice.py pin the async-build marker, intent matches, fact keys, and the 400-token answer-prompt ceiling. Wall-clock timings are not asserted on. - docs/architectures/voice_agent.md is the public docs page. - docs/integration_pipecat.md gains a "Canonical worked example" callout pointing back at the architecture. Shared bookkeeping - Makefile architectures target runs all three architectures. - mkdocs.yml + docs/architectures/index.md link the two new pages. - CHANGELOG.md gains three bullets under [Unreleased]. - tests/test_architectures_slack.py refactored from sys.path injection to importlib.util.spec_from_file_location so the three architecture test files can coexist in one pytest run (a bare ``import main`` from sys.path collides across architectures). Module-size note adapters/fastmcp.py lands at 428 lines, over the soft 300-line guide, in line with adapters/mcp.py (401) and adapters/proxy_runtime.py (462) precedent. Mode B authorised the modest overrun rather than splitting one adapter across two files. Verification ruff format --check src/ tests/ examples/ scripts/ → 145 files clean ruff check src/ tests/ examples/ scripts/ → All checks passed mypy src/ → 0 issues / 64 files pytest --cov=contextweaver -q → 985 passed, 5 skipped (+34 new tests) make example → all 14 scripts ran make demo → clean make scorecard-check → clean (no benchmark drift) make llms-check → up to date Closes #87 Closes #204 Closes #205 https://claude.ai/code/session_01JiR8ZGtuwn7Cv2ahHMwLhL
Benchmark delta (vs
|
| size | recall@k (head Δ vs base) | MRR (head Δ vs base) | p99 (ms) |
|---|---|---|---|
| 50 | ✅ 0.5649 (+0.0000) | ✅ 0.4978 (+0.0000) | ✅ 0.450 (base 0.463) |
| 83 | ✅ 0.3825 (+0.0000) | ✅ 0.3242 (+0.0000) | ✅ 1.100 (base 0.876) |
| 1000 | ✅ 0.1475 (+0.0000) | ✅ 0.1456 (+0.0000) |
Per-backend × per-size matrix
| backend | size | recall@k (Δ) | MRR (Δ) | p99 (ms) |
|---|---|---|---|---|
| bm25 | 100 | ✅ 0.3825 (+0.0000) | ✅ 0.3399 (+0.0000) | ✅ 5.923 (base 5.642) |
| bm25 | 500 | ✅ 0.2250 (+0.0000) | ✅ 0.2165 (+0.0000) | ✅ 30.161 (base 27.538) |
| bm25 | 1000 | ✅ 0.1575 (+0.0000) | ✅ 0.1525 (+0.0000) | ✅ 85.263 (base 78.368) |
| fuzzy | 100 | ✅ 0.0000 (+0.0000) | ✅ 0.0000 (+0.0000) | ✅ 0.000 (base 0.000) |
| fuzzy | 500 | ✅ 0.0000 (+0.0000) | ✅ 0.0000 (+0.0000) | ✅ 0.000 (base 0.000) |
| fuzzy | 1000 | ✅ 0.0000 (+0.0000) | ✅ 0.0000 (+0.0000) | ✅ 0.000 (base 0.000) |
| tfidf | 100 | ✅ 0.3825 (+0.0000) | ✅ 0.3220 (+0.0000) | ✅ 1.007 (base 0.872) |
| tfidf | 500 | ✅ 0.2325 (+0.0000) | ✅ 0.2314 (+0.0000) | ✅ 9.342 (base 8.660) |
| tfidf | 1000 | ✅ 0.1475 (+0.0000) | ✅ 0.1456 (+0.0000) | ✅ 33.268 (base 30.071) |
Context pipeline (per scenario)
| scenario | tokens | dropped | dedup |
|---|---|---|---|
| large_catalog | 1514 (base 1514, Δ+0) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| long_conversation | 2548 (base 2548, Δ+0) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| short_conversation | 496 (base 496, Δ+0) | 0 (base 0, Δ+0) | 0 (base 0, Δ+0) |
| stress_conversation | 6651 (base 6651, Δ+0) | 7 (base 7, Δ+0) | 4 (base 4, Δ+0) |
Numbers come from make benchmark / make benchmark-matrix.
Latency is hardware-dependent — treat the markers as a rough guide.
See benchmarks/scorecard.md for the full picture.
There was a problem hiding this comment.
Pull request overview
Adds three “adoption demonstration” deliverables to the repo: (1) FastMCP CodeMode hook factories in the FastMCP adapter, and (2) two new runnable reference architectures (code-review bot + voice agent) with smoke tests and public docs, all wired into the examples/architectures and docs navigation.
Changes:
- Add
make_discovery_tool()andmake_context_hook()factories tocontextweaver.adapters.fastmcpand re-export them fromcontextweaver.adapters. - Introduce two new reference architectures under
examples/architectures/(code_review_bot, voice_agent) with deterministic outputs, tests, and docs pages. - Wire the new examples/docs into
Makefile,mkdocs.yml,docs/architectures/index.md,pyproject.tomlextras, andCHANGELOG.md.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_architectures_voice.py | Adds smoke tests for the new voice agent architecture by running the script and asserting deterministic invariants. |
| tests/test_architectures_slack.py | Refactors Slack architecture smoke test import to use importlib.util.spec_from_file_location to avoid module-name collisions. |
| tests/test_architectures_code_review.py | Adds smoke tests for the new code-review bot architecture by running the script and asserting deterministic invariants. |
| tests/test_adapters.py | Adds unit tests for FastMCP CodeMode hook factories (make_discovery_tool, make_context_hook). |
| tests/test_adapters_fastmcp_discovery.py | Adds real FastMCP integration coverage for the discovery hook against an in-memory FastMCP server. |
| src/contextweaver/adapters/fastmcp.py | Adds CodeMode-style hook factories for tool discovery + firewall-based context compaction. |
| src/contextweaver/adapters/init.py | Re-exports the new FastMCP hook factories from the adapters package. |
| pyproject.toml | Adds fastmcp to [dev] and introduces a new [voice] optional extra for pipecat-ai. |
| mkdocs.yml | Adds the new architectures pages to the docs nav. |
| Makefile | Adds the FastMCP demo to make example and the two new architectures to make architectures. |
| examples/fastmcp_discovery_demo.py | Adds a standalone demo showing catalog → shortlist compression and firewall compaction via the new hook callables. |
| examples/architectures/voice_agent/README.md | Documents the voice agent architecture, its budgets, and the async build pattern (asyncio.to_thread). |
| examples/architectures/voice_agent/OUTPUT.md | Captured deterministic output for the voice agent architecture. |
| examples/architectures/voice_agent/main.py | Implements the voice agent reference architecture (mock tools, async builds, tight budgets, fact persistence). |
| examples/architectures/voice_agent/catalog.yaml | Provides the 18-tool catalog used by the voice agent architecture. |
| examples/architectures/voice_agent/init.py | Adds a minimal package marker for the voice agent architecture directory. |
| examples/architectures/code_review_bot/README.md | Documents the code-review bot architecture emphasizing firewall + artifact store behavior. |
| examples/architectures/code_review_bot/OUTPUT.md | Captured deterministic output for the code-review bot architecture. |
| examples/architectures/code_review_bot/main.py | Implements the code-review bot reference architecture (mock tools, firewall-heavy steps, fact persistence). |
| examples/architectures/code_review_bot/catalog.yaml | Provides the 24-tool catalog used by the code-review bot architecture. |
| examples/architectures/code_review_bot/init.py | Adds a minimal package marker for the code-review bot architecture directory. |
| docs/integration_pipecat.md | Adds a callout pointing readers to the voice agent as the canonical worked example. |
| docs/architectures/voice_agent.md | Adds the public docs page for the voice agent architecture. |
| docs/architectures/index.md | Updates the architectures index to list Slack ops bot, code-review bot, and voice agent. |
| docs/architectures/code_review_bot.md | Adds the public docs page for the code-review bot architecture. |
| CHANGELOG.md | Documents the additions (FastMCP hooks + two architectures) under Unreleased. |
Comments suppressed due to low confidence (2)
src/contextweaver/adapters/fastmcp.py:410
- The factories raise bare ValueError for invalid firewall_threshold, but AGENTS.md requires using contextweaver.exceptions for errors (no bare ValueError/RuntimeError). Consider raising ConfigError (or another ContextWeaverError subclass) instead and updating the corresponding tests.
if firewall_threshold < 0:
raise ValueError(f"firewall_threshold must be >= 0, got {firewall_threshold}")
tests/test_adapters.py:1347
- These tests assert ValueError for invalid firewall_threshold, but AGENTS.md establishes a repo convention to raise ContextWeaverError subclasses (no bare ValueError/RuntimeError). Once make_context_hook switches to ConfigError (or similar), update this expectation accordingly.
def test_context_hook_negative_threshold_raises() -> None:
"""Negative thresholds are rejected at factory time."""
mgr = ContextManager()
with pytest.raises(ValueError, match="firewall_threshold must be >= 0"):
make_context_hook(mgr, firewall_threshold=-1)
Comment on lines
+348
to
+350
| if top_k is not None and top_k < 0: | ||
| raise ValueError(f"top_k must be >= 0, got {top_k}") | ||
|
|
Owner
Author
There was a problem hiding this comment.
Good catch — fixed in fc95058. Both make_discovery_tool (negative top_k) and make_context_hook (negative firewall_threshold) now raise contextweaver.exceptions.ConfigError instead of bare ValueError, matching the AGENTS.md custom-exception convention. The matching test assertions on tests/test_adapters.py:1260, 1345 were updated to pytest.raises(ConfigError, …). CHANGELOG.md has a Fixed entry under Unreleased.
Generated by Claude Code
Comment on lines
+351
to
+366
| def discover(query: str) -> list[dict[str, Any]]: | ||
| result = router.route(query) | ||
| ids = result.candidate_ids | ||
| if top_k is not None: | ||
| ids = ids[:top_k] | ||
| out: list[dict[str, Any]] = [] | ||
| for tid in ids: | ||
| try: | ||
| hydrated = catalog.hydrate(tid) | ||
| except (ItemNotFoundError, CatalogError): | ||
| # Candidate id not in catalog (graph-only node, e.g. category | ||
| # label). Skip rather than fail — the discovery hook must | ||
| # never reject a valid query just because one node is | ||
| # virtual. | ||
| continue | ||
| out.append( |
Owner
Author
There was a problem hiding this comment.
Agreed — real boundary bug. Fixed in fc95058. discover() now iterates the full result.candidate_ids, skips non-hydratable IDs, and stops after appending top_k real tools. A graph-only candidate appearing early in the list no longer eats a slot in the shortlist. Added a regression test test_discovery_tool_top_k_skips_graph_only_without_eating_slots in tests/test_adapters.py that prepends a synthetic virtual node and asserts the shortlist still has top_k=2 real entries.
Generated by Claude Code
Comment on lines
+367
to
+372
| { | ||
| "name": hydrated.item.name, | ||
| "description": hydrated.item.description, | ||
| "input_schema": dict(hydrated.args_schema), | ||
| } | ||
| ) |
Owner
Author
There was a problem hiding this comment.
You're right — dict(hydrated.args_schema) only protected the top level. Fixed in fc95058 by switching to copy.deepcopy(hydrated.args_schema) so nested dicts / lists stay isolated from catalog state, with a short comment explaining the Catalog.hydrate sharing invariant. Extended the existing copy test (renamed to test_discovery_tool_input_schema_is_deep_copy) to also mutate every reachable nested dict / list and assert the next discover() call sees an unmutated schema.
Generated by Claude Code
Comment on lines
+400
to
+403
| A pure callable ``(query, raw_result) -> str``. *query* is recorded | ||
| as the parent user-turn id for dependency closure; *raw_result* is | ||
| the verbatim tool output. Returns the firewall summary (or the raw | ||
| result if below threshold). |
Owner
Author
There was a problem hiding this comment.
Docstring was indeed lying about behavior — fixed in fc95058 by updating the wording rather than changing the implementation. The hook is intentionally stateless w.r.t. conversation history (see the inline rationale at the setdefault call): query is stamped onto item.metadata["codemode_query"] for trace correlation, and no synthetic user_turn ContextItem is ingested. Synthesising one just for parent_id linkage would inflate the event log on every CodeMode call, so the doc was the side that needed to give. CHANGELOG Unreleased note added.
Generated by Claude Code
Comment on lines
+21
to
+24
| # Module-level importorskip: when the optional ``fastmcp`` extra is not | ||
| # installed (matrix runners that only pull ``[dev]``), skip this entire | ||
| # file rather than fail at collection. The unit tests in test_adapters.py | ||
| # still run, so coverage of the adapter does not regress. |
Owner
Author
There was a problem hiding this comment.
Reworded in fc95058. The skip comment now reflects that fastmcp>=2.0 is part of [dev] so every CI matrix cell runs this file; the importorskip only guards users running tests against a minimal-runtime install where neither [dev] nor [fastmcp] was pulled.
Generated by Claude Code
Comment on lines
+1256
to
+1262
| def test_discovery_tool_negative_top_k_raises() -> None: | ||
| """Negative ceilings are rejected at factory time, not at call time.""" | ||
| catalog = _build_test_catalog() | ||
| router = _build_test_router(catalog) | ||
| with pytest.raises(ValueError, match="top_k must be >= 0"): | ||
| make_discovery_tool(router, catalog, top_k=-1) | ||
|
|
Owner
Author
There was a problem hiding this comment.
Updated in fc95058. Both test_discovery_tool_negative_top_k_raises (line 1260) and test_context_hook_negative_threshold_raises (line 1345) now assert pytest.raises(ConfigError, …), matching the factory-side switch to contextweaver.exceptions.ConfigError. ConfigError is added to the contextweaver.exceptions import at the top of the test file.
Generated by Claude Code
# Conflicts: # CHANGELOG.md
- Raise contextweaver.exceptions.ConfigError (not bare ValueError) from make_discovery_tool and make_context_hook on negative top_k / firewall_threshold, per the AGENTS.md custom-exception convention. Tests updated. - Fix make_discovery_tool top_k boundary bug: the candidate-id list is no longer sliced before hydration, so a graph-only candidate early in the list cannot eat a slot in the shortlist. Adds a regression test that exercises this case. - Deep-copy hydrated.args_schema before returning it from the discovery hook so external runtimes that mutate nested dicts / lists cannot corrupt catalog state across subsequent calls. Existing copy test is extended to assert nested-mutation isolation. - Update make_context_hook docstring to describe the actual behavior (query stamped onto item.metadata["codemode_query"], no synthetic user_turn ingested) and the implementation's stateless rationale. - Reword tests/test_adapters_fastmcp_discovery.py module-level skip comment to reflect that fastmcp>=2.0 is now in [dev]. - CHANGELOG: add a Fixed subsection under Unreleased describing the review-driven changes to the FastMCP CodeMode hook factories.
Follows the in-memory audit-grade review of fc95058. Addresses the one major and three should-fix-elevated findings the audit surfaced. - M-1 (major): make_context_hook now accepts a `tool_name` parameter (default "codemode.tool_result") instead of hardcoding "codemode.discovery". The hardcoded value was both semantically wrong (this is a context / firewall hook, not a discovery hook) and lossy for multi-tool agents whose traces could no longer be sliced by the underlying tool. Adds tests pinning both the default and the override. - m-1: New FastMCP integration test test_context_hook_compacts_real_fastmcp_tool_call drives a real in-memory fastmcp.FastMCP server's tool through `fastmcp.Client`, feeds the wire result to make_context_hook, and asserts firewall behavior + tool_name / codemode_query stamping end-to-end. - m-2: New test_discovery_tool_zero_top_k_returns_empty_list pinning the `top_k=0` boundary (valid config, returns []). - m-3: Tightened the make_context_hook docstring to describe the post-firewall metadata-stamp timing precisely — codemode_query is visible to event-log readers and on_context_built callbacks but not to on_firewall_triggered. Replaced the audit's suggested EventHook demo test with a post-hook event-log consumability pin (test_codemode_query_metadata_consumable_post_hook) once the timing constraint of the existing apply_firewall callback made the firewall-time-visible demonstration architecturally impossible without a larger ingest_tool_result_sync API change. `make ci` clean: 1178 passed, 13 skipped (+5 new tests on top of the Phase 1 baseline). CHANGELOG Unreleased Fixed subsection extended.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands the 3-issue "adoption demonstrations" group selected by the triage
pass. Single combined PR (Mode B, owner-authorised). Same blast radius:
two new reference architectures under examples/architectures/ + two new
adapter callable factories in adapters/fastmcp.py + paired docs and tests.
Zero changes to the context or routing core pipelines.
#87 — FastMCP CodeMode discovery / context hooks
Callable[[str], list[dict]] and Callable[[str, str], str]
respectively, matching the FastMCP CodeMode hook contract
(Coordination: CodeMode vs external context/routing strategies (contextweaver) PrefectHQ/fastmcp#3365) but framework-
agnostic — neither captures any FastMCP reference at runtime.
Catalog into a "given a query, return a shortlist of tools" callable;
graph-only nodes are skipped silently.
the firewall as a "(query, raw_result) → summary" callable; raw bytes
park in the artifact store, query is recorded as item.metadata for
trace correlation.
shortlist with ~86% token reduction.
tests in tests/test_adapters_fastmcp_discovery.py (spins up
fastmcp.FastMCP in-memory server, round-trips through the hook).
integration test runs on every CI matrix cell.
#204 — Code-review bot reference architecture
(24 tools across grep/git/lint/typecheck/test/review) + README.md +
OUTPUT.md + init.py.
firewall is the load-bearing pattern: synthetic ~28 KB diff dump and
~2.5 KB grep result both compact to ~500-char summaries while raw
bytes stay addressable.
deterministic invariants (catalog size, intent matches, firewall
fires=2/6, artifact count=6, fact keys).
#205 — Voice agent reference architecture (Pipecat)
(18 tools across support/orders/shipping/account/callback) +
README.md + OUTPUT.md + init.py.
context build runs via asyncio.to_thread(mgr.build_sync, ...);
ContextBudget(route=200, call=500, interpret=400, answer=1000)
enforces sub-300 ms TTS-friendly answer prompts (max 200 tokens at
five turns).
pyproject.toml grows a [voice] extra (pipecat-ai>=0.0.50) for users
who want the real FrameProcessor.
async-build marker, intent matches, fact keys, and the 400-token
answer-prompt ceiling. Wall-clock timings are not asserted on.
callout pointing back at the architecture.
Shared bookkeeping
to importlib.util.spec_from_file_location so the three architecture
test files can coexist in one pytest run (a bare
import mainfromsys.path collides across architectures).
Module-size note
adapters/fastmcp.py lands at 428 lines, over the soft 300-line guide,
in line with adapters/mcp.py (401) and adapters/proxy_runtime.py (462)
precedent. Mode B authorised the modest overrun rather than splitting
one adapter across two files.
Verification
ruff format --check src/ tests/ examples/ scripts/ → 145 files clean
ruff check src/ tests/ examples/ scripts/ → All checks passed
mypy src/ → 0 issues / 64 files
pytest --cov=contextweaver -q → 985 passed, 5 skipped
(+34 new tests)
make example → all 14 scripts ran
make demo → clean
make scorecard-check → clean (no benchmark drift)
make llms-check → up to date
Closes #87
Closes #204
Closes #205
https://claude.ai/code/session_01JiR8ZGtuwn7Cv2ahHMwLhL