fix: MCP tool discovery pipeline, agentTool migration, and e2e reorganization #276

betterclever · 2026-02-10T12:08:05Z

Summary

Consolidates three major improvements to the tool calling pipeline, MCP architecture, and test organization:

MCP Tool Discovery & Calling Pipeline - Fixed 4 critical bugs preventing agent tool execution
Frontend/Backend Migration - Transitioned from legacy agentTool to new toolProvider API
E2E Test Reorganization - Restructured tests into tiers with shared boilerplate extraction
Mock Agent Component - Added diagnostic component for validating tool discovery pipeline

MCP Tool Discovery Pipeline Fixes

Bugs Fixed:

SDK Initialize Handshake: Tool discovery now uses MCP SDK Client with initialize() instead of raw fetch
Tool Metadata Names: prepareAndRegisterToolActivity uses getToolMetadata(component).name for toolName (not nodeId)
Secret Resolution: executeToolCallSignal passes inputOverrides: request.credentials to resolve secrets at execution time
Race Condition: MCP groups register as "ready" AFTER child server discovery completes (not before)

Key Files: mcp-gateway.service.ts, mcp-group-runtime.ts, mcp.activity.ts, tool-helpers.ts, workflows/index.ts

agentTool → toolProvider Migration

All security components (AbuseIPDB, VirusTotal, AWS MCP, etc.) migrated from legacy agentTool interface to unified toolProvider pattern. Frontend and backend serialization updated to match.

Mock Agent Diagnostic Component

worker/src/components/dev/mock-agent.ts - Connects to MCP gateway, discovers tools, and optionally calls each tool with test arguments. Validates the full pipeline without a real AI agent.

E2E test (pipeline/mock-agent-tool-discovery.test.ts) asserts tool discovery and execution end-to-end.

E2E Test Reorganization

e2e-tests/
  helpers/e2e-harness.ts    (NEW - shared boilerplate)
  core/                     (7 tests, local only, < 6 min each)
  pipeline/                 (2 tests, needs API keys, 5-8 min)
  cloud/                    (1 test, needs AWS + ngrok, 10-15 min)

Tier	Gate	Requirements
Core	`RUN_E2E=true`	Backend + Worker only
Pipeline	`RUN_E2E=true` + API keys	External APIs + Docker
Cloud	`RUN_CLOUD_E2E=true`	AWS infrastructure

Scripts: test:e2e:core, test:e2e:pipeline, test:e2e:cloud

Cleanups

Removed 10 redundant docs, dead mcp:* npm scripts, stale pm2 MCP server config
Renamed .env.eng-104 → .env.e2e
Added .context to .gitignore

Test plan

bun run typecheck passes
bun test - 560 pass, 113 skip, 0 fail
Run test:e2e:core with local services
Run test:e2e:pipeline with API keys configured
Verify mock.agent discovers and calls MCP tools end-to-end

🤖 Generated with Claude Code

- Fix BACKEND_URL default from localhost:3000 to localhost:3211 - Add /api/v1 prefix to internal MCP API calls - Add mode: 'tool' to aws-mcp-group test configuration - Fix duplicate Outputs section in CloudFormation YAML The MCP group was failing because: 1. Worker was calling wrong backend port/path 2. Test wasn't marking aws-mcp-group as a tool provider Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

Add register-group-server endpoint for MCP group runtime to fetch server configuration during workflow execution. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

Signed-off-by: betterclever <paliwal.pranjal83@gmail.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c4208-d8d5-71f6-8874-506f0b67f197 Co-authored-by: Amp <amp@ampcode.com>

…ation Signed-off-by: betterclever <paliwal.pranjal83@gmail.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c4208-d8d5-71f6-8874-506f0b67f197 Co-authored-by: Amp <amp@ampcode.com>

… tools Signed-off-by: betterclever <paliwal.pranjal83@gmail.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c4208-d8d5-71f6-8874-506f0b67f197 Co-authored-by: Amp <amp@ampcode.com>

Signed-off-by: betterclever <paliwal.pranjal83@gmail.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c4208-d8d5-71f6-8874-506f0b67f197 Co-authored-by: Amp <amp@ampcode.com>

- Add sessionIdGenerator: () => randomUUID() for both named-server and single-server transports (SDK 1.26.0+ rejects stateless transport reuse) - Empty built-in named-servers.json to prevent proxy from ignoring MCP_COMMAND env var when image has hardcoded server configs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

…dshake Replace raw fetch()-based tool discovery with MCP SDK Client that performs proper initialize handshake before tools/list. AWS MCP servers reject bare tools/list without initialization, causing silent 0-tool discovery. Changes: - Worker: discoverToolsWithRetry uses SDK Client + StreamableHTTPClientTransport - Worker: MCP group runtime uses command: [] and MCP_NAMED_SERVERS='{}' env - Backend: gateway discoverToolsFromEndpoint uses SDK Client - Backend: add register-mcp-server endpoint with pre-discovered tools - Backend: add tools-ready polling endpoint for workflow coordination - SDK: add exposedToAgent field, RegisterMcpServerInput DTO - Security components: add providerKind and exposedToAgent metadata - Fix default backend port from 3000 to 3211 in mcp-library-utils Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

Move prepareAndRegisterToolActivity for MCP groups from before to after runComponentWithRetry. Previously, the parent group was marked "ready" in Redis before child server discovery, causing the agent's areAllToolsReadyActivity check to pass too early. The agent would start with only 2 tools instead of 47. Now the sequence is: 1. MCP group executes (starts containers, discovers tools, registers children) 2. Parent registers as "ready" in Redis 3. Agent's polling detects all tools ready 4. Agent starts with complete tool set Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

…tion New mock.agent component that connects to the MCP gateway, lists all available tools, and returns them as output. Provides a fast, deterministic way to verify the full tool pipeline without an LLM. - worker/src/components/dev/mock-agent.ts: inline component - worker/src/components/dev/__tests__/mock-agent.test.ts: unit tests - e2e-tests/mock-agent-tool-discovery.test.ts: e2e test (47 tools) - Register component in worker/src/components/index.ts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

Replace all agentTool references with the new toolProvider API: - agentTool.enabled → !!toolProvider - agentTool.toolName → toolProvider.name - agentTool.toolDescription → toolProvider.description - Update Swagger API schema in components controller - Fix stale test name in tool-helpers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

1. Rewrite stdio-proxy to stateless JSON-RPC (docker/mcp-stdio-proxy): Bypass MCP SDK Server class which only accepts one initialize per lifetime. Handle JSON-RPC directly in Express routes so unlimited HTTP clients (worker discovery + gateway tool calls) can share one stdio server. Return 202 for notifications, 405 for GET/DELETE. 2. Persistent MCP client pool in gateway (mcp-gateway.service.ts): Cache one Client per endpoint URL. Reuse for both discovery and tool calls. Evict on error. Close all on run cleanup. 3. Fix tool name registration (mcp.activity.ts): Use getToolMetadata(component).name (e.g. abuseipdb_check) instead of node ID (e.g. abuseipdb) when registering component tools. 4. Fix secret resolution for tool calls (workflows/index.ts): Pass inputOverrides: request.credentials to _runComponentActivity in executeToolCallSignal so resolveSecretInputOverrides can resolve secret names to actual values during agent tool calls. Also: fix mock-agent AWS test args (snake_case), update e2e test to handle createLightweightSummary array truncation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

…docs - Introduce core/, pipeline/, cloud/ directories with shared e2e-harness.ts - Rename .env.eng-104 -> .env.e2e, RUN_GUARDDUTY_E2E -> RUN_CLOUD_E2E - Add test:e2e:core, test:e2e:pipeline, test:e2e:cloud scripts - Remove 10 redundant session-generated docs - Remove dead shipsec-mcp-server pm2 config and package.json scripts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

Move analytics.test.ts (from main's analytics dashboard PR) into the core/ tier and refactor to use shared e2e-harness helpers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

Prevent accidental commit of context dump files that may contain secrets. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c25f26e26a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-10T12:17:11Z

worker/src/components/core/mcp-library-utils.ts

+  return allServers
+    .map((s) => McpServerSchema.parse(s))
+    .filter((s) => enabledServerIds.includes(s.id));


Filter out disabled MCP servers before registering tools

fetchEnabledServers now filters only by selected ID and no longer enforces enabled === true, so a server that was disabled in MCP settings can still be started and registered when mcp.custom executes. This is a behavioral regression from the previous implementation and can re-expose tools administrators explicitly turned off.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-02-10T12:17:11Z

pm2.config.cjs

          TEMPORAL_NAMESPACE: `shipsec-dev-${instanceNum}`,
          TEMPORAL_TASK_QUEUE: `shipsec-dev-${instanceNum}`,
+          // DEBUG: Skip container cleanup for MCP debugging
+          SKIP_CONTAINER_CLEANUP: process.env.SKIP_CONTAINER_CLEANUP || 'true',


Remove default skip for MCP container cleanup

Setting SKIP_CONTAINER_CLEANUP to 'true' by default means PM2 worker runs will always bypass cleanupLocalMcpActivity (which returns early when this env var is true), so MCP containers and volumes accumulate across runs. In local/E2E environments this can quickly cause stale state and Docker resource exhaustion that breaks subsequent experiments.

Useful? React with 👍 / 👎.

- Add `s.enabled` check to `fetchEnabledServers` filter so disabled MCP servers are not started/registered (P1) - Change SKIP_CONTAINER_CLEANUP default from 'true' to 'false' to prevent Docker container accumulation (P2) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

registerLocalMcpActivity and registerRemoteMcpActivity were calling removed register-local/register-remote backend endpoints. Remapped both to the unified register-mcp-server endpoint. Also includes server.id in template hash computation and adds Swagger response typing for GET /mcp-groups/templates. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

betterclever and others added 18 commits February 10, 2026 15:51

feat(backend): add MCP group server config endpoint

11a4e8e

Add register-group-server endpoint for MCP group runtime to fetch server configuration during workflow execution. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

feat(mcp-groups): add AWS group template and runtime support

2a0bc62

Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

fix: disable opencode fail-fast hack to allow full agent execution

c1d129c

Signed-off-by: betterclever <paliwal.pranjal83@gmail.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c4208-d8d5-71f6-8874-506f0b67f197 Co-authored-by: Amp <amp@ampcode.com>

docs: add MCP group registration and tool discovery pipeline document…

267ce86

…ation Signed-off-by: betterclever <paliwal.pranjal83@gmail.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c4208-d8d5-71f6-8874-506f0b67f197 Co-authored-by: Amp <amp@ampcode.com>

fix: disable aws-mcp-group as component tool - only expose discovered…

7cc00c5

… tools Signed-off-by: betterclever <paliwal.pranjal83@gmail.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c4208-d8d5-71f6-8874-506f0b67f197 Co-authored-by: Amp <amp@ampcode.com>

docs: add MCP architecture robustness improvements proposal

520975a

Signed-off-by: betterclever <paliwal.pranjal83@gmail.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c4208-d8d5-71f6-8874-506f0b67f197 Co-authored-by: Amp <amp@ampcode.com>

feat: add exponential backoff retry for MCP endpoint tool discovery

f48feaf

Signed-off-by: betterclever <paliwal.pranjal83@gmail.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c4208-d8d5-71f6-8874-506f0b67f197 Co-authored-by: Amp <amp@ampcode.com>

docs: add summary of MCP robustness fixes and architecture improvements

b586af7

Signed-off-by: betterclever <paliwal.pranjal83@gmail.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c4208-d8d5-71f6-8874-506f0b67f197 Co-authored-by: Amp <amp@ampcode.com>

chore: add .context to gitignore

c25f26e

Prevent accidental commit of context dump files that may contain secrets. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: betterclever <paliwal.pranjal83@gmail.com>

chatgpt-codex-connector bot reviewed Feb 10, 2026

View reviewed changes

betterclever force-pushed the fix/mcp-tool-discovery-pipeline branch from 8888a7e to 8766162 Compare February 10, 2026 13:49

betterclever force-pushed the fix/mcp-tool-discovery-pipeline branch from 8766162 to 035f3d9 Compare February 10, 2026 13:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: MCP tool discovery pipeline, agentTool migration, and e2e reorganization #276

fix: MCP tool discovery pipeline, agentTool migration, and e2e reorganization #276

Uh oh!

betterclever commented Feb 10, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Feb 10, 2026

Uh oh!

chatgpt-codex-connector bot Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

fix: MCP tool discovery pipeline, agentTool migration, and e2e reorganization #276

Are you sure you want to change the base?

fix: MCP tool discovery pipeline, agentTool migration, and e2e reorganization #276

Uh oh!

Conversation

betterclever commented Feb 10, 2026

Summary

MCP Tool Discovery Pipeline Fixes

agentTool → toolProvider Migration

Mock Agent Diagnostic Component

E2E Test Reorganization

Cleanups

Test plan

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant