diff --git a/.gitignore b/.gitignore index 71812a5c..10ef2a05 100644 --- a/.gitignore +++ b/.gitignore @@ -36,6 +36,7 @@ extensions_config.json coverage.xml coverage/ .deer-flow/ +.flowlens-agentops/ .claude/ skills/custom/* logs/ diff --git a/README.md b/README.md index e0424446..9edfa58a 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ English | [中文](./README_zh.md) ![FlowLens AgentOps dashboard](./docs/assets/flowlens-agentops-dashboard.png) -[Pages Demo](https://try1004.github.io/FlowLens-AgentOps/) | [Quick Start](#quick-start) | [Architecture](./docs/architecture.md) +[Pages Demo](https://try1004.github.io/FlowLens-AgentOps/) | [Quick Start](#quick-start) | [Architecture](./docs/architecture.md) | [Dual MCP Validation](./docs/mcp-validation-guide.md) | [Maintainer Handoff](./now.md) > Built on [DeerFlow 2.0](https://github.com/bytedance/deer-flow) under the MIT License. DeerFlow provides the Agent runtime, tools, subagents, memory, MCP, sandbox, and application foundation. FlowLens adds the diagnostics and AgentOps product layer described below. @@ -32,6 +32,7 @@ A checkpoint can restore state, but it does not explain why a multi-agent run fa - **Root Cause**: rule-based primary cause, evidence links, contributing failures, and remediation guidance. - **Replay mode**: success, tool error, subagent timeout, guardrail denial, checkpoint rollback, MCP auth error, and memory error. - **Connected mode**: reads the real DeerFlow Gateway and RunEventStore through owner-checked APIs. +- **Dual MCP validation**: internal SQLite Analytics and FlowLens Inspector services demonstrate bounded read-only data access and user-bound runtime delegation. - **Portable delivery**: Next.js route, Vite static build, Docker/Nginx demo, CI, and GitHub Pages workflow. FlowLens AgentOps mobile Root Cause view @@ -44,6 +45,14 @@ The diagnostics path is read-only: it does not mutate checkpoints, prompts, RunR ## Quick Start +### Choose a mode + +| Mode | What it proves | Requirements | URL | +| --- | --- | --- | --- | +| Replay | Dashboard interactions and deterministic synthetic failure scenarios | Node.js or Docker; no model key | `http://localhost:5173` or `http://localhost:4173` | +| Connected Runtime | Real DeerFlow runs emitted into the AgentOps event pipeline | Normal DeerFlow configuration, account, and model provider | `http://localhost:2026/agentops` | +| Connected + dual MCP | Runtime delegation, read-only SQLite Analytics MCP, and FlowLens Inspector MCP | Docker Desktop plus the Connected Runtime requirements | `http://localhost:2027/agentops` | + ### No-key Replay with pnpm ```bash @@ -66,6 +75,14 @@ make flowlens-demo Open `http://localhost:4173`, then stop it with `make flowlens-stop`. +On Windows without `make`, run the Compose command directly: + +```powershell +docker compose -f docker/docker-compose-flowlens-demo.yaml up --build +``` + +Press `Ctrl+C` to stop the foreground Replay container. + ### Connected DeerFlow runtime ```bash @@ -75,6 +92,39 @@ make up Open `http://localhost:2026/agentops`. Connected mode requires the normal DeerFlow configuration, authentication, and at least one configured model provider. It uses the same React components and Zod contracts as Replay mode. +### Connected runtime with dual MCP services + +The development stack runs two internal-only Streamable HTTP MCP services: + +- **SQLite Analytics MCP** exposes schema inspection and `SELECT`-only local demonstration-data queries. +- **FlowLens Inspector MCP** queries owner-checked diagnostic data through the Gateway. + +Schemas are discovered with a restricted service token. A real tool invocation receives a short-lived token bound to the current `user_id` and `run_id`; the model never receives that token. + +From the repository root in PowerShell: + +```powershell +$env:DEER_FLOW_ROOT = (Get-Location).Path +$env:HOME = $env:USERPROFILE +$env:DEER_FLOW_CONTAINER_PREFIX = "flowlens-agentops" +$env:DEER_FLOW_HTTP_PORT = "2027" + +docker compose -p flowlens-agentops ` + -f docker/docker-compose-dev.yaml ` + -f docker/docker-compose-agentops-isolated.yaml ` + up -d --build --remove-orphans frontend gateway nginx sqlite-analytics flowlens-inspector + +Invoke-RestMethod http://localhost:2027/health +``` + +Open `http://localhost:2027/`, sign in, then open `http://localhost:2027/agentops`. To exercise the real SQLite path, start a new chat and send: + +```text +Use the SQLite Analytics MCP to inspect the available schema, describe the agent_runs table, then count runs by status. Execute read-only SQL only, state the SQL used, and label the result as local demonstration data. +``` + +The selected Run should show MCP events in Timeline and an `MCP > 0` metric. The [Dual MCP Validation Guide](./docs/mcp-validation-guide.md) and [Chinese end-to-end runbook](./docs/flowlens-dual-mcp-test-runbook.zh-CN.md) cover expected evidence, policy-rejection cases, and shutdown. + ## Diagnostic APIs | Method | Endpoint | Purpose | @@ -125,8 +175,10 @@ FlowLens does not claim authorship of the DeerFlow runtime modules. See [NOTICE] | Gate | Result | | ---------------------------------- | ------------------------------------------------------------------ | -| Focused backend FlowLens/API suite | 53 tests passed | -| Frontend Vitest suite | 219 tests passed | +| Focused backend FlowLens/API suite | 58 tests passed on 2026-07-31 | +| Frontend Vitest suite | 232 tests passed on 2026-07-31 | +| Frontend quality gate | ESLint, TypeScript, Next production build, and Vite Replay build passed on 2026-07-31 | +| Real MCP integration probe | `ToolNode -> delegated OAuth token -> SQLite schema tool` completed locally | | Next and static Replay Playwright | 5 tests passed across route, desktop, tablet, and mobile workflows | | Secret scan | No high-confidence credentials in Git-tracked files | | In-process diagnostics benchmark | 2,000 events, 20 iterations, median 24.7578 ms, P95 36.5606 ms | @@ -155,6 +207,9 @@ The machine-readable result is committed at [flowlens-benchmark.json](./docs/ass - [Event schema and instrumentation](./docs/event-schema.md) - [Failure attribution rules](./docs/failure-attribution.md) - [Replay and Connected demo guide](./docs/demo-guide.md) +- [Dual MCP validation guide](./docs/mcp-validation-guide.md) +- [Chinese dual MCP end-to-end runbook](./docs/flowlens-dual-mcp-test-runbook.zh-CN.md) +- [Maintainer handoff and next steps](./now.md) - [Contributing](./CONTRIBUTING.md) - [Changelog](./CHANGELOG.md) diff --git a/backend/app/gateway/app.py b/backend/app/gateway/app.py index 30662d2f..3069a0b6 100644 --- a/backend/app/gateway/app.py +++ b/backend/app/gateway/app.py @@ -19,6 +19,7 @@ channels, feedback, mcp, + mcp_tokens, memory, models, runs, @@ -178,6 +179,11 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]: logger.exception(error_msg) raise RuntimeError(error_msg) from e config = get_gateway_config() + from app.gateway.auth.platform_tokens import ( + ensure_platform_tokens_configured_if_enabled, + ) + + ensure_platform_tokens_configured_if_enabled() logger.info(f"Starting API Gateway on {config.host}:{config.port}") # Pre-warm tiktoken encoding cache so the first memory-injection request @@ -385,6 +391,7 @@ def create_app() -> FastAPI: # Auth API is mounted at /api/v1/auth app.include_router(auth.router) + app.include_router(mcp_tokens.router) # Feedback API is mounted at /api/threads/{thread_id}/runs/{run_id}/feedback app.include_router(feedback.router) diff --git a/backend/app/gateway/auth/models.py b/backend/app/gateway/auth/models.py index 25c6476f..fbf435ec 100644 --- a/backend/app/gateway/auth/models.py +++ b/backend/app/gateway/auth/models.py @@ -18,6 +18,7 @@ class User(BaseModel): model_config = ConfigDict(from_attributes=True) id: UUID = Field(default_factory=uuid4, description="Primary key") + tenant_id: str = Field(default="default", description="Owning tenant") email: EmailStr = Field(..., description="Unique email address") password_hash: str | None = Field(None, description="bcrypt hash, nullable for OAuth users") system_role: Literal["admin", "user"] = Field(default="user") diff --git a/backend/app/gateway/auth/platform_tokens.py b/backend/app/gateway/auth/platform_tokens.py new file mode 100644 index 00000000..88665253 --- /dev/null +++ b/backend/app/gateway/auth/platform_tokens.py @@ -0,0 +1,311 @@ +"""RS256 platform tokens used by DeerFlow Runtime and FlowLens MCP.""" + +from __future__ import annotations + +import base64 +import os +import uuid +from dataclasses import dataclass, field +from datetime import UTC, datetime, timedelta +from pathlib import Path +from typing import Any + +import jwt +from cryptography.hazmat.primitives import serialization +from cryptography.hazmat.primitives.asymmetric.rsa import RSAPublicKey +from pydantic import BaseModel, ValidationError, field_validator + +PLATFORM_ALGORITHM = "RS256" +TOOLS_DISCOVERY_SCOPE = "agentops.tools.discover" +AGENTOPS_READ_SCOPES: tuple[str, ...] = ( + "agentops.health.read", + "agentops.runs.read", + "agentops.timeline.read", + "agentops.diagnostics.read", + "agentops.runs.compare", +) +SQLITE_ANALYTICS_READ_SCOPES: tuple[str, ...] = ( + "analytics.schema.read", + "analytics.query.read", +) +_TRUTHY_VALUES = frozenset({"1", "true", "yes", "on"}) + + +class PlatformTokenError(ValueError): + """Raised when platform token configuration or validation fails.""" + + +@dataclass(frozen=True, slots=True) +class PlatformTokenSettings: + """Key material and claim policy for FlowLens platform tokens.""" + + private_key_pem: str | None = field(repr=False) + public_key_pem: str = field(repr=False) + kid: str + issuer: str + audience: str + client_id: str + client_secret: str = field(repr=False) + token_ttl_seconds: int = 300 + + @classmethod + def from_env( + cls, + *, + require_private_key: bool = True, + ) -> PlatformTokenSettings: + """Load platform token settings from environment-backed PEM files.""" + private_path = os.getenv("FLOWLENS_MCP_JWT_PRIVATE_KEY_PATH", "").strip() + public_path = os.getenv("FLOWLENS_MCP_JWT_PUBLIC_KEY_PATH", "").strip() + client_secret = os.getenv("FLOWLENS_MCP_CLIENT_SECRET", "") + + if not public_path: + raise PlatformTokenError( + "FLOWLENS_MCP_JWT_PUBLIC_KEY_PATH is required", + ) + if require_private_key and not private_path: + raise PlatformTokenError( + "FLOWLENS_MCP_JWT_PRIVATE_KEY_PATH is required", + ) + if not client_secret: + raise PlatformTokenError("FLOWLENS_MCP_CLIENT_SECRET is required") + + try: + ttl_seconds = int( + os.getenv("FLOWLENS_MCP_TOKEN_TTL_SECONDS", "300"), + ) + except ValueError as exc: + raise PlatformTokenError( + "FLOWLENS_MCP_TOKEN_TTL_SECONDS must be an integer", + ) from exc + if ttl_seconds <= 0: + raise PlatformTokenError( + "FLOWLENS_MCP_TOKEN_TTL_SECONDS must be positive", + ) + + return cls( + private_key_pem=(_read_key_file(private_path) if private_path else None), + public_key_pem=_read_key_file(public_path), + kid=os.getenv( + "FLOWLENS_MCP_JWT_KID", + "flowlens-local-1", + ).strip() + or "flowlens-local-1", + issuer=os.getenv( + "FLOWLENS_MCP_ISSUER", + "deerflow-gateway", + ).strip() + or "deerflow-gateway", + audience=os.getenv( + "FLOWLENS_MCP_AUDIENCE", + "flowlens-platform", + ).strip() + or "flowlens-platform", + client_id=os.getenv( + "FLOWLENS_MCP_CLIENT_ID", + "deerflow-runtime", + ).strip() + or "deerflow-runtime", + client_secret=client_secret, + token_ttl_seconds=ttl_seconds, + ) + + +class PlatformTokenClaims(BaseModel): + """Validated immutable identity and authorization claims.""" + + model_config = {"frozen": True} + + iss: str + sub: str + tenant_id: str + aud: str | list[str] + scope: list[str] + origin_run_id: str | None = None + azp: str + jti: str + iat: datetime + exp: datetime + ver: int + + @field_validator("scope") + @classmethod + def _require_unique_scopes(cls, value: list[str]) -> list[str]: + if not value or any(not scope for scope in value): + raise ValueError("scope must contain non-empty values") + if len(value) != len(set(value)): + raise ValueError("scope values must be unique") + return value + + +class PlatformTokenIssuer: + """Issue short-lived service and delegated platform tokens.""" + + def __init__(self, settings: PlatformTokenSettings) -> None: + if not settings.private_key_pem: + raise PlatformTokenError( + "Private key is required for token issuance", + ) + self._settings = settings + + def issue_service_token(self) -> str: + """Issue a least-privilege client token for MCP tool discovery.""" + return self._issue( + subject=self._settings.client_id, + tenant_id="system", + scopes=[TOOLS_DISCOVERY_SCOPE], + origin_run_id=None, + token_version=0, + ) + + def issue_delegated_token( + self, + *, + user_id: str, + tenant_id: str, + token_version: int, + origin_run_id: str, + scopes: tuple[str, ...] = AGENTOPS_READ_SCOPES, + ) -> str: + """Issue a user-bound token for read-only AgentOps inspection.""" + return self._issue( + subject=user_id, + tenant_id=tenant_id, + scopes=list(scopes), + origin_run_id=origin_run_id, + token_version=token_version, + ) + + def _issue( + self, + *, + subject: str, + tenant_id: str, + scopes: list[str], + origin_run_id: str | None, + token_version: int, + ) -> str: + now = datetime.now(UTC) + payload = { + "iss": self._settings.issuer, + "sub": subject, + "tenant_id": tenant_id, + "aud": self._settings.audience, + "scope": scopes, + "origin_run_id": origin_run_id, + "azp": self._settings.client_id, + "jti": str(uuid.uuid4()), + "iat": now, + "exp": now + timedelta(seconds=self._settings.token_ttl_seconds), + "ver": token_version, + } + return jwt.encode( + payload, + self._settings.private_key_pem, + algorithm=PLATFORM_ALGORITHM, + headers={"kid": self._settings.kid}, + ) + + +def ensure_platform_tokens_configured_if_enabled() -> PlatformTokenSettings | None: + """Validate key material at startup when FlowLens MCP is enabled.""" + enabled = os.getenv("FLOWLENS_MCP_ENABLED", "").strip().lower() in _TRUTHY_VALUES + if not enabled: + return None + return PlatformTokenSettings.from_env() + + +def verify_platform_token( + token: str, + settings: PlatformTokenSettings | None = None, +) -> PlatformTokenClaims: + """Verify signature, key id, issuer, audience, time, and claims.""" + resolved_settings = settings or PlatformTokenSettings.from_env( + require_private_key=False, + ) + try: + header = jwt.get_unverified_header(token) + if header.get("alg") != PLATFORM_ALGORITHM: + raise PlatformTokenError("Unsupported platform token algorithm") + if header.get("kid") != resolved_settings.kid: + raise PlatformTokenError("Unknown platform token key id") + + payload = jwt.decode( + token, + resolved_settings.public_key_pem, + algorithms=[PLATFORM_ALGORITHM], + audience=resolved_settings.audience, + issuer=resolved_settings.issuer, + options={ + "require": [ + "iss", + "sub", + "tenant_id", + "aud", + "scope", + "azp", + "jti", + "iat", + "exp", + "ver", + ], + }, + ) + claims = PlatformTokenClaims.model_validate(payload) + if claims.azp != resolved_settings.client_id: + raise PlatformTokenError( + "Untrusted platform token authorized party", + ) + return claims + except PlatformTokenError: + raise + except (jwt.PyJWTError, ValidationError, TypeError, ValueError) as exc: + raise PlatformTokenError("Invalid platform token") from exc + + +def build_platform_jwks( + settings: PlatformTokenSettings | None = None, +) -> dict[str, list[dict[str, Any]]]: + """Return the public key as an RFC 7517 JSON Web Key Set.""" + resolved_settings = settings or PlatformTokenSettings.from_env( + require_private_key=False, + ) + try: + public_key = serialization.load_pem_public_key( + resolved_settings.public_key_pem.encode(), + ) + except (TypeError, ValueError) as exc: + raise PlatformTokenError("Invalid RSA public key") from exc + if not isinstance(public_key, RSAPublicKey): + raise PlatformTokenError("Platform public key must be RSA") + + numbers = public_key.public_numbers() + return { + "keys": [ + { + "kty": "RSA", + "use": "sig", + "alg": PLATFORM_ALGORITHM, + "kid": resolved_settings.kid, + "n": _base64url_uint(numbers.n), + "e": _base64url_uint(numbers.e), + } + ] + } + + +def _read_key_file(path: str) -> str: + try: + return Path(path).read_text(encoding="utf-8") + except OSError as exc: + raise PlatformTokenError( + f"Unable to read platform key file: {path}", + ) from exc + + +def _base64url_uint(value: int) -> str: + length = max(1, (value.bit_length() + 7) // 8) + encoded = base64.urlsafe_b64encode( + value.to_bytes(length, "big"), + ) + return encoded.rstrip(b"=").decode("ascii") diff --git a/backend/app/gateway/auth/repositories/sqlite.py b/backend/app/gateway/auth/repositories/sqlite.py index 3ee3978e..39f4a61c 100644 --- a/backend/app/gateway/auth/repositories/sqlite.py +++ b/backend/app/gateway/auth/repositories/sqlite.py @@ -36,6 +36,7 @@ def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None: def _row_to_user(row: UserRow) -> User: return User( id=UUID(row.id), + tenant_id=row.tenant_id, email=row.email, password_hash=row.password_hash, system_role=row.system_role, # type: ignore[arg-type] @@ -52,6 +53,7 @@ def _row_to_user(row: UserRow) -> User: def _user_to_row(user: User) -> UserRow: return UserRow( id=str(user.id), + tenant_id=user.tenant_id, email=user.email, password_hash=user.password_hash, system_role=user.system_role, @@ -100,6 +102,7 @@ async def update_user(self, user: User) -> User: # a row that no longer exists. raise UserNotFoundError(f"User {user.id} no longer exists") row.email = user.email + row.tenant_id = user.tenant_id row.password_hash = user.password_hash row.system_role = user.system_role row.oauth_provider = user.oauth_provider diff --git a/backend/app/gateway/auth_middleware.py b/backend/app/gateway/auth_middleware.py index 6b645226..1aa77340 100644 --- a/backend/app/gateway/auth_middleware.py +++ b/backend/app/gateway/auth_middleware.py @@ -17,7 +17,12 @@ from starlette.types import ASGIApp from app.gateway.auth.errors import AuthErrorCode, AuthErrorResponse -from app.gateway.authz import _ALL_PERMISSIONS, AuthContext +from app.gateway.auth.platform_tokens import ( + AGENTOPS_READ_SCOPES, + PlatformTokenError, + verify_platform_token, +) +from app.gateway.authz import _ALL_PERMISSIONS, AuthContext, Permissions from app.gateway.internal_auth import INTERNAL_AUTH_HEADER_NAME, get_internal_user, is_valid_internal_auth_token from deerflow.runtime.user_context import reset_current_user, set_current_user @@ -38,10 +43,35 @@ "/api/v1/auth/logout", "/api/v1/auth/setup-status", "/api/v1/auth/initialize", + "/api/v1/auth/mcp/token", + "/.well-known/jwks.json", } ) +def _bearer_token(request: Request) -> str | None: + authorization = request.headers.get("Authorization", "") + scheme, separator, token = authorization.partition(" ") + if not separator or scheme.lower() != "bearer": + return None + return token.strip() or None + + +def _authentication_error( + code: AuthErrorCode, + message: str, +) -> JSONResponse: + return JSONResponse( + status_code=401, + content={ + "detail": AuthErrorResponse( + code=code, + message=message, + ).model_dump() + }, + ) + + def _is_public(path: str) -> bool: stripped = path.rstrip("/") if stripped in _PUBLIC_EXACT_PATHS: @@ -77,19 +107,49 @@ async def dispatch(self, request: Request, call_next: Callable) -> Response: return await call_next(request) internal_user = None + platform_user = None + platform_scopes: frozenset[str] = frozenset() if is_valid_internal_auth_token(request.headers.get(INTERNAL_AUTH_HEADER_NAME)): internal_user = get_internal_user() - # Non-public path: require session cookie - if internal_user is None and not request.cookies.get("access_token"): - return JSONResponse( - status_code=401, - content={ - "detail": AuthErrorResponse( - code=AuthErrorCode.NOT_AUTHENTICATED, - message="Authentication required", - ).model_dump() - }, + bearer_token = _bearer_token(request) if internal_user is None else None + if bearer_token is not None: + try: + claims = verify_platform_token(bearer_token) + except PlatformTokenError: + return _authentication_error( + AuthErrorCode.TOKEN_INVALID, + "Invalid platform token", + ) + + from app.gateway.deps import get_local_provider + + platform_user = await get_local_provider().get_user(claims.sub) + if platform_user is None: + return _authentication_error( + AuthErrorCode.USER_NOT_FOUND, + "Platform token user not found", + ) + if ( + platform_user.token_version != claims.ver + or platform_user.tenant_id != claims.tenant_id + ): + return _authentication_error( + AuthErrorCode.TOKEN_INVALID, + "Platform token is stale", + ) + platform_scopes = frozenset(claims.scope) + request.state.platform_claims = claims + request.state.platform_scopes = platform_scopes + + if ( + internal_user is None + and platform_user is None + and not request.cookies.get("access_token") + ): + return _authentication_error( + AuthErrorCode.NOT_AUTHENTICATED, + "Authentication required", ) # Strict JWT validation: reject junk/expired tokens with 401 @@ -107,18 +167,30 @@ async def dispatch(self, request: Request, call_next: Callable) -> Response: if internal_user is not None: user = internal_user + permissions = _ALL_PERMISSIONS + elif platform_user is not None: + user = platform_user + permissions = ( + [Permissions.RUNS_READ] + if platform_scopes.intersection(AGENTOPS_READ_SCOPES) + else [] + ) else: try: user = await get_current_user_from_request(request) except HTTPException as exc: return JSONResponse(status_code=exc.status_code, content={"detail": exc.detail}) + permissions = _ALL_PERMISSIONS # Stamp both request.state.user (for the contextvar pattern) # and request.state.auth (so @require_permission's "auth is # None" branch short-circuits instead of running the entire # JWT-decode + DB-lookup pipeline a second time per request). request.state.user = user - request.state.auth = AuthContext(user=user, permissions=_ALL_PERMISSIONS) + request.state.auth = AuthContext( + user=user, + permissions=permissions, + ) token = set_current_user(user) try: return await call_next(request) diff --git a/backend/app/gateway/csrf_middleware.py b/backend/app/gateway/csrf_middleware.py index f3488203..61b12ad9 100644 --- a/backend/app/gateway/csrf_middleware.py +++ b/backend/app/gateway/csrf_middleware.py @@ -51,6 +51,7 @@ def should_check_csrf(request: Request) -> bool: "/api/v1/auth/logout", "/api/v1/auth/register", "/api/v1/auth/initialize", + "/api/v1/auth/mcp/token", } ) diff --git a/backend/app/gateway/deps.py b/backend/app/gateway/deps.py index 5739d217..56e99091 100644 --- a/backend/app/gateway/deps.py +++ b/backend/app/gateway/deps.py @@ -31,6 +31,7 @@ from deerflow.runtime import RunContext, RunManager, StreamBridge from deerflow.runtime.events.store.base import RunEventStore from deerflow.runtime.runs.store.base import RunStore +from deerflow.runtime.user_context import DEFAULT_TENANT_ID logger = logging.getLogger(__name__) @@ -378,11 +379,23 @@ async def get_optional_user_from_request(request: Request): async def get_current_user(request: Request) -> str | None: - """Extract user_id from request cookie, or None if not authenticated. + """Return the middleware-validated user id, with cookie fallback. Thin adapter that returns the string id for callers that only need identification (e.g., ``feedback.py``). Full-user callers should use ``get_current_user_from_request`` or ``get_optional_user_from_request``. """ + state_user = getattr(request.state, "user", None) + if state_user is not None: + return str(state_user.id) user = await get_optional_user_from_request(request) return str(user.id) if user else None + + +def get_current_tenant_id(request: Request) -> str: + """Return the server-authenticated tenant, defaulting legacy users.""" + user = getattr(request.state, "user", None) + return str( + getattr(user, "tenant_id", None) + or DEFAULT_TENANT_ID + ) diff --git a/backend/app/gateway/routers/agentops.py b/backend/app/gateway/routers/agentops.py index f4ff0aee..8b50e6ed 100644 --- a/backend/app/gateway/routers/agentops.py +++ b/backend/app/gateway/routers/agentops.py @@ -2,17 +2,26 @@ from __future__ import annotations +import asyncio +import logging from datetime import datetime from typing import Any from fastapi import APIRouter, HTTPException, Query, Request from app.gateway.authz import require_permission -from app.gateway.deps import get_current_user, get_run_manager +from app.gateway.deps import ( + get_current_tenant_id, + get_current_user, + get_run_event_store, + get_run_manager, +) from deerflow.runtime import RunRecord, RunStatus +from deerflow.runtime.diagnostics import build_diagnostics_response from deerflow.runtime.runs.cursor import InvalidRunCursor, decode_run_cursor, encode_run_cursor router = APIRouter(prefix="/api/agentops", tags=["agentops"]) +logger = logging.getLogger(__name__) def _run_summary(record: RunRecord) -> dict[str, Any]: @@ -30,6 +39,28 @@ def _run_summary(record: RunRecord) -> dict[str, Any]: } +async def _run_summary_with_diagnostics(record: RunRecord, event_store: Any) -> dict[str, Any]: + """Enrich the list view without redefining raw run lifecycle status. + + A run may complete its outer loop successfully after an unrecovered tool + incident. The index exposes that derived incident separately so the UI can + show an operator-facing error state while retaining the factual run status. + """ + + summary = _run_summary(record) + try: + events = await event_store.list_events(record.thread_id, record.run_id, limit=5000) + summary["diagnostic_category"] = build_diagnostics_response( + record, + events, + limit=5000, + )["failure_attribution"]["primary_cause"] + except Exception: + logger.warning("Could not derive AgentOps incident for run %s", record.run_id, exc_info=True) + summary["diagnostic_category"] = None + return summary + + @router.get("/runs") @require_permission("runs", "read") async def list_agentops_runs( @@ -37,6 +68,7 @@ async def list_agentops_runs( limit: int = Query(default=50, ge=1, le=200), cursor: str | None = Query(default=None), status: RunStatus | None = Query(default=None), + thread_id: str | None = Query(default=None, min_length=1), started_after: datetime | None = Query(default=None), started_before: datetime | None = Query(default=None), ) -> dict[str, Any]: @@ -51,8 +83,11 @@ async def list_agentops_runs( user_id = await get_current_user(request) if user_id is None: raise HTTPException(status_code=401, detail="Authentication required") + tenant_id = get_current_tenant_id(request) records = await get_run_manager(request).list_recent( user_id=user_id, + tenant_id=tenant_id, + thread_id=thread_id, limit=limit + 1, cursor=cursor, status=status.value if status is not None else None, @@ -65,8 +100,38 @@ async def list_agentops_runs( if has_more and page_records: last = page_records[-1] next_cursor = encode_run_cursor(last.created_at, last.run_id) + event_store = get_run_event_store(request) + semaphore = asyncio.Semaphore(8) + + async def _enrich(record: RunRecord) -> dict[str, Any]: + async with semaphore: + return await _run_summary_with_diagnostics(record, event_store) + return { - "items": [_run_summary(record) for record in page_records], + "items": await asyncio.gather(*(_enrich(record) for record in page_records)), "has_more": has_more, "next_cursor": next_cursor, } + + +@router.get("/runs/{run_id}") +@require_permission("runs", "read") +async def get_agentops_run( + run_id: str, + request: Request, +) -> dict[str, Any]: + """Return an owner-safe stable run summary for MCP consumers.""" + user_id = await get_current_user(request) + if user_id is None: + raise HTTPException( + status_code=401, + detail="Authentication required", + ) + record = await get_run_manager(request).get( + run_id, + user_id=user_id, + tenant_id=get_current_tenant_id(request), + ) + if record is None: + raise HTTPException(status_code=404, detail="Run not found") + return _run_summary(record) diff --git a/backend/app/gateway/routers/mcp_tokens.py b/backend/app/gateway/routers/mcp_tokens.py new file mode 100644 index 00000000..1ce4561d --- /dev/null +++ b/backend/app/gateway/routers/mcp_tokens.py @@ -0,0 +1,123 @@ +"""OAuth-style token and JWKS endpoints for FlowLens MCP.""" + +from __future__ import annotations + +import hmac + +from fastapi import APIRouter, Form, HTTPException, status +from pydantic import BaseModel + +from app.gateway.auth.platform_tokens import ( + AGENTOPS_READ_SCOPES, + SQLITE_ANALYTICS_READ_SCOPES, + TOOLS_DISCOVERY_SCOPE, + PlatformTokenIssuer, + PlatformTokenSettings, + build_platform_jwks, +) +from app.gateway.deps import get_local_provider + +RUNTIME_DELEGATION_GRANT = "urn:deerflow:params:oauth:grant-type:runtime-delegation" +_RESOURCE_SCOPES: dict[str, tuple[str, ...]] = { + "flowlens-inspector": AGENTOPS_READ_SCOPES, + "sqlite-analytics": SQLITE_ANALYTICS_READ_SCOPES, +} + +router = APIRouter(tags=["mcp-auth"]) + + +class PlatformTokenResponse(BaseModel): + """OAuth-compatible bearer token response.""" + + access_token: str + token_type: str = "Bearer" + expires_in: int + scope: str + + +def _authenticate_client( + client_id: str, + client_secret: str, + settings: PlatformTokenSettings, +) -> None: + valid_id = hmac.compare_digest(client_id, settings.client_id) + valid_secret = hmac.compare_digest( + client_secret, + settings.client_secret, + ) + if not (valid_id and valid_secret): + raise HTTPException( + status_code=status.HTTP_401_UNAUTHORIZED, + detail="Invalid MCP client credentials", + headers={"WWW-Authenticate": "Basic"}, + ) + + +@router.get("/.well-known/jwks.json") +async def platform_jwks() -> dict: + """Expose the current FlowLens platform verification key.""" + settings = PlatformTokenSettings.from_env( + require_private_key=False, + ) + return build_platform_jwks(settings) + + +@router.post( + "/api/v1/auth/mcp/token", + response_model=PlatformTokenResponse, +) +async def issue_mcp_token( + grant_type: str = Form(...), + client_id: str = Form(...), + client_secret: str = Form(...), + subject_user_id: str | None = Form(default=None), + origin_run_id: str | None = Form(default=None), + resource: str = Form(default="flowlens-inspector"), +) -> PlatformTokenResponse: + """Issue least-privilege service or user-delegated MCP tokens.""" + settings = PlatformTokenSettings.from_env() + _authenticate_client(client_id, client_secret, settings) + issuer = PlatformTokenIssuer(settings) + + if grant_type == "client_credentials": + return PlatformTokenResponse( + access_token=issuer.issue_service_token(), + expires_in=settings.token_ttl_seconds, + scope=TOOLS_DISCOVERY_SCOPE, + ) + + if grant_type != RUNTIME_DELEGATION_GRANT: + raise HTTPException( + status_code=status.HTTP_400_BAD_REQUEST, + detail="Unsupported grant_type", + ) + if not subject_user_id or not origin_run_id: + raise HTTPException( + status_code=status.HTTP_400_BAD_REQUEST, + detail=("subject_user_id and origin_run_id are required for runtime delegation"), + ) + scopes = _RESOURCE_SCOPES.get(resource) + if scopes is None: + raise HTTPException( + status_code=status.HTTP_400_BAD_REQUEST, + detail="Unsupported MCP resource", + ) + + user = await get_local_provider().get_user(subject_user_id) + if user is None: + raise HTTPException( + status_code=status.HTTP_400_BAD_REQUEST, + detail="Invalid delegation subject", + ) + + return PlatformTokenResponse( + access_token=issuer.issue_delegated_token( + user_id=str(user.id), + tenant_id=user.tenant_id, + token_version=user.token_version, + origin_run_id=origin_run_id, + scopes=scopes, + ), + expires_in=settings.token_ttl_seconds, + scope=" ".join(scopes), + ) diff --git a/backend/app/gateway/routers/thread_runs.py b/backend/app/gateway/routers/thread_runs.py index dfebc6ec..d67a6e53 100644 --- a/backend/app/gateway/routers/thread_runs.py +++ b/backend/app/gateway/routers/thread_runs.py @@ -22,7 +22,16 @@ from sqlalchemy.exc import SQLAlchemyError from app.gateway.authz import require_permission -from app.gateway.deps import get_checkpointer, get_current_user, get_feedback_repo, get_run_event_store, get_run_manager, get_run_store, get_stream_bridge +from app.gateway.deps import ( + get_checkpointer, + get_current_tenant_id, + get_current_user, + get_feedback_repo, + get_run_event_store, + get_run_manager, + get_run_store, + get_stream_bridge, +) from app.gateway.pagination import trim_run_message_page from app.gateway.services import sse_consumer, start_run, wait_for_run_completion from deerflow.models import create_chat_model @@ -146,7 +155,11 @@ def _record_to_response(record: RunRecord) -> RunResponse: async def _get_run_or_404(thread_id: str, run_id: str, request: Request) -> RunRecord: run_mgr = get_run_manager(request) user_id = await get_current_user(request) - record = await run_mgr.get(run_id, user_id=user_id) + record = await run_mgr.get( + run_id, + user_id=user_id, + tenant_id=get_current_tenant_id(request), + ) if record is None or record.thread_id != thread_id: raise HTTPException(status_code=404, detail=f"Run {run_id} not found") return record @@ -248,7 +261,11 @@ async def list_runs(thread_id: str, request: Request) -> list[RunResponse]: """List all runs for a thread.""" run_mgr = get_run_manager(request) user_id = await get_current_user(request) - records = await run_mgr.list_by_thread(thread_id, user_id=user_id) + records = await run_mgr.list_by_thread( + thread_id, + user_id=user_id, + tenant_id=get_current_tenant_id(request), + ) return [_record_to_response(r) for r in records] @@ -277,9 +294,7 @@ async def cancel_run( - wait=false: Return immediately with 202 """ run_mgr = get_run_manager(request) - record = await run_mgr.get(run_id) - if record is None or record.thread_id != thread_id: - raise HTTPException(status_code=404, detail=f"Run {run_id} not found") + record = await _get_run_or_404(thread_id, run_id, request) cancelled = await run_mgr.cancel(run_id, action=action) if not cancelled: @@ -300,9 +315,7 @@ async def cancel_run( async def join_run(thread_id: str, run_id: str, request: Request) -> StreamingResponse: """Join an existing run's SSE stream.""" run_mgr = get_run_manager(request) - record = await run_mgr.get(run_id) - if record is None or record.thread_id != thread_id: - raise HTTPException(status_code=404, detail=f"Run {run_id} not found") + record = await _get_run_or_404(thread_id, run_id, request) if record.store_only: raise HTTPException(status_code=409, detail=f"Run {run_id} is not active on this worker and cannot be streamed") @@ -340,9 +353,7 @@ async def stream_existing_run( remaining buffered events so the client observes a clean shutdown. """ run_mgr = get_run_manager(request) - record = await run_mgr.get(run_id) - if record is None or record.thread_id != thread_id: - raise HTTPException(status_code=404, detail=f"Run {run_id} not found") + record = await _get_run_or_404(thread_id, run_id, request) if record.store_only and action is None: raise HTTPException(status_code=409, detail=f"Run {run_id} is not active on this worker and cannot be streamed") diff --git a/backend/app/gateway/services.py b/backend/app/gateway/services.py index 2c5c01e6..524915d5 100644 --- a/backend/app/gateway/services.py +++ b/backend/app/gateway/services.py @@ -35,6 +35,7 @@ run_agent, ) from deerflow.runtime.runs.naming import resolve_root_run_name +from deerflow.runtime.user_context import DEFAULT_TENANT_ID logger = logging.getLogger(__name__) @@ -182,6 +183,9 @@ def inject_authenticated_user_context(config: dict[str, Any], request: Request) runtime_context = config.setdefault("context", {}) if isinstance(runtime_context, dict): runtime_context["user_id"] = str(user_id) + runtime_context["tenant_id"] = str( + getattr(user, "tenant_id", None) or DEFAULT_TENANT_ID + ) def resolve_agent_factory(assistant_id: str | None): diff --git a/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/__init__.py b/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/__init__.py new file mode 100644 index 00000000..6bc2c6a5 --- /dev/null +++ b/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/__init__.py @@ -0,0 +1 @@ +"""Read-only FlowLens Inspector MCP service.""" diff --git a/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/auth.py b/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/auth.py new file mode 100644 index 00000000..31d40a93 --- /dev/null +++ b/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/auth.py @@ -0,0 +1,30 @@ +"""FastMCP adapter for DeerFlow's signed platform tokens.""" + +from __future__ import annotations + +from mcp.server.auth.provider import AccessToken + +from app.gateway.auth.platform_tokens import ( + PlatformTokenError, + PlatformTokenSettings, + verify_platform_token, +) + + +class PlatformTokenVerifier: + """Verify a DeerFlow platform token at the Inspector MCP boundary.""" + + def __init__(self, settings: PlatformTokenSettings | None = None) -> None: + self._settings = settings + + async def verify_token(self, token: str) -> AccessToken | None: + try: + claims = verify_platform_token(token, self._settings) + except PlatformTokenError: + return None + return AccessToken( + token=token, + client_id=claims.azp, + scopes=claims.scope, + expires_at=int(claims.exp.timestamp()), + ) diff --git a/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/gateway_client.py b/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/gateway_client.py new file mode 100644 index 00000000..28d6887b --- /dev/null +++ b/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/gateway_client.py @@ -0,0 +1,63 @@ +"""Bounded HTTP client for Gateway-owned AgentOps data.""" + +from __future__ import annotations + +from collections.abc import Callable +from typing import Any + +import httpx +from mcp.server.auth.middleware.auth_context import get_access_token + + +class GatewayClient: + """Call only existing read-only Gateway endpoints with one bearer token.""" + + def __init__( + self, + base_url: str, + *, + token_provider: Callable[[], str] | None = None, + ) -> None: + self._base_url = base_url.rstrip("/") + self._token_provider = token_provider + + @property + def request_headers(self) -> dict[str, str]: + """Return a caller-bound header for the current tool invocation.""" + + if self._token_provider is not None: + token = self._token_provider() + else: + access_token = get_access_token() + if access_token is None: + raise RuntimeError("Inspector MCP request is not authenticated") + token = access_token.token + return {"Authorization": f"Bearer {token}"} + + async def health_check(self) -> dict[str, Any]: + return await self._get("/health") + + async def list_runs(self, **params: Any) -> dict[str, Any]: + return await self._get("/api/agentops/runs", params=params) + + async def get_diagnostics(self, thread_id: str, run_id: str) -> dict[str, Any]: + return await self._get( + f"/api/threads/{thread_id}/runs/{run_id}/diagnostics", + ) + + async def get_timeline(self, thread_id: str, run_id: str, **params: Any) -> dict[str, Any]: + return await self._get( + f"/api/threads/{thread_id}/runs/{run_id}/timeline", + params=params, + ) + + async def _get(self, path: str, *, params: dict[str, Any] | None = None) -> dict[str, Any]: + async with httpx.AsyncClient(timeout=5.0) as client: + response = await client.get( + f"{self._base_url}{path}", + headers=self.request_headers, + params=params, + ) + response.raise_for_status() + payload = response.json() + return payload if isinstance(payload, dict) else {"items": payload} diff --git a/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/server.py b/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/server.py new file mode 100644 index 00000000..3a6f5a8a --- /dev/null +++ b/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/server.py @@ -0,0 +1,200 @@ +"""Streamable HTTP MCP server for FlowLens read-only investigation tools.""" + +from __future__ import annotations + +from typing import Any, Protocol + +from mcp.server.auth.middleware.auth_context import get_access_token +from mcp.server.auth.provider import TokenVerifier +from mcp.server.auth.settings import AuthSettings +from mcp.server.fastmcp import FastMCP + +from flowlens_inspector_mcp.auth import PlatformTokenVerifier +from flowlens_inspector_mcp.gateway_client import GatewayClient +from flowlens_inspector_mcp.settings import InspectorSettings + + +class InspectorGateway(Protocol): + async def health_check(self) -> dict[str, Any]: ... + + async def list_runs(self, **params: Any) -> dict[str, Any]: ... + + async def get_diagnostics(self, thread_id: str, run_id: str) -> dict[str, Any]: ... + + async def get_timeline(self, thread_id: str, run_id: str, **params: Any) -> dict[str, Any]: ... + + +def require_scope(scope: str) -> None: + """Require a delegated caller scope after MCP tool discovery succeeds.""" + + access_token = get_access_token() + if access_token is None or scope not in access_token.scopes: + raise PermissionError(f"MCP tool requires delegated scope: {scope}") + + +class FlowLensInspectorTools: + """Transport-independent handlers for the five read-only tools.""" + + def __init__(self, gateway: InspectorGateway) -> None: + self._gateway = gateway + + @staticmethod + def _bounded_limit(limit: int) -> int: + if not 1 <= limit <= 100: + raise ValueError("limit must be between 1 and 100") + return limit + + async def health_check(self) -> dict[str, Any]: + """Return MCP's dependency health without exposing configuration.""" + + return {"schema_version": 1, **await self._gateway.health_check()} + + async def list_runs(self, limit: int = 20, thread_id: str | None = None) -> dict[str, Any]: + """List only runs Gateway authorizes for the bearer identity.""" + + return { + "schema_version": 1, + **await self._gateway.list_runs( + limit=self._bounded_limit(limit), + thread_id=thread_id, + ), + } + + async def get_run_diagnostics(self, thread_id: str, run_id: str) -> dict[str, Any]: + """Return deterministic diagnostics for one authorized run.""" + + return {"schema_version": 1, **await self._gateway.get_diagnostics(thread_id, run_id)} + + async def get_timeline_window( + self, + thread_id: str, + run_id: str, + limit: int = 20, + after_seq: int | None = None, + ) -> dict[str, Any]: + """Return a bounded normalized Timeline window for one run.""" + + return { + "schema_version": 1, + **await self._gateway.get_timeline( + thread_id, + run_id, + limit=self._bounded_limit(limit), + after_seq=after_seq, + ), + } + + async def compare_runs( + self, + base_run_id: str, + base_thread_id: str, + candidate_run_id: str, + candidate_thread_id: str, + ) -> dict[str, Any]: + """Compare two Gateway-authorized diagnostics without invoking an LLM.""" + + base, candidate = await self._gateway.get_diagnostics(base_thread_id, base_run_id), await self._gateway.get_diagnostics(candidate_thread_id, candidate_run_id) + base_metrics = base.get("metrics", {}) + candidate_metrics = candidate.get("metrics", {}) + return { + "schema_version": 1, + "base_run_id": base_run_id, + "candidate_run_id": candidate_run_id, + "duration_delta_ms": int(candidate_metrics.get("duration_ms", 0)) - int(base_metrics.get("duration_ms", 0)), + "token_delta": int(candidate_metrics.get("total_tokens", 0)) - int(base_metrics.get("total_tokens", 0)), + "base_primary_cause": base.get("failure_attribution", {}).get("primary_cause", "UNKNOWN"), + "candidate_primary_cause": candidate.get("failure_attribution", {}).get("primary_cause", "UNKNOWN"), + } + + +def create_server( + gateway: InspectorGateway, + *, + host: str, + port: int, + token_verifier: TokenVerifier | None = None, + auth: AuthSettings | None = None, +) -> FastMCP: + """Register Inspector handlers on a Streamable HTTP FastMCP server.""" + + if (token_verifier is None) != (auth is None): + raise ValueError("token_verifier and auth must be configured together") + + handlers = FlowLensInspectorTools(gateway) + server = FastMCP( + "FlowLens Inspector MCP", + instructions="Use these read-only tools to inspect authorized FlowLens AgentOps runs.", + host=host, + port=port, + streamable_http_path="/mcp", + token_verifier=token_verifier, + auth=auth, + ) + + async def health_check() -> dict[str, Any]: + require_scope("agentops.health.read") + return await handlers.health_check() + + async def list_runs( + limit: int = 20, + thread_id: str | None = None, + ) -> dict[str, Any]: + require_scope("agentops.runs.read") + return await handlers.list_runs(limit=limit, thread_id=thread_id) + + async def get_run_diagnostics(thread_id: str, run_id: str) -> dict[str, Any]: + require_scope("agentops.diagnostics.read") + return await handlers.get_run_diagnostics(thread_id, run_id) + + async def get_timeline_window( + thread_id: str, + run_id: str, + limit: int = 20, + after_seq: int | None = None, + ) -> dict[str, Any]: + require_scope("agentops.timeline.read") + return await handlers.get_timeline_window( + thread_id, + run_id, + limit=limit, + after_seq=after_seq, + ) + + async def compare_runs( + base_run_id: str, + base_thread_id: str, + candidate_run_id: str, + candidate_thread_id: str, + ) -> dict[str, Any]: + require_scope("agentops.runs.compare") + return await handlers.compare_runs( + base_run_id, + base_thread_id, + candidate_run_id, + candidate_thread_id, + ) + + server.tool()(health_check) + server.tool()(list_runs) + server.tool()(get_run_diagnostics) + server.tool()(get_timeline_window) + server.tool()(compare_runs) + return server + + +def main() -> None: + """Start the authenticated Streamable HTTP Inspector MCP service.""" + + settings = InspectorSettings.from_env() + server = create_server( + GatewayClient(settings.gateway_url), + host=settings.host, + port=settings.port, + token_verifier=PlatformTokenVerifier(), + auth=AuthSettings( + issuer_url=settings.issuer_url, + resource_server_url=settings.resource_url, + required_scopes=[], + ), + ) + server.run(transport="streamable-http") diff --git a/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/settings.py b/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/settings.py new file mode 100644 index 00000000..2df2f97c --- /dev/null +++ b/backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/settings.py @@ -0,0 +1,36 @@ +"""Environment-backed settings for the FlowLens Inspector MCP service.""" + +from __future__ import annotations + +import os +from dataclasses import dataclass + + +@dataclass(frozen=True, slots=True) +class InspectorSettings: + """Network configuration only; identity comes from each MCP request.""" + + gateway_url: str + host: str + port: int + issuer_url: str + resource_url: str + + @classmethod + def from_env(cls) -> InspectorSettings: + return cls( + gateway_url=os.getenv( + "FLOWLENS_INSPECTOR_GATEWAY_URL", + "http://gateway:8000", + ).rstrip("/"), + host=os.getenv("FLOWLENS_INSPECTOR_HOST", "0.0.0.0"), + port=int(os.getenv("FLOWLENS_INSPECTOR_PORT", "8012")), + issuer_url=os.getenv( + "FLOWLENS_INSPECTOR_ISSUER_URL", + "http://gateway:8000", + ).rstrip("/"), + resource_url=os.getenv( + "FLOWLENS_INSPECTOR_RESOURCE_URL", + "http://flowlens-inspector:8012", + ).rstrip("/"), + ) diff --git a/backend/packages/flowlens-inspector-mcp/pyproject.toml b/backend/packages/flowlens-inspector-mcp/pyproject.toml new file mode 100644 index 00000000..d2c62f39 --- /dev/null +++ b/backend/packages/flowlens-inspector-mcp/pyproject.toml @@ -0,0 +1,13 @@ +[project] +name = "flowlens-inspector-mcp" +version = "0.1.0" +description = "Read-only FlowLens AgentOps Inspector MCP service" +requires-python = ">=3.12" +dependencies = ["httpx>=0.28.0", "mcp>=1.0.0"] + +[project.scripts] +flowlens-inspector-mcp = "flowlens_inspector_mcp.server:main" + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" diff --git a/backend/packages/harness/deerflow/agents/lead_agent/prompt.py b/backend/packages/harness/deerflow/agents/lead_agent/prompt.py index 2e4d32ce..f9769ae6 100644 --- a/backend/packages/harness/deerflow/agents/lead_agent/prompt.py +++ b/backend/packages/harness/deerflow/agents/lead_agent/prompt.py @@ -449,6 +449,12 @@ def _build_subagent_section(max_concurrent: int, *, app_config: AppConfig | None {deferred_tools_section} + +- If a Tool or MCP operation fails and you use a replacement path, call `report_fallback` only after the replacement succeeds. +- Declare the exact failed and replacement components, whether the user's explicit constraints remain satisfied, and a concrete verification method. +- If the user required the original source, freshness, tool, or write guarantee, set `constraint_satisfied=false`; do not claim the task was completed under the original constraint. + + {subagent_section} diff --git a/backend/packages/harness/deerflow/config/database_config.py b/backend/packages/harness/deerflow/config/database_config.py index 37cfd579..02a946df 100644 --- a/backend/packages/harness/deerflow/config/database_config.py +++ b/backend/packages/harness/deerflow/config/database_config.py @@ -67,16 +67,22 @@ class DatabaseConfig(BaseModel): # -- Derived helpers (not user-configured) -- @property - def _resolved_sqlite_dir(self) -> str: - """Resolve sqlite_dir to an absolute path (relative to CWD).""" + def resolved_sqlite_dir(self) -> str: + """Resolve the configured SQLite directory to an absolute path. + + ``DEER_FLOW_SQLITE_DIR`` is an explicit runtime-only override for + isolated deployments. It keeps a shared config file usable while a + Docker acceptance stack needs a clean database location. + """ from pathlib import Path - return str(Path(self.sqlite_dir).resolve()) + sqlite_dir = os.getenv("DEER_FLOW_SQLITE_DIR", self.sqlite_dir) + return str(Path(sqlite_dir).resolve()) @property def sqlite_path(self) -> str: """Unified SQLite file path shared by checkpointer and app.""" - return os.path.join(self._resolved_sqlite_dir, "deerflow.db") + return os.path.join(self.resolved_sqlite_dir, "deerflow.db") # Backward-compatible aliases @property diff --git a/backend/packages/harness/deerflow/config/extensions_config.py b/backend/packages/harness/deerflow/config/extensions_config.py index 18b8cd5e..a264d316 100644 --- a/backend/packages/harness/deerflow/config/extensions_config.py +++ b/backend/packages/harness/deerflow/config/extensions_config.py @@ -15,10 +15,14 @@ class McpOAuthConfig(BaseModel): enabled: bool = Field(default=True, description="Whether OAuth token injection is enabled") token_url: str = Field(description="OAuth token endpoint URL") - grant_type: Literal["client_credentials", "refresh_token"] = Field( + grant_type: str = Field( default="client_credentials", description="OAuth grant type", ) + discovery_grant_type: Literal["client_credentials", "refresh_token"] | None = Field( + default=None, + description="Optional grant used only to discover protected MCP tool schemas", + ) client_id: str | None = Field(default=None, description="OAuth client ID") client_secret: str | None = Field(default=None, description="OAuth client secret") refresh_token: str | None = Field(default=None, description="OAuth refresh token (for refresh_token grant)") diff --git a/backend/packages/harness/deerflow/mcp/oauth.py b/backend/packages/harness/deerflow/mcp/oauth.py index df0e1ead..112be3e0 100644 --- a/backend/packages/harness/deerflow/mcp/oauth.py +++ b/backend/packages/harness/deerflow/mcp/oauth.py @@ -13,6 +13,7 @@ from deerflow.runtime.diagnostics.emitter import DiagnosticRecorder, find_diagnostic_recorder logger = logging.getLogger(__name__) +RUNTIME_DELEGATION_GRANT = "urn:deerflow:params:oauth:grant-type:runtime-delegation" def _record_oauth_event( @@ -83,11 +84,24 @@ async def get_authorization_header( server_name: str, *, recorder: DiagnosticRecorder | None = None, + runtime: Any | None = None, + for_discovery: bool = False, ) -> str | None: oauth = self._oauth_by_server.get(server_name) if not oauth: return None + grant_type = oauth.discovery_grant_type if for_discovery and oauth.discovery_grant_type else oauth.grant_type + if grant_type == RUNTIME_DELEGATION_GRANT: + fresh = await self._fetch_token( + server_name, + oauth, + grant_type=grant_type, + runtime=runtime, + recorder=recorder, + ) + return f"{fresh.token_type} {fresh.access_token}" + token = self._tokens.get(server_name) if token and not self._is_expiring(token, oauth): return f"{token.token_type} {token.access_token}" @@ -98,7 +112,12 @@ async def get_authorization_header( if token and not self._is_expiring(token, oauth): return f"{token.token_type} {token.access_token}" - fresh = await self._fetch_token(server_name, oauth, recorder=recorder) + fresh = await self._fetch_token( + server_name, + oauth, + grant_type=grant_type, + recorder=recorder, + ) self._tokens[server_name] = fresh logger.info(f"Refreshed OAuth access token for MCP server: {server_name}") return f"{fresh.token_type} {fresh.access_token}" @@ -113,6 +132,8 @@ async def _fetch_token( server_name: str, oauth: McpOAuthConfig, *, + grant_type: str, + runtime: Any | None = None, recorder: DiagnosticRecorder | None = None, ) -> _OAuthToken: started_at = time.monotonic() @@ -121,18 +142,22 @@ async def _fetch_token( recorder, "mcp.oauth.refresh", server_name=server_name, - grant_type=oauth.grant_type, + grant_type=grant_type, status="running", span_id=span_id, ) try: - token = await self._fetch_token_unobserved(oauth) + token = await self._fetch_token_unobserved( + oauth, + grant_type=grant_type, + runtime=runtime, + ) except Exception as exc: _record_oauth_event( recorder, "mcp.oauth.error", server_name=server_name, - grant_type=oauth.grant_type, + grant_type=grant_type, status="error", span_id=span_id, duration_ms=int((time.monotonic() - started_at) * 1000), @@ -143,18 +168,24 @@ async def _fetch_token( recorder, "mcp.oauth.refresh", server_name=server_name, - grant_type=oauth.grant_type, + grant_type=grant_type, status="success", span_id=span_id, duration_ms=int((time.monotonic() - started_at) * 1000), ) return token - async def _fetch_token_unobserved(self, oauth: McpOAuthConfig) -> _OAuthToken: + async def _fetch_token_unobserved( + self, + oauth: McpOAuthConfig, + *, + grant_type: str, + runtime: Any | None = None, + ) -> _OAuthToken: import httpx # pyright: ignore[reportMissingImports] data: dict[str, str] = { - "grant_type": oauth.grant_type, + "grant_type": grant_type, **oauth.extra_token_params, } @@ -163,12 +194,12 @@ async def _fetch_token_unobserved(self, oauth: McpOAuthConfig) -> _OAuthToken: if oauth.audience: data["audience"] = oauth.audience - if oauth.grant_type == "client_credentials": + if grant_type == "client_credentials": if not oauth.client_id or not oauth.client_secret: raise ValueError("OAuth client_credentials requires client_id and client_secret") data["client_id"] = oauth.client_id data["client_secret"] = oauth.client_secret - elif oauth.grant_type == "refresh_token": + elif grant_type == "refresh_token": if not oauth.refresh_token: raise ValueError("OAuth refresh_token grant requires refresh_token") data["refresh_token"] = oauth.refresh_token @@ -176,8 +207,26 @@ async def _fetch_token_unobserved(self, oauth: McpOAuthConfig) -> _OAuthToken: data["client_id"] = oauth.client_id if oauth.client_secret: data["client_secret"] = oauth.client_secret + elif grant_type == RUNTIME_DELEGATION_GRANT: + if not oauth.client_id or not oauth.client_secret: + raise ValueError("Runtime delegation requires client_id and client_secret") + context = getattr(runtime, "context", None) + if not isinstance(context, dict): + raise ValueError("Runtime delegation requires an MCP tool runtime") + user_id = context.get("user_id") + run_id = context.get("run_id") + if not user_id or not run_id: + raise ValueError("Runtime delegation requires user_id and run_id") + data.update( + { + "client_id": oauth.client_id, + "client_secret": oauth.client_secret, + "subject_user_id": str(user_id), + "origin_run_id": str(run_id), + } + ) else: - raise ValueError(f"Unsupported OAuth grant type: {oauth.grant_type}") + raise ValueError(f"Unsupported OAuth grant type: {grant_type}") async with httpx.AsyncClient(timeout=15.0) as client: response = await client.post(oauth.token_url, data=data) @@ -208,7 +257,11 @@ def build_oauth_tool_interceptor(extensions_config: ExtensionsConfig) -> Any | N async def oauth_interceptor(request: Any, handler: Any) -> Any: recorder = find_diagnostic_recorder(getattr(request, "runtime", None)) - header = await token_manager.get_authorization_header(request.server_name, recorder=recorder) + header = await token_manager.get_authorization_header( + request.server_name, + recorder=recorder, + runtime=getattr(request, "runtime", None), + ) if not header: return await handler(request) @@ -227,6 +280,12 @@ async def get_initial_oauth_headers(extensions_config: ExtensionsConfig) -> dict headers: dict[str, str] = {} for server_name in token_manager.oauth_server_names(): - headers[server_name] = await token_manager.get_authorization_header(server_name) or "" + headers[server_name] = ( + await token_manager.get_authorization_header( + server_name, + for_discovery=True, + ) + or "" + ) return {name: value for name, value in headers.items() if value} diff --git a/backend/packages/harness/deerflow/persistence/engine.py b/backend/packages/harness/deerflow/persistence/engine.py index 127ee73f..1523707a 100644 --- a/backend/packages/harness/deerflow/persistence/engine.py +++ b/backend/packages/harness/deerflow/persistence/engine.py @@ -175,7 +175,7 @@ async def init_engine_from_config(config) -> None: url=config.app_sqlalchemy_url, echo=config.echo_sql, pool_size=config.pool_size, - sqlite_dir=config.sqlite_dir if config.backend == "sqlite" else "", + sqlite_dir=config.resolved_sqlite_dir if config.backend == "sqlite" else "", ) diff --git a/backend/packages/harness/deerflow/persistence/migrations/versions/20260728_0001_flowlens_mcp_tenant.py b/backend/packages/harness/deerflow/persistence/migrations/versions/20260728_0001_flowlens_mcp_tenant.py new file mode 100644 index 00000000..9146dde5 --- /dev/null +++ b/backend/packages/harness/deerflow/persistence/migrations/versions/20260728_0001_flowlens_mcp_tenant.py @@ -0,0 +1,93 @@ +"""Add tenant ownership to users and runs. + +Revision ID: 20260728_0001 +Revises: +Create Date: 2026-07-28 +""" + +from __future__ import annotations + +from collections.abc import Mapping +from typing import Any + +import sqlalchemy as sa +from alembic import op + +revision = "20260728_0001" +down_revision = None +branch_labels = None +depends_on = None + +_DEFAULT_TENANT_ID = "default" +_INDEX_NAME = "ix_runs_tenant_user_created" + + +def _columns(table_name: str) -> dict[str, Mapping[str, Any]]: + inspector = sa.inspect(op.get_bind()) + return { + column["name"]: column + for column in inspector.get_columns(table_name) + } + + +def _index_names(table_name: str) -> set[str]: + inspector = sa.inspect(op.get_bind()) + return { + index["name"] + for index in inspector.get_indexes(table_name) + if index.get("name") + } + + +def _ensure_tenant_column(table_name: str) -> None: + columns = _columns(table_name) + if "tenant_id" not in columns: + op.add_column( + table_name, + sa.Column( + "tenant_id", + sa.String(length=64), + nullable=False, + server_default=sa.text(f"'{_DEFAULT_TENANT_ID}'"), + ), + ) + return + + op.execute( + sa.text( + f"UPDATE {table_name} " + "SET tenant_id = :tenant_id " + "WHERE tenant_id IS NULL OR tenant_id = ''" + ).bindparams(tenant_id=_DEFAULT_TENANT_ID) + ) + if columns["tenant_id"].get("nullable", True): + with op.batch_alter_table(table_name) as batch_op: + batch_op.alter_column( + "tenant_id", + existing_type=sa.String(length=64), + nullable=False, + server_default=sa.text(f"'{_DEFAULT_TENANT_ID}'"), + ) + + +def upgrade() -> None: + _ensure_tenant_column("users") + _ensure_tenant_column("runs") + + if _INDEX_NAME not in _index_names("runs"): + op.create_index( + _INDEX_NAME, + "runs", + ["tenant_id", "user_id", "created_at"], + unique=False, + ) + + +def downgrade() -> None: + if _INDEX_NAME in _index_names("runs"): + op.drop_index(_INDEX_NAME, table_name="runs") + + for table_name in ("runs", "users"): + if "tenant_id" in _columns(table_name): + with op.batch_alter_table(table_name) as batch_op: + batch_op.drop_column("tenant_id") diff --git a/backend/packages/harness/deerflow/persistence/run/model.py b/backend/packages/harness/deerflow/persistence/run/model.py index d0dfe408..d6396303 100644 --- a/backend/packages/harness/deerflow/persistence/run/model.py +++ b/backend/packages/harness/deerflow/persistence/run/model.py @@ -17,6 +17,12 @@ class RunRow(Base): thread_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True) assistant_id: Mapped[str | None] = mapped_column(String(128)) user_id: Mapped[str | None] = mapped_column(String(64), index=True) + tenant_id: Mapped[str] = mapped_column( + String(64), + nullable=False, + default="default", + server_default="default", + ) status: Mapped[str] = mapped_column(String(20), default="pending") # "pending" | "running" | "success" | "error" | "timeout" | "interrupted" @@ -46,4 +52,12 @@ class RunRow(Base): created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC)) updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC)) - __table_args__ = (Index("ix_runs_thread_status", "thread_id", "status"),) + __table_args__ = ( + Index("ix_runs_thread_status", "thread_id", "status"), + Index( + "ix_runs_tenant_user_created", + "tenant_id", + "user_id", + "created_at", + ), + ) diff --git a/backend/packages/harness/deerflow/persistence/run/sql.py b/backend/packages/harness/deerflow/persistence/run/sql.py index b0b0bf97..8bc01a37 100644 --- a/backend/packages/harness/deerflow/persistence/run/sql.py +++ b/backend/packages/harness/deerflow/persistence/run/sql.py @@ -17,7 +17,12 @@ from deerflow.persistence.run.model import RunRow from deerflow.runtime.runs.cursor import decode_run_cursor from deerflow.runtime.runs.store.base import RunStore -from deerflow.runtime.user_context import AUTO, _AutoSentinel, resolve_user_id +from deerflow.runtime.user_context import ( + AUTO, + _AutoSentinel, + resolve_tenant_id, + resolve_user_id, +) from deerflow.utils.time import coerce_iso @@ -86,6 +91,7 @@ async def put( thread_id, assistant_id=None, user_id: str | None | _AutoSentinel = AUTO, + tenant_id: str | None | _AutoSentinel = AUTO, model_name: str | None = None, status="pending", multitask_strategy="reject", @@ -102,12 +108,17 @@ async def put( commit from turning the retry into a primary-key failure. """ resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.put") + resolved_tenant_id = resolve_tenant_id( + tenant_id, + method_name="RunRepository.put", + ) now = datetime.now(UTC) created = datetime.fromisoformat(created_at) if created_at else now values = { "thread_id": thread_id, "assistant_id": assistant_id, "user_id": resolved_user_id, + "tenant_id": resolved_tenant_id, "model_name": self._normalize_model_name(model_name), "status": status, "multitask_strategy": multitask_strategy, @@ -131,14 +142,24 @@ async def get( run_id, *, user_id: str | None | _AutoSentinel = AUTO, + tenant_id: str | None | _AutoSentinel = AUTO, ): resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.get") + resolved_tenant_id = resolve_tenant_id( + tenant_id, + method_name="RunRepository.get", + ) async with self._sf() as session: row = await session.get(RunRow, run_id) if row is None: return None if resolved_user_id is not None and row.user_id != resolved_user_id: return None + if ( + resolved_tenant_id is not None + and row.tenant_id != resolved_tenant_id + ): + return None return self._row_to_dict(row) async def list_by_thread( @@ -146,12 +167,19 @@ async def list_by_thread( thread_id, *, user_id: str | None | _AutoSentinel = AUTO, + tenant_id: str | None | _AutoSentinel = AUTO, limit=100, ): resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.list_by_thread") + resolved_tenant_id = resolve_tenant_id( + tenant_id, + method_name="RunRepository.list_by_thread", + ) stmt = select(RunRow).where(RunRow.thread_id == thread_id) if resolved_user_id is not None: stmt = stmt.where(RunRow.user_id == resolved_user_id) + if resolved_tenant_id is not None: + stmt = stmt.where(RunRow.tenant_id == resolved_tenant_id) stmt = stmt.order_by(RunRow.created_at.desc()).limit(limit) async with self._sf() as session: result = await session.execute(stmt) @@ -161,6 +189,8 @@ async def list_recent( self, *, user_id: str | None | _AutoSentinel = AUTO, + tenant_id: str | None | _AutoSentinel = AUTO, + thread_id: str | None = None, limit: int = 50, cursor: str | None = None, status: str | None = None, @@ -168,9 +198,17 @@ async def list_recent( started_before: str | None = None, ) -> list[dict[str, Any]]: resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.list_recent") + resolved_tenant_id = resolve_tenant_id( + tenant_id, + method_name="RunRepository.list_recent", + ) stmt = select(RunRow) if resolved_user_id is not None: stmt = stmt.where(RunRow.user_id == resolved_user_id) + if resolved_tenant_id is not None: + stmt = stmt.where(RunRow.tenant_id == resolved_tenant_id) + if thread_id is not None: + stmt = stmt.where(RunRow.thread_id == thread_id) if status is not None: stmt = stmt.where(RunRow.status == status) if started_after is not None: @@ -210,14 +248,24 @@ async def delete( run_id, *, user_id: str | None | _AutoSentinel = AUTO, + tenant_id: str | None | _AutoSentinel = AUTO, ): resolved_user_id = resolve_user_id(user_id, method_name="RunRepository.delete") + resolved_tenant_id = resolve_tenant_id( + tenant_id, + method_name="RunRepository.delete", + ) async with self._sf() as session: row = await session.get(RunRow, run_id) if row is None: return if resolved_user_id is not None and row.user_id != resolved_user_id: return + if ( + resolved_tenant_id is not None + and row.tenant_id != resolved_tenant_id + ): + return await session.delete(row) await session.commit() diff --git a/backend/packages/harness/deerflow/persistence/user/model.py b/backend/packages/harness/deerflow/persistence/user/model.py index 130d4bfc..53fc5225 100644 --- a/backend/packages/harness/deerflow/persistence/user/model.py +++ b/backend/packages/harness/deerflow/persistence/user/model.py @@ -24,6 +24,12 @@ class UserRow(Base): # UUIDs are stored as 36-char strings for cross-backend portability. id: Mapped[str] = mapped_column(String(36), primary_key=True) + tenant_id: Mapped[str] = mapped_column( + String(64), + nullable=False, + default="default", + server_default="default", + ) email: Mapped[str] = mapped_column(String(320), unique=True, nullable=False, index=True) password_hash: Mapped[str | None] = mapped_column(String(128), nullable=True) diff --git a/backend/packages/harness/deerflow/runtime/diagnostics/attribution.py b/backend/packages/harness/deerflow/runtime/diagnostics/attribution.py index 98cae860..88867753 100644 --- a/backend/packages/harness/deerflow/runtime/diagnostics/attribution.py +++ b/backend/packages/harness/deerflow/runtime/diagnostics/attribution.py @@ -41,6 +41,67 @@ def _find(timeline: list[dict[str, Any]], predicate) -> dict[str, Any] | None: return next((item for item in timeline if predicate(item)), None) +def _component_name(item: dict[str, Any]) -> str: + """Return the normalized component identity used by fallback declarations.""" + + component = str(item.get("component") or "").strip() + if component and component not in {"mcp", "tool", "runtime"}: + return component + + metadata = item.get("metadata") if isinstance(item.get("metadata"), dict) else {} + if item.get("phase") == "mcp": + server_name = str(metadata.get("server_name") or "mcp") + tool_name = metadata.get("tool_name") + return f"{server_name}.{tool_name}" if tool_name else server_name + if item.get("phase") == "tool": + return str(metadata.get("tool_name") or "tool") + return component + + +def _has_verified_fallback(timeline: list[dict[str, Any]], error_index: int) -> bool: + """Require an explicit declaration tied to a successful fallback tool.""" + + error = timeline[error_index] + failed_component = _component_name(error) + for declaration_index, declaration in enumerate(timeline[error_index + 1 :], start=error_index + 1): + if declaration.get("event_type") != "runtime.fallback.declared": + continue + metadata = declaration.get("metadata") if isinstance(declaration.get("metadata"), dict) else {} + if metadata.get("failed_component") != failed_component: + continue + if metadata.get("constraint_satisfied") is not True or metadata.get("result_verified") is not True: + continue + fallback_component = str(metadata.get("fallback_component") or "").strip() + verification = str(metadata.get("verification") or "").strip() + if not fallback_component or not verification: + continue + if any( + item.get("status") == "success" and _component_name(item) == fallback_component + for item in timeline[error_index + 1 : declaration_index] + ): + return True + return False + + +def _find_actionable_error(timeline: list[dict[str, Any]], predicate) -> dict[str, Any] | None: + """Ignore retried errors and explicitly verified, constraint-safe fallbacks.""" + + for index, item in enumerate(timeline): + if not predicate(item): + continue + phase = item.get("phase") + component = item.get("component") + retried = any( + later.get("phase") == phase + and later.get("component") == component + and later.get("status") == "success" + for later in timeline[index + 1 :] + ) + if not retried and not _has_verified_fallback(timeline, index): + return item + return None + + def _contains(item: dict[str, Any], *needles: str) -> bool: haystack = " ".join( ( @@ -80,9 +141,6 @@ def attribute_failure(record: Any | None, timeline: list[dict[str, Any]]) -> dic """Classify primary and contributing causes using deterministic rules.""" status = _status(record) - if status == "success": - return _response([_match("NONE", "runtime", 1.0, "Run completed successfully.", [], "No action required.")]) - matches: list[CauseMatch] = [] abort_action = getattr(record, "abort_action", None) if record is not None else None error = str(getattr(record, "error", "") or "") @@ -139,7 +197,7 @@ def attribute_failure(record: Any | None, timeline: list[dict[str, Any]]) -> dic ) ) - mcp_auth = _find( + mcp_auth = _find_actionable_error( timeline, lambda item: item.get("phase") == "mcp" and item.get("status") == "error" and _contains(item, "auth", "oauth", "unauthorized", "401"), ) @@ -155,7 +213,7 @@ def attribute_failure(record: Any | None, timeline: list[dict[str, Any]]) -> dic ) ) - mcp_session = _find( + mcp_session = _find_actionable_error( timeline, lambda item: item.get("phase") == "mcp" and item.get("status") == "error" and _contains(item, "session", "transport", "connect"), ) @@ -171,7 +229,7 @@ def attribute_failure(record: Any | None, timeline: list[dict[str, Any]]) -> dic ) ) - memory_error = _find( + memory_error = _find_actionable_error( timeline, lambda item: item.get("phase") == "memory" and item.get("status") == "error", ) @@ -203,7 +261,7 @@ def attribute_failure(record: Any | None, timeline: list[dict[str, Any]]) -> dic ) ) else: - subagent_error = _find( + subagent_error = _find_actionable_error( timeline, lambda item: item.get("phase") == "subagent" and item.get("status") == "error", ) @@ -219,7 +277,7 @@ def attribute_failure(record: Any | None, timeline: list[dict[str, Any]]) -> dic ) ) - tool_error = _find( + tool_error = _find_actionable_error( timeline, lambda item: item.get("phase") == "tool" and item.get("status") == "error", ) @@ -235,7 +293,7 @@ def attribute_failure(record: Any | None, timeline: list[dict[str, Any]]) -> dic ) ) - mcp_tool_error = _find( + mcp_tool_error = _find_actionable_error( timeline, lambda item: item.get("phase") == "mcp" and item.get("status") == "error" and _contains(item, "tool"), ) @@ -251,7 +309,7 @@ def attribute_failure(record: Any | None, timeline: list[dict[str, Any]]) -> dic ) ) - model_error = _find( + model_error = _find_actionable_error( timeline, lambda item: item.get("phase") == "model" and item.get("status") == "error", ) @@ -280,5 +338,8 @@ def attribute_failure(record: Any | None, timeline: list[dict[str, Any]]) -> dic ) if not matches: - matches.append(_match("NONE", "runtime", 0.8, "No failure was detected.", [], "No action required.")) + if status == "success": + matches.append(_match("NONE", "runtime", 1.0, "Run completed successfully.", [], "No action required.")) + else: + matches.append(_match("NONE", "runtime", 0.8, "No failure was detected.", [], "No action required.")) return _response(matches) diff --git a/backend/packages/harness/deerflow/runtime/runs/manager.py b/backend/packages/harness/deerflow/runtime/runs/manager.py index 67d1daec..eef800a3 100644 --- a/backend/packages/harness/deerflow/runtime/runs/manager.py +++ b/backend/packages/harness/deerflow/runtime/runs/manager.py @@ -10,6 +10,12 @@ from dataclasses import dataclass, field from typing import TYPE_CHECKING, Any +from deerflow.runtime.user_context import ( + DEFAULT_TENANT_ID, + DEFAULT_USER_ID, + get_effective_tenant_id, + get_effective_user_id, +) from deerflow.utils.time import now_iso as _now_iso from .schemas import DisconnectMode, RunStatus @@ -80,6 +86,8 @@ class RunRecord: assistant_id: str | None status: RunStatus on_disconnect: DisconnectMode + user_id: str = DEFAULT_USER_ID + tenant_id: str = DEFAULT_TENANT_ID multitask_strategy: str = "reject" metadata: dict = field(default_factory=dict) kwargs: dict = field(default_factory=dict) @@ -127,6 +135,8 @@ def _store_put_payload(record: RunRecord, *, error: str | None = None) -> dict[s return { "thread_id": record.thread_id, "assistant_id": record.assistant_id, + "user_id": record.user_id, + "tenant_id": record.tenant_id, "status": record.status.value, "multitask_strategy": record.multitask_strategy, "metadata": record.metadata or {}, @@ -236,6 +246,8 @@ def _record_from_store(row: dict[str, Any]) -> RunRecord: assistant_id=row.get("assistant_id"), status=RunStatus(row.get("status") or RunStatus.pending.value), on_disconnect=DisconnectMode(row.get("on_disconnect") or DisconnectMode.cancel.value), + user_id=str(row.get("user_id") or DEFAULT_USER_ID), + tenant_id=str(row.get("tenant_id") or DEFAULT_TENANT_ID), multitask_strategy=row.get("multitask_strategy") or "reject", metadata=row.get("metadata") or {}, kwargs=row.get("kwargs") or {}, @@ -330,6 +342,8 @@ async def create( assistant_id=assistant_id, status=RunStatus.pending, on_disconnect=on_disconnect, + user_id=get_effective_user_id(), + tenant_id=get_effective_tenant_id(), multitask_strategy=multitask_strategy, metadata=metadata or {}, kwargs=kwargs or {}, @@ -352,21 +366,36 @@ async def create( logger.info("Run created: run_id=%s thread_id=%s", run_id, thread_id) return record - async def get(self, run_id: str, *, user_id: str | None = None) -> RunRecord | None: + async def get( + self, + run_id: str, + *, + user_id: str | None = None, + tenant_id: str | None = None, + ) -> RunRecord | None: """Return a run record by ID, or ``None``. Args: run_id: The run ID to look up. - user_id: Optional user ID for permission filtering when hydrating from store. + user_id: Optional user ID for permission filtering. + tenant_id: Optional tenant ID for permission filtering. """ async with self._lock: record = self._runs.get(run_id) if record is not None: + if user_id is not None and record.user_id != user_id: + return None + if tenant_id is not None and record.tenant_id != tenant_id: + return None return record if self._store is None: return None try: - row = await self._store.get(run_id, user_id=user_id) + row = await self._store.get( + run_id, + user_id=user_id, + tenant_id=tenant_id, + ) except Exception: logger.warning("Failed to hydrate run %s from store", run_id, exc_info=True) return None @@ -375,6 +404,10 @@ async def get(self, run_id: str, *, user_id: str | None = None) -> RunRecord | N async with self._lock: record = self._runs.get(run_id) if record is not None: + if user_id is not None and record.user_id != user_id: + return None + if tenant_id is not None and record.tenant_id != tenant_id: + return None return record if row is None: return None @@ -384,14 +417,31 @@ async def get(self, run_id: str, *, user_id: str | None = None) -> RunRecord | N logger.warning("Failed to map store row for run %s", run_id, exc_info=True) return None - async def aget(self, run_id: str, *, user_id: str | None = None) -> RunRecord | None: + async def aget( + self, + run_id: str, + *, + user_id: str | None = None, + tenant_id: str | None = None, + ) -> RunRecord | None: """Return a run record by ID, checking the persistent store as fallback. Alias for :meth:`get` for backward compatibility. """ - return await self.get(run_id, user_id=user_id) + return await self.get( + run_id, + user_id=user_id, + tenant_id=tenant_id, + ) - async def list_by_thread(self, thread_id: str, *, user_id: str | None = None, limit: int = 100) -> list[RunRecord]: + async def list_by_thread( + self, + thread_id: str, + *, + user_id: str | None = None, + tenant_id: str | None = None, + limit: int = 100, + ) -> list[RunRecord]: """Return runs for a given thread, newest first, at most ``limit`` records. In-memory runs take precedence only when the same ``run_id`` exists in both @@ -405,13 +455,24 @@ async def list_by_thread(self, thread_id: str, *, user_id: str | None = None, li """ async with self._lock: # Dict insertion order gives deterministic results when timestamps tie. - memory_records = [r for r in self._runs.values() if r.thread_id == thread_id] + memory_records = [ + record + for record in self._runs.values() + if record.thread_id == thread_id + and (user_id is None or record.user_id == user_id) + and (tenant_id is None or record.tenant_id == tenant_id) + ] if self._store is None: return sorted(memory_records, key=lambda r: r.created_at, reverse=True)[:limit] records_by_id = {record.run_id: record for record in memory_records} store_limit = max(0, limit - len(memory_records)) try: - rows = await self._store.list_by_thread(thread_id, user_id=user_id, limit=store_limit) + rows = await self._store.list_by_thread( + thread_id, + user_id=user_id, + tenant_id=tenant_id, + limit=store_limit, + ) except Exception: logger.warning("Failed to hydrate runs for thread %s from store", thread_id, exc_info=True) return sorted(memory_records, key=lambda r: r.created_at, reverse=True)[:limit] @@ -428,6 +489,8 @@ async def list_recent( self, *, user_id: str | None, + tenant_id: str | None = None, + thread_id: str | None = None, limit: int = 50, cursor: str | None = None, status: str | None = None, @@ -440,6 +503,8 @@ async def list_recent( return [] rows = await self._store.list_recent( user_id=user_id, + tenant_id=tenant_id, + thread_id=thread_id, limit=limit, cursor=cursor, status=status, @@ -573,6 +638,8 @@ async def create_or_reject( assistant_id=assistant_id, status=RunStatus.pending, on_disconnect=on_disconnect, + user_id=get_effective_user_id(), + tenant_id=get_effective_tenant_id(), multitask_strategy=multitask_strategy, metadata=metadata or {}, kwargs=kwargs or {}, diff --git a/backend/packages/harness/deerflow/runtime/runs/store/base.py b/backend/packages/harness/deerflow/runtime/runs/store/base.py index 05bf0caf..bf5015e1 100644 --- a/backend/packages/harness/deerflow/runtime/runs/store/base.py +++ b/backend/packages/harness/deerflow/runtime/runs/store/base.py @@ -23,6 +23,7 @@ async def put( thread_id: str, assistant_id: str | None = None, user_id: str | None = None, + tenant_id: str | None = None, model_name: str | None = None, status: str = "pending", multitask_strategy: str = "reject", @@ -39,6 +40,7 @@ async def get( run_id: str, *, user_id: str | None = None, + tenant_id: str | None = None, ) -> dict[str, Any] | None: pass @@ -48,6 +50,7 @@ async def list_by_thread( thread_id: str, *, user_id: str | None = None, + tenant_id: str | None = None, limit: int = 100, ) -> list[dict[str, Any]]: pass @@ -56,6 +59,8 @@ async def list_recent( self, *, user_id: str | None, + tenant_id: str | None = None, + thread_id: str | None = None, limit: int = 50, cursor: str | None = None, status: str | None = None, diff --git a/backend/packages/harness/deerflow/runtime/runs/store/memory.py b/backend/packages/harness/deerflow/runtime/runs/store/memory.py index 7985d89a..20b21502 100644 --- a/backend/packages/harness/deerflow/runtime/runs/store/memory.py +++ b/backend/packages/harness/deerflow/runtime/runs/store/memory.py @@ -23,6 +23,7 @@ async def put( thread_id, assistant_id=None, user_id=None, + tenant_id=None, model_name=None, status="pending", multitask_strategy="reject", @@ -37,6 +38,7 @@ async def put( "thread_id": thread_id, "assistant_id": assistant_id, "user_id": user_id, + "tenant_id": tenant_id, "model_name": model_name, "status": status, "multitask_strategy": multitask_strategy, @@ -47,16 +49,31 @@ async def put( "updated_at": now, } - async def get(self, run_id, *, user_id=None): + async def get(self, run_id, *, user_id=None, tenant_id=None): run = self._runs.get(run_id) if run is None: return None if user_id is not None and run.get("user_id") != user_id: return None + if tenant_id is not None and run.get("tenant_id") != tenant_id: + return None return run - async def list_by_thread(self, thread_id, *, user_id=None, limit=100): - results = [r for r in self._runs.values() if r["thread_id"] == thread_id and (user_id is None or r.get("user_id") == user_id)] + async def list_by_thread( + self, + thread_id, + *, + user_id=None, + tenant_id=None, + limit=100, + ): + results = [ + run + for run in self._runs.values() + if run["thread_id"] == thread_id + and (user_id is None or run.get("user_id") == user_id) + and (tenant_id is None or run.get("tenant_id") == tenant_id) + ] results.sort(key=lambda r: r["created_at"], reverse=True) return results[:limit] @@ -64,13 +81,21 @@ async def list_recent( self, *, user_id, + tenant_id=None, + thread_id=None, limit=50, cursor=None, status=None, started_after=None, started_before=None, ): - results = [run for run in self._runs.values() if user_id is None or run.get("user_id") == user_id] + results = [ + run + for run in self._runs.values() + if (user_id is None or run.get("user_id") == user_id) + and (tenant_id is None or run.get("tenant_id") == tenant_id) + and (thread_id is None or run.get("thread_id") == thread_id) + ] if status is not None: results = [run for run in results if run.get("status") == status] if started_after is not None: diff --git a/backend/packages/harness/deerflow/runtime/runs/worker.py b/backend/packages/harness/deerflow/runtime/runs/worker.py index e0c1aefa..cbb25835 100644 --- a/backend/packages/harness/deerflow/runtime/runs/worker.py +++ b/backend/packages/harness/deerflow/runtime/runs/worker.py @@ -220,6 +220,7 @@ async def run_agent( # manually here because we drive the graph through ``agent.astream(config=...)`` # without passing the official ``context=`` parameter. runtime_ctx = _build_runtime_context(thread_id, run_id, config.get("context"), ctx.app_config) + runtime_ctx.setdefault("user_id", get_effective_user_id()) # Expose the run-scoped journal under a sentinel key so middleware can # write audit events (e.g. SafetyFinishReasonMiddleware recording # suppressed tool calls). Double-underscore prefix marks it as a @@ -252,10 +253,13 @@ async def run_agent( # the agent name that this run will actually execute. config.setdefault("run_name", resolve_root_run_name(config, record.assistant_id)) runnable_config = RunnableConfig(**config) + # Agent construction can perform MCP discovery. HTTP MCP discovery may + # request a token from this Gateway, so keep the server event loop free + # to serve that request instead of synchronously blocking it here. if ctx.app_config is not None and _agent_factory_supports_app_config(agent_factory): - agent = agent_factory(config=runnable_config, app_config=ctx.app_config) + agent = await asyncio.to_thread(agent_factory, config=runnable_config, app_config=ctx.app_config) else: - agent = agent_factory(config=runnable_config) + agent = await asyncio.to_thread(agent_factory, config=runnable_config) # Capture the effective (resolved) model name from the agent's metadata. # _resolve_model_name in agent.py may return the default model if the diff --git a/backend/packages/harness/deerflow/runtime/user_context.py b/backend/packages/harness/deerflow/runtime/user_context.py index cfbb68c9..a21f19fc 100644 --- a/backend/packages/harness/deerflow/runtime/user_context.py +++ b/backend/packages/harness/deerflow/runtime/user_context.py @@ -95,6 +95,7 @@ def require_current_user() -> CurrentUser: # --------------------------------------------------------------------------- DEFAULT_USER_ID: Final[str] = "default" +DEFAULT_TENANT_ID: Final[str] = "default" def get_effective_user_id() -> str: @@ -109,6 +110,13 @@ def get_effective_user_id() -> str: return str(user.id) +def get_effective_tenant_id() -> str: + """Return the current tenant id, defaulting legacy users to ``default``.""" + user = _current_user.get() + tenant_id = getattr(user, "tenant_id", None) if user is not None else None + return str(tenant_id or DEFAULT_TENANT_ID) + + def resolve_runtime_user_id(runtime: object | None) -> str: """Single source of truth for a tool/middleware's effective user_id. @@ -137,6 +145,16 @@ def resolve_runtime_user_id(runtime: object | None) -> str: return get_effective_user_id() +def resolve_runtime_tenant_id(runtime: object | None) -> str: + """Resolve tenant identity from runtime context, then request context.""" + context = getattr(runtime, "context", None) + if isinstance(context, dict): + tenant_id = context.get("tenant_id") + if tenant_id: + return str(tenant_id) + return get_effective_tenant_id() + + # --------------------------------------------------------------------------- # Sentinel-based user_id resolution # --------------------------------------------------------------------------- @@ -193,3 +211,21 @@ def resolve_user_id( # rather than ripple a type change through every caller. return str(user.id) return value + + +def resolve_tenant_id( + value: str | None | _AutoSentinel, + *, + method_name: str = "repository method", +) -> str | None: + """Resolve a repository tenant filter with the same semantics as user_id.""" + if isinstance(value, _AutoSentinel): + user = _current_user.get() + if user is None: + raise RuntimeError( + f"{method_name} called with tenant_id=AUTO but no user context is set; " + "pass an explicit tenant_id, set the contextvar via auth middleware, " + "or opt out with tenant_id=None for migration/CLI paths." + ) + return str(getattr(user, "tenant_id", None) or DEFAULT_TENANT_ID) + return value diff --git a/backend/packages/harness/deerflow/tools/builtins/__init__.py b/backend/packages/harness/deerflow/tools/builtins/__init__.py index 4b1afa03..a28a155b 100644 --- a/backend/packages/harness/deerflow/tools/builtins/__init__.py +++ b/backend/packages/harness/deerflow/tools/builtins/__init__.py @@ -1,5 +1,6 @@ from .clarification_tool import ask_clarification_tool from .present_file_tool import present_file_tool +from .report_fallback_tool import report_fallback from .setup_agent_tool import setup_agent from .task_tool import task_tool from .update_agent_tool import update_agent @@ -9,6 +10,7 @@ "setup_agent", "update_agent", "present_file_tool", + "report_fallback", "ask_clarification_tool", "view_image_tool", "task_tool", diff --git a/backend/packages/harness/deerflow/tools/builtins/report_fallback_tool.py b/backend/packages/harness/deerflow/tools/builtins/report_fallback_tool.py new file mode 100644 index 00000000..35a87827 --- /dev/null +++ b/backend/packages/harness/deerflow/tools/builtins/report_fallback_tool.py @@ -0,0 +1,62 @@ +"""Structured, auditable fallback reporting for AgentOps diagnostics.""" + +from __future__ import annotations + +from langchain.tools import tool + +from deerflow.runtime.diagnostics.emitter import find_diagnostic_recorder +from deerflow.tools.types import Runtime + + +@tool("report_fallback", parse_docstring=True) +def report_fallback( + runtime: Runtime, + failed_component: str, + fallback_component: str, + constraint_satisfied: bool, + result_verified: bool, + verification: str, + reason: str, +) -> str: + """Declare a verified fallback after a tool or MCP operation failed. + + Call this only after the replacement component has completed successfully. + Set constraint_satisfied to false when the user required the original tool, + source, freshness, or write guarantee. Set result_verified to true only + when verification is concrete and describe that check in verification. + + Args: + failed_component: Exact failed component shown in the execution trace. + fallback_component: Exact replacement component that completed successfully. + constraint_satisfied: Whether the fallback still meets the user's explicit constraints. + result_verified: Whether the fallback result was concretely verified. + verification: Privacy-safe description of the verification performed. + reason: Privacy-safe reason the original component was replaced. + """ + + failed_component = failed_component.strip() + fallback_component = fallback_component.strip() + verification = verification.strip() + reason = reason.strip() + if not failed_component or not fallback_component or not verification or not reason: + return "Fallback declaration rejected: component names, verification, and reason are required." + + recorder = find_diagnostic_recorder(runtime) + if recorder is None: + return "Fallback declaration could not be recorded because diagnostics are unavailable." + + recorder.record_diagnostic_event( + "runtime.fallback.declared", + component="fallback", + status="success", + content="Verified fallback declared.", + metadata={ + "failed_component": failed_component, + "fallback_component": fallback_component, + "constraint_satisfied": constraint_satisfied, + "result_verified": result_verified, + "verification": verification, + "reason": reason, + }, + ) + return "Fallback declaration recorded for FlowLens diagnostics." diff --git a/backend/packages/harness/deerflow/tools/sync.py b/backend/packages/harness/deerflow/tools/sync.py index 7521dd7b..c4dd2586 100644 --- a/backend/packages/harness/deerflow/tools/sync.py +++ b/backend/packages/harness/deerflow/tools/sync.py @@ -79,6 +79,7 @@ def run_coroutine(*args: Any, **kwargs: Any) -> Any: if config_param: + @functools.wraps(coro) def sync_wrapper(*args: Any, config: RunnableConfig = None, **kwargs: Any) -> Any: if config is not None or config_param not in kwargs: kwargs[config_param] = config @@ -86,6 +87,7 @@ def sync_wrapper(*args: Any, config: RunnableConfig = None, **kwargs: Any) -> An return sync_wrapper + @functools.wraps(coro) def sync_wrapper(*args: Any, **kwargs: Any) -> Any: return run_coroutine(*args, **kwargs) diff --git a/backend/packages/harness/deerflow/tools/tools.py b/backend/packages/harness/deerflow/tools/tools.py index bd8daed9..547df79f 100644 --- a/backend/packages/harness/deerflow/tools/tools.py +++ b/backend/packages/harness/deerflow/tools/tools.py @@ -6,7 +6,7 @@ from deerflow.config.app_config import AppConfig from deerflow.reflection import resolve_variable from deerflow.sandbox.security import is_host_bash_allowed -from deerflow.tools.builtins import ask_clarification_tool, present_file_tool, task_tool, view_image_tool +from deerflow.tools.builtins import ask_clarification_tool, present_file_tool, report_fallback, task_tool, view_image_tool from deerflow.tools.mcp_metadata import tag_mcp_tool from deerflow.tools.sync import make_sync_tool_wrapper @@ -15,6 +15,7 @@ BUILTIN_TOOLS = [ present_file_tool, ask_clarification_tool, + report_fallback, ] SUBAGENT_TOOLS = [ diff --git a/backend/packages/sqlite-analytics-mcp/pyproject.toml b/backend/packages/sqlite-analytics-mcp/pyproject.toml new file mode 100644 index 00000000..7c4a90d2 --- /dev/null +++ b/backend/packages/sqlite-analytics-mcp/pyproject.toml @@ -0,0 +1,13 @@ +[project] +name = "sqlite-analytics-mcp" +version = "0.1.0" +description = "Read-only local SQLite analytics MCP service for DeerFlow demonstrations" +requires-python = ">=3.12" +dependencies = ["mcp>=1.0.0"] + +[project.scripts] +sqlite-analytics-mcp = "sqlite_analytics_mcp.server:main" + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" diff --git a/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/__init__.py b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/__init__.py new file mode 100644 index 00000000..d043f699 --- /dev/null +++ b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/__init__.py @@ -0,0 +1 @@ +"""Read-only SQLite analytics MCP service.""" diff --git a/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/query_guard.py b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/query_guard.py new file mode 100644 index 00000000..5ea6eee1 --- /dev/null +++ b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/query_guard.py @@ -0,0 +1,36 @@ +"""Conservative SQL allowlist for the SQLite Analytics MCP service.""" + +from __future__ import annotations + +import re + + +class ReadonlyQueryError(ValueError): + """Raised when a query is outside the read-only analytics contract.""" + + +_MUTATING_OR_ESCAPE_KEYWORDS = re.compile( + r"\b(" + r"alter|analyze|attach|begin|commit|create|delete|detach|drop|" + r"insert|pragma|reindex|release|replace|rollback|savepoint|update|vacuum" + r")\b", + re.IGNORECASE, +) +_READONLY_PREFIX = re.compile(r"^(select|with\b|explain\s+query\s+plan\s+(select|with\b))", re.IGNORECASE) + + +def validate_readonly_sql(statement: str) -> str: + """Return a single permitted query or reject it before SQLite sees it.""" + + normalized = statement.strip() + if not normalized: + raise ReadonlyQueryError("A SQL statement is required") + if normalized.endswith(";"): + normalized = normalized[:-1].rstrip() + if ";" in normalized: + raise ReadonlyQueryError("Multiple SQL statements are not allowed") + if not _READONLY_PREFIX.match(normalized): + raise ReadonlyQueryError("Only SELECT, WITH ... SELECT, and EXPLAIN QUERY PLAN are allowed") + if _MUTATING_OR_ESCAPE_KEYWORDS.search(normalized): + raise ReadonlyQueryError("The SQLite Analytics MCP only permits read-only queries") + return normalized diff --git a/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/repository.py b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/repository.py new file mode 100644 index 00000000..48bd6fd6 --- /dev/null +++ b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/repository.py @@ -0,0 +1,128 @@ +"""Bounded, read-only SQLite access for analytics MCP tools.""" + +from __future__ import annotations + +import sqlite3 +from dataclasses import dataclass +from pathlib import Path + +from .query_guard import validate_readonly_sql + + +@dataclass(frozen=True, slots=True) +class QueryResult: + """Serializable result returned by a bounded analytics query.""" + + columns: list[str] + rows: list[list[object]] + truncated: bool + + +class SQLiteAnalyticsRepository: + """Owns database initialization and per-request read-only connections.""" + + def __init__(self, database_path: Path) -> None: + self._database_path = database_path + + def initialize(self) -> None: + """Create the deterministic local demonstration schema once.""" + + self._database_path.parent.mkdir(parents=True, exist_ok=True) + with sqlite3.connect(self._database_path) as connection: + connection.executescript( + (Path(__file__).parent / "seed.sql").read_text( + encoding="utf-8", + ), + ) + + def execute_readonly(self, statement: str, *, limit: int) -> QueryResult: + """Execute a permitted query and return at most ``limit`` rows.""" + + if limit < 1 or limit > 100: + raise ValueError("limit must be between 1 and 100") + query = validate_readonly_sql(statement) + with self._connect() as connection: + cursor = connection.execute(query) + columns = [column[0] for column in cursor.description or []] + fetched = cursor.fetchmany(limit + 1) + return QueryResult( + columns=columns, + rows=[list(row) for row in fetched[:limit]], + truncated=len(fetched) > limit, + ) + + def execute_raw_for_defense_test(self, statement: str) -> None: + """Exercise SQLite authorization without bypassing connection policy.""" + + with self._connect() as connection: + connection.execute(statement) + + def schema_overview(self) -> list[dict[str, str]]: + """Return permitted tables and their SQLite-defined columns.""" + + with self._connect() as connection: + tables = connection.execute( + "SELECT name FROM sqlite_master WHERE type = 'table' ORDER BY name", + ).fetchall() + return [{"name": str(row[0])} for row in tables] + + def describe_table(self, table_name: str) -> list[dict[str, str]]: + """Return column names and declared types for an allowed table.""" + + if table_name not in {"agent_runs", "tool_calls"}: + raise ValueError("Unknown analytics table") + with self._connect() as connection: + rows = connection.execute( + f"PRAGMA table_info({table_name})", + ).fetchall() + return [{"name": str(row[1]), "type": str(row[2])} for row in rows] + + def _connect(self) -> sqlite3.Connection: + connection = sqlite3.connect(self._database_path) + connection.set_authorizer(_readonly_authorizer) + return connection + + +def _readonly_authorizer( + action_code: int, + parameter_one: str | None, + parameter_two: str | None, + database_name: str | None, + trigger_name: str | None, +) -> int: + """Deny mutations and escape hatches at the SQLite virtual machine layer.""" + + del parameter_one, database_name, trigger_name + denied_actions = { + sqlite3.SQLITE_ALTER_TABLE, + sqlite3.SQLITE_ANALYZE, + sqlite3.SQLITE_ATTACH, + sqlite3.SQLITE_CREATE_INDEX, + sqlite3.SQLITE_CREATE_TABLE, + sqlite3.SQLITE_CREATE_TEMP_INDEX, + sqlite3.SQLITE_CREATE_TEMP_TABLE, + sqlite3.SQLITE_CREATE_TEMP_TRIGGER, + sqlite3.SQLITE_CREATE_TEMP_VIEW, + sqlite3.SQLITE_CREATE_TRIGGER, + sqlite3.SQLITE_CREATE_VIEW, + sqlite3.SQLITE_DELETE, + sqlite3.SQLITE_DETACH, + sqlite3.SQLITE_DROP_INDEX, + sqlite3.SQLITE_DROP_TABLE, + sqlite3.SQLITE_DROP_TEMP_INDEX, + sqlite3.SQLITE_DROP_TEMP_TABLE, + sqlite3.SQLITE_DROP_TEMP_TRIGGER, + sqlite3.SQLITE_DROP_TEMP_VIEW, + sqlite3.SQLITE_DROP_TRIGGER, + sqlite3.SQLITE_DROP_VIEW, + sqlite3.SQLITE_INSERT, + sqlite3.SQLITE_PRAGMA, + sqlite3.SQLITE_REINDEX, + sqlite3.SQLITE_TRANSACTION, + sqlite3.SQLITE_UPDATE, + } + if action_code in denied_actions: + return sqlite3.SQLITE_DENY + if action_code == sqlite3.SQLITE_FUNCTION and parameter_two == "load_extension": + return sqlite3.SQLITE_DENY + return sqlite3.SQLITE_OK diff --git a/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/seed.sql b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/seed.sql new file mode 100644 index 00000000..73c11b3c --- /dev/null +++ b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/seed.sql @@ -0,0 +1,28 @@ +CREATE TABLE IF NOT EXISTS agent_runs ( + run_id TEXT PRIMARY KEY, + scenario TEXT NOT NULL, + status TEXT NOT NULL, + duration_ms INTEGER NOT NULL, + token_count INTEGER NOT NULL, + created_at TEXT NOT NULL +); + +CREATE TABLE IF NOT EXISTS tool_calls ( + call_id INTEGER PRIMARY KEY, + run_id TEXT NOT NULL, + tool_name TEXT NOT NULL, + outcome TEXT NOT NULL, + duration_ms INTEGER NOT NULL, + FOREIGN KEY (run_id) REFERENCES agent_runs(run_id) +); + +INSERT OR IGNORE INTO agent_runs VALUES + ('demo-run-001', 'web-research', 'success', 12460, 3460, '2026-07-29T09:00:00Z'), + ('demo-run-002', 'incident-analysis', 'error', 9800, 2190, '2026-07-29T10:00:00Z'), + ('demo-run-003', 'report-generation', 'success', 15120, 4210, '2026-07-29T11:00:00Z'); + +INSERT OR IGNORE INTO tool_calls VALUES + (1, 'demo-run-001', 'web_search', 'success', 2150), + (2, 'demo-run-001', 'fetch_url', 'success', 3120), + (3, 'demo-run-002', 'bash', 'error', 420), + (4, 'demo-run-003', 'write_file', 'success', 180); diff --git a/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/server.py b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/server.py new file mode 100644 index 00000000..2a566eb6 --- /dev/null +++ b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/server.py @@ -0,0 +1,163 @@ +"""Streamable HTTP MCP server for bounded SQLite analytics.""" + +from __future__ import annotations + +from typing import Any + +from mcp.server.auth.middleware.auth_context import get_access_token +from mcp.server.auth.provider import AccessToken, TokenVerifier +from mcp.server.auth.settings import AuthSettings +from mcp.server.fastmcp import FastMCP + +from app.gateway.auth.platform_tokens import ( + PlatformTokenError, + PlatformTokenSettings, + verify_platform_token, +) + +from .repository import SQLiteAnalyticsRepository +from .settings import SQLiteAnalyticsSettings + + +class PlatformTokenVerifier: + """Adapt signed DeerFlow platform tokens to FastMCP authentication.""" + + def __init__(self, settings: PlatformTokenSettings | None = None) -> None: + self._settings = settings + + async def verify_token(self, token: str) -> AccessToken | None: + try: + claims = verify_platform_token(token, self._settings) + except PlatformTokenError: + return None + return AccessToken( + token=token, + client_id=claims.azp, + scopes=claims.scope, + expires_at=int(claims.exp.timestamp()), + ) + + +def require_scope(scope: str) -> None: + """Require an analytics scope after unauthenticated discovery is complete.""" + + access_token = get_access_token() + if access_token is None or scope not in access_token.scopes: + raise PermissionError(f"MCP tool requires delegated scope: {scope}") + + +class SQLiteAnalyticsTools: + """Tool handlers with no dependency on MCP transport internals.""" + + def __init__(self, repository: SQLiteAnalyticsRepository) -> None: + self._repository = repository + + def schema_overview(self) -> dict[str, Any]: + """List the read-only local demonstration tables.""" + + return { + "schema_version": 1, + "tables": self._repository.schema_overview(), + } + + def describe_table(self, table_name: str) -> dict[str, Any]: + """Describe one documented demonstration table.""" + + return { + "schema_version": 1, + "table": table_name, + "columns": self._repository.describe_table(table_name), + } + + def query_readonly(self, statement: str, limit: int = 50) -> dict[str, Any]: + """Run a bounded SQL query accepted by the read-only policy.""" + + result = self._repository.execute_readonly(statement, limit=limit) + return { + "schema_version": 1, + "columns": result.columns, + "rows": result.rows, + "row_count": len(result.rows), + "truncated": result.truncated, + } + + +def create_server( + repository: SQLiteAnalyticsRepository, + *, + host: str, + port: int, + token_verifier: TokenVerifier | None = None, + auth: AuthSettings | None = None, +) -> FastMCP: + """Create an MCP application that exposes the three analytics tools.""" + + if (token_verifier is None) != (auth is None): + raise ValueError("token_verifier and auth must be configured together") + + handlers = SQLiteAnalyticsTools(repository) + server = FastMCP( + "SQLite Analytics MCP", + instructions=("Use schema_overview and describe_table before query_readonly. All database access is read-only and bounded."), + host=host, + port=port, + streamable_http_path="/mcp", + token_verifier=token_verifier, + auth=auth, + ) + + def schema_overview() -> dict[str, Any]: + """List the allowed local demonstration tables before any SQL query. + + Use this first to discover the read-only analytics schema. The result is + local demonstration data and never represents production telemetry. + """ + require_scope("analytics.schema.read") + return handlers.schema_overview() + + def describe_table(table_name: str) -> dict[str, Any]: + """Describe columns for one allowed local demonstration table. + + Call this after schema_overview and before querying a table for the + first time. Supported tables are agent_runs and tool_calls. + """ + require_scope("analytics.schema.read") + return handlers.describe_table(table_name) + + def query_readonly(statement: str, limit: int = 50) -> dict[str, Any]: + """Run one bounded read-only SQL statement against local demo analytics. + + Only SELECT, WITH, and EXPLAIN QUERY PLAN statements are accepted. Do + not use writes, DDL, PRAGMA, ATTACH, or multiple statements. + """ + require_scope("analytics.query.read") + return handlers.query_readonly(statement, limit=limit) + + server.tool()(schema_overview) + server.tool()(describe_table) + server.tool()(query_readonly) + return server + + +def main() -> None: + """Initialize local data and serve MCP over Streamable HTTP.""" + + settings = SQLiteAnalyticsSettings.from_env() + repository = SQLiteAnalyticsRepository(settings.database_path) + repository.initialize() + server = create_server( + repository, + host=settings.host, + port=settings.port, + token_verifier=PlatformTokenVerifier(), + auth=AuthSettings( + issuer_url=settings.issuer_url, + resource_server_url=settings.resource_url, + required_scopes=[], + ), + ) + server.run(transport="streamable-http") + + +if __name__ == "__main__": + main() diff --git a/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/settings.py b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/settings.py new file mode 100644 index 00000000..49b97d1a --- /dev/null +++ b/backend/packages/sqlite-analytics-mcp/sqlite_analytics_mcp/settings.py @@ -0,0 +1,37 @@ +"""Environment-backed settings for SQLite Analytics MCP.""" + +from __future__ import annotations + +import os +from dataclasses import dataclass +from pathlib import Path + + +@dataclass(frozen=True, slots=True) +class SQLiteAnalyticsSettings: + database_path: Path + host: str + port: int + issuer_url: str + resource_url: str + + @classmethod + def from_env(cls) -> SQLiteAnalyticsSettings: + return cls( + database_path=Path( + os.getenv( + "SQLITE_ANALYTICS_DATABASE_PATH", + "/var/lib/sqlite-analytics/analytics.db", + ), + ), + host=os.getenv("SQLITE_ANALYTICS_HOST", "0.0.0.0"), + port=int(os.getenv("SQLITE_ANALYTICS_PORT", "8011")), + issuer_url=os.getenv( + "SQLITE_ANALYTICS_ISSUER_URL", + "http://gateway:8000", + ).rstrip("/"), + resource_url=os.getenv( + "SQLITE_ANALYTICS_RESOURCE_URL", + "http://sqlite-analytics:8011", + ).rstrip("/"), + ) diff --git a/backend/pyproject.toml b/backend/pyproject.toml index d9dfddda..788e5e1b 100644 --- a/backend/pyproject.toml +++ b/backend/pyproject.toml @@ -46,7 +46,11 @@ markers = [ index-url = "https://pypi.org/simple" [tool.uv.workspace] -members = ["packages/harness"] +members = [ + "packages/harness", + "packages/sqlite-analytics-mcp", + "packages/flowlens-inspector-mcp", +] [tool.uv.sources] deerflow-harness = { workspace = true } diff --git a/backend/scripts/generate_flowlens_mcp_keys.py b/backend/scripts/generate_flowlens_mcp_keys.py new file mode 100644 index 00000000..74e6fb66 --- /dev/null +++ b/backend/scripts/generate_flowlens_mcp_keys.py @@ -0,0 +1,48 @@ +"""Generate local-only FlowLens MCP RSA key material for Docker development.""" + +from __future__ import annotations + +import argparse +import os +from pathlib import Path + +from cryptography.hazmat.primitives import serialization +from cryptography.hazmat.primitives.asymmetric import rsa + + +def generate(directory: Path) -> None: + """Create a key pair only when it is absent, preserving existing sessions.""" + + private_path = directory / "private.pem" + public_path = directory / "public.pem" + if private_path.exists() and public_path.exists(): + return + + directory.mkdir(parents=True, exist_ok=True) + private_key = rsa.generate_private_key(public_exponent=65537, key_size=2048) + private_path.write_bytes( + private_key.private_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PrivateFormat.PKCS8, + encryption_algorithm=serialization.NoEncryption(), + ), + ) + public_path.write_bytes( + private_key.public_key().public_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PublicFormat.SubjectPublicKeyInfo, + ), + ) + os.chmod(private_path, 0o600) + os.chmod(public_path, 0o644) + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--directory", type=Path, required=True) + args = parser.parse_args() + generate(args.directory) + + +if __name__ == "__main__": + main() diff --git a/backend/tests/conftest.py b/backend/tests/conftest.py index 03dee4b0..82f82eca 100644 --- a/backend/tests/conftest.py +++ b/backend/tests/conftest.py @@ -17,6 +17,22 @@ # Make 'app' and 'deerflow' importable from any working directory sys.path.insert(0, str(Path(__file__).parent.parent)) sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "scripts")) +sys.path.insert( + 0, + str( + Path(__file__).resolve().parents[1] + / "packages" + / "sqlite-analytics-mcp", + ), +) +sys.path.insert( + 0, + str( + Path(__file__).resolve().parents[1] + / "packages" + / "flowlens-inspector-mcp", + ), +) # Break the circular import chain that exists in production code: # deerflow.subagents.__init__ diff --git a/backend/tests/flowlens_inspector_mcp/test_gateway_client.py b/backend/tests/flowlens_inspector_mcp/test_gateway_client.py new file mode 100644 index 00000000..d413dbb8 --- /dev/null +++ b/backend/tests/flowlens_inspector_mcp/test_gateway_client.py @@ -0,0 +1,14 @@ +"""Inspector Gateway client credential-boundary tests.""" + +from __future__ import annotations + +from flowlens_inspector_mcp.gateway_client import GatewayClient + + +def test_gateway_client_uses_the_request_scoped_bearer_token() -> None: + client = GatewayClient( + "http://gateway:8000", + token_provider=lambda: "delegated-user-token", + ) + + assert client.request_headers == {"Authorization": "Bearer delegated-user-token"} diff --git a/backend/tests/flowlens_inspector_mcp/test_platform_auth.py b/backend/tests/flowlens_inspector_mcp/test_platform_auth.py new file mode 100644 index 00000000..cb317f58 --- /dev/null +++ b/backend/tests/flowlens_inspector_mcp/test_platform_auth.py @@ -0,0 +1,52 @@ +"""Platform-token verification for the Inspector MCP boundary.""" + +from __future__ import annotations + +import asyncio + +from cryptography.hazmat.primitives import serialization +from cryptography.hazmat.primitives.asymmetric import rsa +from flowlens_inspector_mcp.auth import PlatformTokenVerifier + +from app.gateway.auth.platform_tokens import ( + PlatformTokenIssuer, + PlatformTokenSettings, +) + + +def test_verifier_converts_a_valid_delegated_platform_token() -> None: + private_key = rsa.generate_private_key(public_exponent=65537, key_size=2048) + settings = PlatformTokenSettings( + private_key_pem=private_key.private_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PrivateFormat.PKCS8, + encryption_algorithm=serialization.NoEncryption(), + ).decode(), + public_key_pem=private_key.public_key() + .public_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PublicFormat.SubjectPublicKeyInfo, + ) + .decode(), + kid="flowlens-test-key", + issuer="deerflow-gateway", + audience="flowlens-platform", + client_id="deerflow-runtime", + client_secret="test-client-secret", + ) + token = PlatformTokenIssuer(settings).issue_delegated_token( + user_id="user-a", + tenant_id="tenant-a", + token_version=2, + origin_run_id="run-origin", + ) + + access_token = asyncio.run(PlatformTokenVerifier(settings).verify_token(token)) + + assert access_token is not None + assert access_token.client_id == "deerflow-runtime" + assert "agentops.runs.read" in access_token.scopes + + +def test_verifier_rejects_malformed_tokens() -> None: + assert asyncio.run(PlatformTokenVerifier().verify_token("not-a-jwt")) is None diff --git a/backend/tests/flowlens_inspector_mcp/test_settings.py b/backend/tests/flowlens_inspector_mcp/test_settings.py new file mode 100644 index 00000000..78ee05ac --- /dev/null +++ b/backend/tests/flowlens_inspector_mcp/test_settings.py @@ -0,0 +1,23 @@ +"""Environment configuration for the Inspector MCP service.""" + +from __future__ import annotations + +from flowlens_inspector_mcp.settings import InspectorSettings + + +def test_settings_use_container_safe_defaults(monkeypatch) -> None: + monkeypatch.delenv("FLOWLENS_INSPECTOR_GATEWAY_URL", raising=False) + monkeypatch.delenv("FLOWLENS_INSPECTOR_PORT", raising=False) + + settings = InspectorSettings.from_env() + + assert settings.gateway_url == "http://gateway:8000" + assert settings.port == 8012 + + +def test_settings_accept_explicit_public_resource_url(monkeypatch) -> None: + monkeypatch.setenv("FLOWLENS_INSPECTOR_RESOURCE_URL", "http://localhost:8012") + + settings = InspectorSettings.from_env() + + assert settings.resource_url == "http://localhost:8012" diff --git a/backend/tests/flowlens_inspector_mcp/test_tools.py b/backend/tests/flowlens_inspector_mcp/test_tools.py new file mode 100644 index 00000000..e3931531 --- /dev/null +++ b/backend/tests/flowlens_inspector_mcp/test_tools.py @@ -0,0 +1,105 @@ +"""Read-only FlowLens Inspector MCP tool contracts.""" + +from __future__ import annotations + +import asyncio +from dataclasses import dataclass +from typing import Any + +import pytest +from flowlens_inspector_mcp.auth import PlatformTokenVerifier +from flowlens_inspector_mcp.server import FlowLensInspectorTools, require_scope +from mcp.server.auth.settings import AuthSettings + + +@dataclass +class FakeGatewayClient: + runs: list[dict[str, Any]] + + async def health_check(self) -> dict[str, Any]: + return {"gateway": "healthy", "data_source": "sqlite"} + + async def list_runs(self, **_: Any) -> dict[str, Any]: + return {"items": self.runs, "has_more": False, "next_cursor": None} + + async def get_diagnostics(self, thread_id: str, run_id: str) -> dict[str, Any]: + return { + "run_id": run_id, + "thread_id": thread_id, + "metrics": {"duration_ms": 1200 if run_id == "run-a" else 900, "total_tokens": 200 if run_id == "run-a" else 150}, + "failure_attribution": {"primary_cause": "NONE"}, + } + + async def get_timeline(self, thread_id: str, run_id: str, **_: Any) -> dict[str, Any]: + return {"run_id": run_id, "thread_id": thread_id, "items": [{"seq": 1, "event_type": "run.start"}], "has_more": False} + + +def test_health_check_returns_gateway_status() -> None: + tools = FlowLensInspectorTools(FakeGatewayClient([])) + + result = asyncio.run(tools.health_check()) + + assert result["schema_version"] == 1 + assert result["gateway"] == "healthy" + + +def test_list_runs_returns_only_gateway_authorized_results() -> None: + tools = FlowLensInspectorTools(FakeGatewayClient([{"run_id": "run-a", "thread_id": "thread-a"}])) + + result = asyncio.run(tools.list_runs(limit=20)) + + assert result["items"] == [{"run_id": "run-a", "thread_id": "thread-a"}] + + +def test_compare_runs_derives_deterministic_metric_delta() -> None: + tools = FlowLensInspectorTools(FakeGatewayClient([])) + + result = asyncio.run( + tools.compare_runs( + base_run_id="run-a", + base_thread_id="thread-a", + candidate_run_id="run-b", + candidate_thread_id="thread-b", + ), + ) + + assert result["duration_delta_ms"] == -300 + assert result["token_delta"] == -50 + assert result["base_primary_cause"] == "NONE" + + +@pytest.mark.parametrize("limit", [0, 101]) +def test_list_runs_rejects_unbounded_windows(limit: int) -> None: + tools = FlowLensInspectorTools(FakeGatewayClient([])) + + with pytest.raises(ValueError, match="limit"): + asyncio.run(tools.list_runs(limit=limit)) + + +def test_server_requires_a_platform_token_verifier() -> None: + verifier = PlatformTokenVerifier() + auth = AuthSettings( + issuer_url="http://gateway:8000", + resource_server_url="http://flowlens-inspector:8012", + required_scopes=["agentops.runs.read"], + ) + + from flowlens_inspector_mcp.server import create_server + + server = create_server( + FakeGatewayClient([]), + host="0.0.0.0", + port=8012, + token_verifier=verifier, + auth=auth, + ) + + assert server._token_verifier is verifier + assert server.settings.auth.required_scopes == ["agentops.runs.read"] + + +def test_read_tools_require_delegated_read_scope(monkeypatch) -> None: + monkeypatch.setattr("flowlens_inspector_mcp.server.get_access_token", lambda: None) + + with pytest.raises(PermissionError, match="agentops.runs.read"): + require_scope("agentops.runs.read") diff --git a/backend/tests/sqlite_analytics_mcp/test_query_guard.py b/backend/tests/sqlite_analytics_mcp/test_query_guard.py new file mode 100644 index 00000000..0012328c --- /dev/null +++ b/backend/tests/sqlite_analytics_mcp/test_query_guard.py @@ -0,0 +1,34 @@ +"""Read-only SQL policy tests for SQLite Analytics MCP.""" + +from __future__ import annotations + +import pytest +from sqlite_analytics_mcp.query_guard import ReadonlyQueryError, validate_readonly_sql + + +@pytest.mark.parametrize( + "statement", + [ + "SELECT category, COUNT(*) FROM incidents GROUP BY category", + "WITH recent AS (SELECT * FROM incidents) SELECT * FROM recent", + "EXPLAIN QUERY PLAN SELECT * FROM incidents", + ], +) +def test_allows_readonly_query_forms(statement: str) -> None: + assert validate_readonly_sql(statement) == statement + + +@pytest.mark.parametrize( + "statement", + [ + "INSERT INTO incidents VALUES (1, 'bad')", + "UPDATE incidents SET severity = 'high'", + "DELETE FROM incidents", + "CREATE TABLE hidden (id INTEGER)", + "ATTACH DATABASE 'other.db' AS other", + "PRAGMA writable_schema = 1", + ], +) +def test_rejects_mutating_or_escape_sql(statement: str) -> None: + with pytest.raises(ReadonlyQueryError): + validate_readonly_sql(statement) diff --git a/backend/tests/sqlite_analytics_mcp/test_repository.py b/backend/tests/sqlite_analytics_mcp/test_repository.py new file mode 100644 index 00000000..1dabb226 --- /dev/null +++ b/backend/tests/sqlite_analytics_mcp/test_repository.py @@ -0,0 +1,37 @@ +"""SQLite execution boundary tests for SQLite Analytics MCP.""" + +from __future__ import annotations + +import sqlite3 +from pathlib import Path + +import pytest +from sqlite_analytics_mcp.repository import SQLiteAnalyticsRepository + + +@pytest.fixture() +def repository(tmp_path: Path) -> SQLiteAnalyticsRepository: + database_path = tmp_path / "analytics.db" + repository = SQLiteAnalyticsRepository(database_path) + repository.initialize() + return repository + + +def test_executes_readonly_query_with_bounded_rows(repository: SQLiteAnalyticsRepository) -> None: + result = repository.execute_readonly( + "SELECT run_id, status FROM agent_runs ORDER BY run_id", + limit=1, + ) + + assert result.columns == ["run_id", "status"] + assert len(result.rows) == 1 + assert result.truncated is True + + +def test_sqlite_authorizer_rejects_write_even_when_called_directly( + repository: SQLiteAnalyticsRepository, +) -> None: + with pytest.raises(sqlite3.DatabaseError, match="not authorized"): + repository.execute_raw_for_defense_test( + "DELETE FROM agent_runs", + ) diff --git a/backend/tests/sqlite_analytics_mcp/test_server_tools.py b/backend/tests/sqlite_analytics_mcp/test_server_tools.py new file mode 100644 index 00000000..27921dcd --- /dev/null +++ b/backend/tests/sqlite_analytics_mcp/test_server_tools.py @@ -0,0 +1,100 @@ +"""Public tool contract tests for SQLite Analytics MCP.""" + +from __future__ import annotations + +import asyncio +from pathlib import Path + +import pytest +from mcp.server.auth.settings import AuthSettings +from sqlite_analytics_mcp.repository import SQLiteAnalyticsRepository +from sqlite_analytics_mcp.server import ( + PlatformTokenVerifier, + SQLiteAnalyticsTools, + create_server, + require_scope, +) + + +@pytest.fixture() +def tools(tmp_path: Path) -> SQLiteAnalyticsTools: + repository = SQLiteAnalyticsRepository(tmp_path / "analytics.db") + repository.initialize() + return SQLiteAnalyticsTools(repository) + + +def test_schema_overview_exposes_only_documented_tables( + tools: SQLiteAnalyticsTools, +) -> None: + result = tools.schema_overview() + + assert result["schema_version"] == 1 + assert [table["name"] for table in result["tables"]] == [ + "agent_runs", + "tool_calls", + ] + + +def test_query_readonly_returns_bounded_structured_rows( + tools: SQLiteAnalyticsTools, +) -> None: + result = tools.query_readonly( + "SELECT status, COUNT(*) AS run_count FROM agent_runs GROUP BY status", + limit=10, + ) + + assert result["columns"] == ["status", "run_count"] + assert result["row_count"] == 2 + assert result["truncated"] is False + + +def test_describe_table_rejects_unknown_table(tools: SQLiteAnalyticsTools) -> None: + with pytest.raises(ValueError, match="Unknown analytics table"): + tools.describe_table("sqlite_master") + + +def test_server_registers_tools_on_the_configured_http_endpoint( + tmp_path: Path, +) -> None: + repository = SQLiteAnalyticsRepository(tmp_path / "analytics.db") + repository.initialize() + + server = create_server(repository, host="127.0.0.1", port=8013) + + registered_tools = asyncio.run(server.list_tools()) + assert {tool.name for tool in registered_tools} == { + "schema_overview", + "describe_table", + "query_readonly", + } + assert all(tool.description for tool in registered_tools) + assert server.settings.host == "127.0.0.1" + assert server.settings.port == 8013 + + +def test_server_can_require_platform_token_authentication(tmp_path: Path) -> None: + repository = SQLiteAnalyticsRepository(tmp_path / "analytics.db") + repository.initialize() + verifier = PlatformTokenVerifier() + auth = AuthSettings( + issuer_url="http://gateway:8000", + resource_server_url="http://sqlite-analytics:8011", + required_scopes=[], + ) + + server = create_server( + repository, + host="127.0.0.1", + port=8013, + token_verifier=verifier, + auth=auth, + ) + + assert server._token_verifier is verifier + + +def test_sql_tools_require_a_delegated_query_scope(monkeypatch) -> None: + monkeypatch.setattr("sqlite_analytics_mcp.server.get_access_token", lambda: None) + + with pytest.raises(PermissionError, match="analytics.query.read"): + require_scope("analytics.query.read") diff --git a/backend/tests/test_agentops_run_lookup_api.py b/backend/tests/test_agentops_run_lookup_api.py new file mode 100644 index 00000000..b92e4ce0 --- /dev/null +++ b/backend/tests/test_agentops_run_lookup_api.py @@ -0,0 +1,79 @@ +"""Owner-safe AgentOps run lookup contract tests.""" + +from __future__ import annotations + +from types import SimpleNamespace +from unittest.mock import AsyncMock, patch + +from _router_auth_helpers import make_authed_test_app +from fastapi.testclient import TestClient + +from app.gateway.routers import agentops + + +def _record(): + return SimpleNamespace( + run_id="run-a", + thread_id="thread-a", + status=SimpleNamespace(value="error"), + created_at="2026-01-01T00:00:00+00:00", + updated_at="2026-01-01T00:00:02+00:00", + model_name="test-model", + total_tokens=120, + llm_call_count=2, + message_count=4, + ) + + +def _make_client(manager): + app = make_authed_test_app() + app.state.run_manager = manager + app.include_router(agentops.router) + return TestClient(app) + + +def test_run_lookup_returns_stable_mcp_summary(): + manager = SimpleNamespace(get=AsyncMock(return_value=_record())) + + with patch.object( + agentops, + "get_current_user", + AsyncMock(return_value="user-a"), + ): + with _make_client(manager) as client: + response = client.get("/api/agentops/runs/run-a") + + assert response.status_code == 200 + assert response.json() == { + "run_id": "run-a", + "thread_id": "thread-a", + "status": "error", + "created_at": "2026-01-01T00:00:00+00:00", + "updated_at": "2026-01-01T00:00:02+00:00", + "model_name": "test-model", + "total_tokens": 120, + "llm_call_count": 2, + "message_count": 4, + } + manager.get.assert_awaited_once_with( + "run-a", + user_id="user-a", + tenant_id="default", + ) + + +def test_run_lookup_hides_inaccessible_run_as_not_found(): + manager = SimpleNamespace(get=AsyncMock(return_value=None)) + + with patch.object( + agentops, + "get_current_user", + AsyncMock(return_value="user-a"), + ): + with _make_client(manager) as client: + response = client.get( + "/api/agentops/runs/run-owned-by-tenant-b", + ) + + assert response.status_code == 404 + assert response.json()["detail"] == "Run not found" diff --git a/backend/tests/test_agentops_runs_api.py b/backend/tests/test_agentops_runs_api.py index 1b0b02d2..747ffec3 100644 --- a/backend/tests/test_agentops_runs_api.py +++ b/backend/tests/test_agentops_runs_api.py @@ -16,11 +16,11 @@ def _agentops_module(): pytest.fail("AgentOps run index router is not implemented") -def _record(run_id: str, created_at: str): +def _record(run_id: str, created_at: str, *, status: str = "error"): return SimpleNamespace( run_id=run_id, thread_id=f"thread-{run_id}", - status=SimpleNamespace(value="error"), + status=SimpleNamespace(value=status), created_at=created_at, updated_at=created_at, model_name="test-model", @@ -42,19 +42,26 @@ def test_agentops_runs_returns_current_user_rows_and_cursor(): ) app = make_authed_test_app() app.state.run_manager = manager + app.state.run_event_store = SimpleNamespace(list_events=AsyncMock(return_value=[])) app.include_router(agentops.router) with patch.object(agentops, "get_current_user", AsyncMock(return_value="user-1")): with TestClient(app) as client: - response = client.get("/api/agentops/runs?limit=1&status=error") + response = client.get( + "/api/agentops/runs" + "?limit=1&status=error&thread_id=thread-r2" + ) assert response.status_code == 200 body = response.json() assert [row["run_id"] for row in body["items"]] == ["r2"] assert body["has_more"] is True assert body["next_cursor"] + assert body["items"][0]["diagnostic_category"] == "UNKNOWN_RUNTIME_ERROR" manager.list_recent.assert_awaited_once_with( user_id="user-1", + tenant_id="default", + thread_id="thread-r2", limit=2, cursor=None, status="error", @@ -76,3 +83,46 @@ def test_agentops_runs_rejects_malformed_cursor_before_store_access(): assert response.status_code == 422 manager.list_recent.assert_not_awaited() + + +def test_agentops_runs_exposes_unrecovered_diagnostic_incidents(): + agentops = _agentops_module() + manager = SimpleNamespace( + list_recent=AsyncMock( + return_value=[ + _record( + "run-incident", + "2026-01-02T00:00:00+00:00", + status="success", + ) + ] + ) + ) + event_store = SimpleNamespace( + list_events=AsyncMock( + return_value=[ + { + "seq": 6, + "event_type": "mcp.tool.error", + "category": "error", + "content": "ValueError", + "metadata": { + "server_name": "sqlite_analytics", + "tool_name": "schema_overview", + }, + "created_at": "2026-01-02T00:00:01+00:00", + } + ] + ) + ) + app = make_authed_test_app() + app.state.run_manager = manager + app.state.run_event_store = event_store + app.include_router(agentops.router) + + with patch.object(agentops, "get_current_user", AsyncMock(return_value="user-1")): + with TestClient(app) as client: + response = client.get("/api/agentops/runs") + + assert response.status_code == 200 + assert response.json()["items"][0]["diagnostic_category"] == "MCP_TOOL_ERROR" diff --git a/backend/tests/test_cancel_run_idempotent.py b/backend/tests/test_cancel_run_idempotent.py index 0bf2548d..058b6d50 100644 --- a/backend/tests/test_cancel_run_idempotent.py +++ b/backend/tests/test_cancel_run_idempotent.py @@ -9,14 +9,24 @@ from __future__ import annotations import asyncio +from uuid import uuid4 from _router_auth_helpers import make_authed_test_app from fastapi.testclient import TestClient +from app.gateway.auth.models import User from app.gateway.routers import thread_runs from deerflow.runtime import RunManager, RunStatus +from deerflow.runtime.user_context import ( + reset_current_user, + set_current_user, +) THREAD_ID = "thread-cancel-test" +ROUTER_USER = User( + id=uuid4(), + email="cancel-route@example.com", +) # --------------------------------------------------------------------------- @@ -25,7 +35,9 @@ def _make_app(mgr: RunManager) -> TestClient: - app = make_authed_test_app() + app = make_authed_test_app( + user_factory=lambda: ROUTER_USER, + ) app.include_router(thread_runs.router) app.state.run_manager = mgr return TestClient(app, raise_server_exceptions=False) @@ -40,7 +52,11 @@ async def _setup(): await mgr.cancel(record.run_id) return record.run_id - return asyncio.run(_setup()) + token = set_current_user(ROUTER_USER) + try: + return asyncio.run(_setup()) + finally: + reset_current_user(token) # --------------------------------------------------------------------------- @@ -116,7 +132,11 @@ async def _setup(): await mgr.set_status(record.run_id, RunStatus.success) return mgr, record.run_id - mgr, run_id = asyncio.run(_setup()) + token = set_current_user(ROUTER_USER) + try: + mgr, run_id = asyncio.run(_setup()) + finally: + reset_current_user(token) client = _make_app(mgr) resp = client.post(f"/api/threads/{THREAD_ID}/runs/{run_id}/cancel") assert resp.status_code == 409 diff --git a/backend/tests/test_fallback_reporting_tool.py b/backend/tests/test_fallback_reporting_tool.py new file mode 100644 index 00000000..cadabbe4 --- /dev/null +++ b/backend/tests/test_fallback_reporting_tool.py @@ -0,0 +1,41 @@ +from types import SimpleNamespace +from unittest.mock import MagicMock + +from deerflow.tools.builtins.report_fallback_tool import report_fallback +from deerflow.tools.tools import BUILTIN_TOOLS + + +def test_fallback_reporting_is_exposed_as_a_builtin_tool(): + assert any(tool.name == "report_fallback" for tool in BUILTIN_TOOLS) + + +def test_fallback_reporting_records_a_structured_diagnostic_event(): + recorder = MagicMock() + recorder.record_diagnostic_event_threadsafe = MagicMock() + runtime = SimpleNamespace(config={"callbacks": [recorder]}) + + result = report_fallback.func( + runtime=runtime, + failed_component="sqlite_analytics.schema_overview", + fallback_component="analytics_snapshot", + constraint_satisfied=True, + result_verified=True, + verification="Snapshot version matched the requested dataset.", + reason="The primary MCP transport timed out.", + ) + + assert result == "Fallback declaration recorded for FlowLens diagnostics." + recorder.record_diagnostic_event.assert_called_once_with( + "runtime.fallback.declared", + component="fallback", + status="success", + content="Verified fallback declared.", + metadata={ + "failed_component": "sqlite_analytics.schema_overview", + "fallback_component": "analytics_snapshot", + "constraint_satisfied": True, + "result_verified": True, + "verification": "Snapshot version matched the requested dataset.", + "reason": "The primary MCP transport timed out.", + }, + ) diff --git a/backend/tests/test_flowlens_attribution.py b/backend/tests/test_flowlens_attribution.py index 81c97206..eda5934b 100644 --- a/backend/tests/test_flowlens_attribution.py +++ b/backend/tests/test_flowlens_attribution.py @@ -26,6 +26,72 @@ def test_successful_run_has_no_failure(): assert result["evidence"] == [] +def test_successful_run_with_mcp_tool_error_keeps_the_mcp_incident_as_root_cause(): + timeline = [ + _item( + 6, + "mcp", + "mcp.tool.error", + summary="SQLite Analytics MCP rejected the tool call: ValueError", + ) + ] + + result = attribute_failure(_record(status="success", error=None), timeline) + + assert result["primary_cause"] == "MCP_TOOL_ERROR" + assert result["evidence"][0]["seq"] == 6 + + +def test_verified_constraint_satisfying_fallback_clears_mcp_root_cause(): + timeline = [ + _item(6, "mcp", "mcp.tool.error", summary="SQLite MCP failed", metadata={"server_name": "sqlite_analytics", "tool_name": "schema_overview"}), + _item(7, "tool", "tool.end", status="success", summary="Read snapshot", metadata={"tool_name": "analytics_snapshot"}), + _item( + 8, + "runtime", + "runtime.fallback.declared", + status="success", + summary="Verified analytics fallback", + metadata={ + "failed_component": "sqlite_analytics.schema_overview", + "fallback_component": "analytics_snapshot", + "constraint_satisfied": True, + "result_verified": True, + "verification": "Snapshot version matched the requested dataset.", + }, + ), + ] + + result = attribute_failure(_record(status="success", error=None), timeline) + + assert result["primary_cause"] == "NONE" + + +def test_constraint_violating_fallback_does_not_clear_mcp_root_cause(): + timeline = [ + _item(6, "mcp", "mcp.tool.error", summary="SQLite MCP failed", metadata={"server_name": "sqlite_analytics", "tool_name": "schema_overview"}), + _item(7, "tool", "tool.end", status="success", summary="Read snapshot", metadata={"tool_name": "analytics_snapshot"}), + _item( + 8, + "runtime", + "runtime.fallback.declared", + status="success", + summary="Fallback cannot satisfy the request", + metadata={ + "failed_component": "sqlite_analytics.schema_overview", + "fallback_component": "analytics_snapshot", + "constraint_satisfied": False, + "result_verified": True, + "verification": "Snapshot version matched the requested dataset.", + }, + ), + ] + + result = attribute_failure(_record(status="success", error=None), timeline) + + assert result["primary_cause"] == "MCP_TOOL_ERROR" + + def test_rollback_takes_priority_over_tool_error(): timeline = [_item(2, "tool", "tool.error", summary="bash failed")] diff --git a/backend/tests/test_flowlens_tenant_migration.py b/backend/tests/test_flowlens_tenant_migration.py new file mode 100644 index 00000000..4896e674 --- /dev/null +++ b/backend/tests/test_flowlens_tenant_migration.py @@ -0,0 +1,75 @@ +"""Integration test for the idempotent FlowLens tenant migration.""" + +from __future__ import annotations + +import importlib.util +from pathlib import Path + +import sqlalchemy as sa +from alembic.migration import MigrationContext +from alembic.operations import Operations + + +def _load_migration(): + migration_path = ( + Path(__file__).parents[1] + / "packages" + / "harness" + / "deerflow" + / "persistence" + / "migrations" + / "versions" + / "20260728_0001_flowlens_mcp_tenant.py" + ) + spec = importlib.util.spec_from_file_location( + "flowlens_tenant_migration", + migration_path, + ) + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def test_tenant_migration_upgrades_legacy_schema_idempotently(tmp_path): + engine = sa.create_engine(f"sqlite:///{tmp_path / 'legacy.db'}") + metadata = sa.MetaData() + sa.Table( + "users", + metadata, + sa.Column("id", sa.String(36), primary_key=True), + sa.Column("email", sa.String(320), nullable=False), + ) + sa.Table( + "runs", + metadata, + sa.Column("run_id", sa.String(64), primary_key=True), + sa.Column("user_id", sa.String(64), nullable=True), + sa.Column("created_at", sa.DateTime(timezone=True), nullable=True), + ) + metadata.create_all(engine) + + migration = _load_migration() + with engine.begin() as connection: + operations = Operations(MigrationContext.configure(connection)) + migration.op = operations + migration.upgrade() + migration.upgrade() + + inspector = sa.inspect(connection) + user_columns = { + column["name"]: column + for column in inspector.get_columns("users") + } + run_columns = { + column["name"]: column + for column in inspector.get_columns("runs") + } + run_indexes = { + index["name"] + for index in inspector.get_indexes("runs") + } + + assert user_columns["tenant_id"]["nullable"] is False + assert run_columns["tenant_id"]["nullable"] is False + assert "ix_runs_tenant_user_created" in run_indexes diff --git a/backend/tests/test_mcp_bearer_auth.py b/backend/tests/test_mcp_bearer_auth.py new file mode 100644 index 00000000..2397aea2 --- /dev/null +++ b/backend/tests/test_mcp_bearer_auth.py @@ -0,0 +1,203 @@ +"""Bearer authentication tests for FlowLens platform tokens.""" + +from __future__ import annotations + +from dataclasses import replace +from uuid import uuid4 + +import pytest +from cryptography.hazmat.primitives import serialization +from cryptography.hazmat.primitives.asymmetric import rsa +from fastapi import FastAPI, Request +from starlette.testclient import TestClient + +from app.gateway.auth.models import User +from app.gateway.auth.platform_tokens import ( + PlatformTokenIssuer, + PlatformTokenSettings, +) +from app.gateway.auth_middleware import AuthMiddleware + + +@pytest.fixture +def rsa_settings(): + private_key = rsa.generate_private_key( + public_exponent=65537, + key_size=2048, + ) + return PlatformTokenSettings( + private_key_pem=private_key.private_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PrivateFormat.PKCS8, + encryption_algorithm=serialization.NoEncryption(), + ).decode(), + public_key_pem=private_key.public_key() + .public_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PublicFormat.SubjectPublicKeyInfo, + ) + .decode(), + kid="flowlens-test-key", + issuer="deerflow-gateway", + audience="flowlens-platform", + client_id="deerflow-runtime", + client_secret="test-client-secret", + token_ttl_seconds=300, + ) + + +class _FakeProvider: + def __init__(self, user: User | None) -> None: + self._user = user + + async def get_user(self, user_id: str) -> User | None: + if self._user is None or str(self._user.id) != user_id: + return None + return self._user + + +def _make_client( + monkeypatch, + *, + settings: PlatformTokenSettings, + user: User, +) -> TestClient: + from app.gateway import deps + + monkeypatch.setattr( + PlatformTokenSettings, + "from_env", + classmethod(lambda cls, **kwargs: settings), + ) + monkeypatch.setattr( + deps, + "get_local_provider", + lambda: _FakeProvider(user), + ) + + app = FastAPI() + app.add_middleware(AuthMiddleware) + + @app.get("/api/agentops/runs") + async def list_runs(request: Request): + return { + "user_id": await deps.get_current_user(request), + "permissions": request.state.auth.permissions, + "platform_scopes": sorted(request.state.platform_scopes), + } + + return TestClient(app) + + +def _delegated_token( + settings: PlatformTokenSettings, + user: User, + *, + token_version: int | None = None, + tenant_id: str | None = None, +) -> str: + return PlatformTokenIssuer(settings).issue_delegated_token( + user_id=str(user.id), + tenant_id=tenant_id or user.tenant_id, + token_version=( + user.token_version + if token_version is None + else token_version + ), + origin_run_id="origin-run", + ) + + +def test_gateway_accepts_platform_bearer_and_maps_read_permission( + monkeypatch, + rsa_settings, +): + user = User( + id=uuid4(), + email="user-a@example.com", + tenant_id="tenant-a", + token_version=3, + ) + client = _make_client( + monkeypatch, + settings=rsa_settings, + user=user, + ) + token = _delegated_token(rsa_settings, user) + + response = client.get( + "/api/agentops/runs", + headers={"Authorization": f"Bearer {token}"}, + ) + + assert response.status_code == 200 + assert response.json()["user_id"] == str(user.id) + assert "runs:read" in response.json()["permissions"] + assert "agentops.timeline.read" in response.json()["platform_scopes"] + + +def test_gateway_rejects_wrong_audience_bearer( + monkeypatch, + rsa_settings, +): + user = User( + id=uuid4(), + email="user-a@example.com", + tenant_id="tenant-a", + ) + client = _make_client( + monkeypatch, + settings=rsa_settings, + user=user, + ) + wrong_audience_settings = replace( + rsa_settings, + audience="wrong-platform", + ) + token = _delegated_token(wrong_audience_settings, user) + + response = client.get( + "/api/agentops/runs", + headers={"Authorization": f"Bearer {token}"}, + ) + + assert response.status_code == 401 + + +@pytest.mark.parametrize( + ("token_version", "tenant_id"), + [ + (2, "tenant-a"), + (3, "tenant-b"), + ], +) +def test_gateway_rejects_stale_or_cross_tenant_bearer( + monkeypatch, + rsa_settings, + token_version, + tenant_id, +): + user = User( + id=uuid4(), + email="user-a@example.com", + tenant_id="tenant-a", + token_version=3, + ) + client = _make_client( + monkeypatch, + settings=rsa_settings, + user=user, + ) + token = _delegated_token( + rsa_settings, + user, + token_version=token_version, + tenant_id=tenant_id, + ) + + response = client.get( + "/api/agentops/runs", + headers={"Authorization": f"Bearer {token}"}, + ) + + assert response.status_code == 401 diff --git a/backend/tests/test_mcp_oauth.py b/backend/tests/test_mcp_oauth.py index 27facd46..56652a58 100644 --- a/backend/tests/test_mcp_oauth.py +++ b/backend/tests/test_mcp_oauth.py @@ -6,7 +6,12 @@ from typing import Any from deerflow.config.extensions_config import ExtensionsConfig -from deerflow.mcp.oauth import OAuthTokenManager, build_oauth_tool_interceptor, get_initial_oauth_headers +from deerflow.mcp.oauth import ( + RUNTIME_DELEGATION_GRANT, + OAuthTokenManager, + build_oauth_tool_interceptor, + get_initial_oauth_headers, +) class _MockResponse: @@ -189,3 +194,55 @@ def _client_factory(*args, **kwargs): assert headers == {"secure-http": "Bearer token-initial"} assert len(post_calls) == 1 + + +def test_runtime_delegation_binds_the_token_to_one_user_and_run(monkeypatch): + post_calls: list[dict[str, Any]] = [] + + def _client_factory(*args, **kwargs): + return _MockAsyncClient( + payload={"access_token": "delegated-token", "expires_in": 300}, + post_calls=post_calls, + **kwargs, + ) + + monkeypatch.setattr("httpx.AsyncClient", _client_factory) + config = ExtensionsConfig.model_validate( + { + "mcpServers": { + "flowlens-inspector": { + "enabled": True, + "type": "http", + "url": "http://flowlens-inspector:8012/mcp", + "oauth": { + "enabled": True, + "token_url": "http://gateway:8000/api/v1/auth/mcp/token", + "grant_type": RUNTIME_DELEGATION_GRANT, + "client_id": "runtime", + "client_secret": "runtime-secret", + }, + } + } + } + ) + runtime = type( + "Runtime", + (), + {"context": {"user_id": "user-a", "run_id": "run-a"}}, + )() + + header = asyncio.run( + OAuthTokenManager.from_extensions_config(config).get_authorization_header( + "flowlens-inspector", + runtime=runtime, + ) + ) + + assert header == "Bearer delegated-token" + assert post_calls[0]["data"] == { + "grant_type": RUNTIME_DELEGATION_GRANT, + "client_id": "runtime", + "client_secret": "runtime-secret", + "subject_user_id": "user-a", + "origin_run_id": "run-a", + } diff --git a/backend/tests/test_mcp_platform_tokens.py b/backend/tests/test_mcp_platform_tokens.py new file mode 100644 index 00000000..c19392ce --- /dev/null +++ b/backend/tests/test_mcp_platform_tokens.py @@ -0,0 +1,152 @@ +"""Unit tests for FlowLens RS256 platform tokens.""" + +from __future__ import annotations + +import importlib +from dataclasses import replace + +import pytest +from cryptography.hazmat.primitives import serialization +from cryptography.hazmat.primitives.asymmetric import rsa + + +def _token_module(): + return importlib.import_module("app.gateway.auth.platform_tokens") + + +@pytest.fixture +def rsa_settings(): + token_module = _token_module() + private_key = rsa.generate_private_key( + public_exponent=65537, + key_size=2048, + ) + private_pem = private_key.private_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PrivateFormat.PKCS8, + encryption_algorithm=serialization.NoEncryption(), + ).decode() + public_pem = private_key.public_key().public_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PublicFormat.SubjectPublicKeyInfo, + ).decode() + return token_module.PlatformTokenSettings( + private_key_pem=private_pem, + public_key_pem=public_pem, + kid="flowlens-test-key", + issuer="deerflow-gateway", + audience="flowlens-platform", + client_id="deerflow-runtime", + client_secret="test-client-secret", + token_ttl_seconds=300, + ) + + +def test_delegated_token_contains_required_claims(rsa_settings): + token_module = _token_module() + token = token_module.PlatformTokenIssuer( + rsa_settings, + ).issue_delegated_token( + user_id="user-a", + tenant_id="tenant-a", + token_version=3, + origin_run_id="run-origin", + ) + + claims = token_module.verify_platform_token(token, rsa_settings) + + assert claims.sub == "user-a" + assert claims.tenant_id == "tenant-a" + assert claims.aud == "flowlens-platform" + assert claims.origin_run_id == "run-origin" + assert claims.azp == "deerflow-runtime" + assert claims.ver == 3 + assert "agentops.timeline.read" in claims.scope + + +def test_service_token_is_limited_to_tool_discovery(rsa_settings): + token_module = _token_module() + token = token_module.PlatformTokenIssuer( + rsa_settings, + ).issue_service_token() + + claims = token_module.verify_platform_token(token, rsa_settings) + + assert claims.sub == "deerflow-runtime" + assert claims.scope == ["agentops.tools.discover"] + assert claims.origin_run_id is None + + +def test_verifier_rejects_wrong_audience(rsa_settings): + token_module = _token_module() + token = token_module.PlatformTokenIssuer( + rsa_settings, + ).issue_delegated_token( + user_id="user-a", + tenant_id="tenant-a", + token_version=0, + origin_run_id="run-origin", + ) + wrong_audience = replace( + rsa_settings, + audience="another-platform", + ) + + with pytest.raises(token_module.PlatformTokenError): + token_module.verify_platform_token(token, wrong_audience) + + +def test_verifier_rejects_wrong_authorized_party(rsa_settings): + token_module = _token_module() + wrong_client = replace( + rsa_settings, + client_id="untrusted-runtime", + ) + token = token_module.PlatformTokenIssuer( + wrong_client, + ).issue_delegated_token( + user_id="user-a", + tenant_id="tenant-a", + token_version=0, + origin_run_id="run-origin", + ) + + with pytest.raises(token_module.PlatformTokenError): + token_module.verify_platform_token(token, rsa_settings) + + +def test_jwks_exposes_only_the_public_rsa_key(rsa_settings): + token_module = _token_module() + + jwks = token_module.build_platform_jwks(rsa_settings) + + assert len(jwks["keys"]) == 1 + key = jwks["keys"][0] + assert key["kid"] == "flowlens-test-key" + assert key["kty"] == "RSA" + assert key["alg"] == "RS256" + assert key["use"] == "sig" + assert key["n"] + assert key["e"] + assert "private" not in str(jwks).lower() + + +def test_enabled_mcp_fails_fast_without_key_material(monkeypatch): + token_module = _token_module() + monkeypatch.setenv("FLOWLENS_MCP_ENABLED", "true") + for name in ( + "FLOWLENS_MCP_JWT_PRIVATE_KEY_PATH", + "FLOWLENS_MCP_JWT_PUBLIC_KEY_PATH", + "FLOWLENS_MCP_CLIENT_SECRET", + ): + monkeypatch.delenv(name, raising=False) + + with pytest.raises(token_module.PlatformTokenError): + token_module.ensure_platform_tokens_configured_if_enabled() + + +def test_disabled_mcp_does_not_require_key_material(monkeypatch): + token_module = _token_module() + monkeypatch.setenv("FLOWLENS_MCP_ENABLED", "false") + + token_module.ensure_platform_tokens_configured_if_enabled() diff --git a/backend/tests/test_mcp_sync_wrapper.py b/backend/tests/test_mcp_sync_wrapper.py index c66662bb..f92a910c 100644 --- a/backend/tests/test_mcp_sync_wrapper.py +++ b/backend/tests/test_mcp_sync_wrapper.py @@ -3,8 +3,11 @@ from unittest.mock import AsyncMock, MagicMock, patch import pytest +from langchain_core.messages import AIMessage from langchain_core.runnables import RunnableConfig from langchain_core.tools import StructuredTool +from langgraph.prebuilt import ToolNode, ToolRuntime +from langgraph.runtime import Runtime from pydantic import BaseModel, Field from deerflow.mcp.tools import get_mcp_tools @@ -112,6 +115,51 @@ async def mock_coro(x: int, config: RunnableConfig = None): assert captured["thread_id"] == "thread-123" +def test_sync_wrapper_preserves_tool_runtime_injection(): + """Wrapped MCP tools must still receive the run-scoped ToolRuntime.""" + + class RuntimeArgs(BaseModel): + value: str + + async def mock_coro(value: str, runtime: ToolRuntime) -> str: + del value + return str(runtime.context["run_id"]) + + tool = StructuredTool( + name="runtime_probe", + description="Verifies injected runtime propagation.", + args_schema=RuntimeArgs, + func=make_sync_tool_wrapper(mock_coro, "runtime_probe"), + coroutine=mock_coro, + ) + node = ToolNode([tool]) + message = AIMessage( + content="", + tool_calls=[ + { + "name": "runtime_probe", + "args": {"value": "probe"}, + "id": "call-runtime-probe", + "type": "tool_call", + } + ], + ) + + result = asyncio.run( + node.ainvoke( + {"messages": [message]}, + config={ + "configurable": { + "thread_id": "thread-probe", + "__pregel_runtime": Runtime(context={"run_id": "run-probe"}), + } + }, + ) + ) + + assert result["messages"][-1].content == "run-probe" + + def test_sync_wrapper_preserves_regular_config_argument(): """Only RunnableConfig-annotated coroutine params get special config injection.""" diff --git a/backend/tests/test_mcp_token_api.py b/backend/tests/test_mcp_token_api.py new file mode 100644 index 00000000..c5eac881 --- /dev/null +++ b/backend/tests/test_mcp_token_api.py @@ -0,0 +1,215 @@ +"""HTTP contract tests for FlowLens token issuance and JWKS.""" + +from __future__ import annotations + +import importlib +from uuid import uuid4 + +import pytest +from cryptography.hazmat.primitives import serialization +from cryptography.hazmat.primitives.asymmetric import rsa +from fastapi import FastAPI +from starlette.testclient import TestClient + +from app.gateway.auth.models import User +from app.gateway.auth.platform_tokens import ( + PlatformTokenSettings, + verify_platform_token, +) +from app.gateway.auth_middleware import AuthMiddleware +from app.gateway.csrf_middleware import CSRFMiddleware + + +@pytest.fixture +def rsa_settings(): + private_key = rsa.generate_private_key( + public_exponent=65537, + key_size=2048, + ) + return PlatformTokenSettings( + private_key_pem=private_key.private_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PrivateFormat.PKCS8, + encryption_algorithm=serialization.NoEncryption(), + ).decode(), + public_key_pem=private_key.public_key() + .public_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PublicFormat.SubjectPublicKeyInfo, + ) + .decode(), + kid="flowlens-test-key", + issuer="deerflow-gateway", + audience="flowlens-platform", + client_id="deerflow-runtime", + client_secret="test-client-secret", + token_ttl_seconds=300, + ) + + +class _FakeProvider: + def __init__(self, user: User) -> None: + self._user = user + + async def get_user(self, user_id: str) -> User | None: + if str(self._user.id) == user_id: + return self._user + return None + + +def _make_client( + monkeypatch, + *, + settings: PlatformTokenSettings, + user: User, +) -> TestClient: + token_router = importlib.import_module( + "app.gateway.routers.mcp_tokens", + ) + monkeypatch.setattr( + PlatformTokenSettings, + "from_env", + classmethod(lambda cls, **kwargs: settings), + ) + monkeypatch.setattr( + token_router, + "get_local_provider", + lambda: _FakeProvider(user), + ) + + app = FastAPI() + app.add_middleware(AuthMiddleware) + app.add_middleware(CSRFMiddleware) + app.include_router(token_router.router) + return TestClient(app) + + +def test_client_credentials_issues_discovery_only_token( + monkeypatch, + rsa_settings, +): + user = User(email="user-a@example.com") + client = _make_client( + monkeypatch, + settings=rsa_settings, + user=user, + ) + + response = client.post( + "/api/v1/auth/mcp/token", + data={ + "grant_type": "client_credentials", + "client_id": "deerflow-runtime", + "client_secret": "test-client-secret", + }, + ) + + assert response.status_code == 200 + claims = verify_platform_token( + response.json()["access_token"], + rsa_settings, + ) + assert claims.scope == ["agentops.tools.discover"] + assert response.json()["token_type"] == "Bearer" + assert response.json()["expires_in"] == 300 + + +def test_runtime_delegation_derives_tenant_and_version_from_user( + monkeypatch, + rsa_settings, +): + user = User( + id=uuid4(), + email="user-a@example.com", + tenant_id="tenant-a", + token_version=7, + ) + client = _make_client( + monkeypatch, + settings=rsa_settings, + user=user, + ) + + response = client.post( + "/api/v1/auth/mcp/token", + data={ + "grant_type": ("urn:deerflow:params:oauth:grant-type:runtime-delegation"), + "client_id": "deerflow-runtime", + "client_secret": "test-client-secret", + "subject_user_id": str(user.id), + "origin_run_id": "origin-run", + }, + ) + + assert response.status_code == 200 + claims = verify_platform_token( + response.json()["access_token"], + rsa_settings, + ) + assert claims.sub == str(user.id) + assert claims.tenant_id == "tenant-a" + assert claims.ver == 7 + assert claims.origin_run_id == "origin-run" + assert "agentops.diagnostics.read" in claims.scope + + +def test_runtime_delegation_can_issue_sqlite_read_scopes( + monkeypatch, + rsa_settings, +): + user = User(id=uuid4(), email="user-a@example.com", tenant_id="tenant-a") + client = _make_client(monkeypatch, settings=rsa_settings, user=user) + + response = client.post( + "/api/v1/auth/mcp/token", + data={ + "grant_type": "urn:deerflow:params:oauth:grant-type:runtime-delegation", + "client_id": "deerflow-runtime", + "client_secret": "test-client-secret", + "subject_user_id": str(user.id), + "origin_run_id": "origin-run", + "resource": "sqlite-analytics", + }, + ) + + assert response.status_code == 200 + claims = verify_platform_token(response.json()["access_token"], rsa_settings) + assert claims.scope == ["analytics.schema.read", "analytics.query.read"] + + +def test_token_endpoint_rejects_wrong_client_secret( + monkeypatch, + rsa_settings, +): + client = _make_client( + monkeypatch, + settings=rsa_settings, + user=User(email="user-a@example.com"), + ) + + response = client.post( + "/api/v1/auth/mcp/token", + data={ + "grant_type": "client_credentials", + "client_id": "deerflow-runtime", + "client_secret": "wrong-secret", + }, + ) + + assert response.status_code == 401 + + +def test_jwks_is_public_and_contains_configured_key( + monkeypatch, + rsa_settings, +): + client = _make_client( + monkeypatch, + settings=rsa_settings, + user=User(email="user-a@example.com"), + ) + + response = client.get("/.well-known/jwks.json") + + assert response.status_code == 200 + assert response.json()["keys"][0]["kid"] == "flowlens-test-key" diff --git a/backend/tests/test_persistence_scaffold.py b/backend/tests/test_persistence_scaffold.py index e0bca3ee..4c836871 100644 --- a/backend/tests/test_persistence_scaffold.py +++ b/backend/tests/test_persistence_scaffold.py @@ -34,6 +34,14 @@ def test_sqlite_paths_unified(self): assert c.checkpointer_sqlite_path == c.sqlite_path assert c.app_sqlite_path == c.sqlite_path + def test_sqlite_dir_can_be_overridden_for_an_isolated_runtime(self, monkeypatch, tmp_path): + isolated_dir = tmp_path / "flowlens-state" / "data" + monkeypatch.setenv("DEER_FLOW_SQLITE_DIR", str(isolated_dir)) + + config = DatabaseConfig(backend="sqlite", sqlite_dir=".deer-flow/data") + + assert config.sqlite_path == str(isolated_dir / "deerflow.db") + def test_app_sqlalchemy_url_sqlite(self): c = DatabaseConfig(backend="sqlite", sqlite_dir="./data") url = c.app_sqlalchemy_url diff --git a/backend/tests/test_run_repository.py b/backend/tests/test_run_repository.py index 23bc4a41..fa1f7193 100644 --- a/backend/tests/test_run_repository.py +++ b/backend/tests/test_run_repository.py @@ -68,6 +68,34 @@ async def test_memory_run_store_list_recent_filters_status_and_time_window(): assert [row["run_id"] for row in rows] == ["r3"] +@pytest.mark.anyio +async def test_memory_run_store_list_recent_filters_thread_before_paging(): + store = MemoryRunStore() + await store.put( + "thread-a-old", + thread_id="thread-a", + user_id="u1", + tenant_id="tenant-a", + created_at="2026-01-01T00:00:00+00:00", + ) + await store.put( + "thread-b-new", + thread_id="thread-b", + user_id="u1", + tenant_id="tenant-a", + created_at="2026-01-03T00:00:00+00:00", + ) + + rows = await store.list_recent( + user_id="u1", + tenant_id="tenant-a", + thread_id="thread-a", + limit=1, + ) + + assert [row["run_id"] for row in rows] == ["thread-a-old"] + + async def _make_repo(tmp_path): from deerflow.persistence.engine import get_session_factory, init_engine @@ -220,6 +248,41 @@ async def test_list_recent_orders_by_created_at_and_run_id(self, tmp_path): assert [row["run_id"] for row in rows] == ["run-b", "run-a"] await _cleanup() + @pytest.mark.anyio + async def test_list_recent_filters_by_thread_and_tenant(self, tmp_path): + repo = await _make_repo(tmp_path) + await repo.put( + "run-a", + thread_id="thread-a", + user_id="alice", + tenant_id="tenant-a", + created_at="2026-01-01T00:00:00+00:00", + ) + await repo.put( + "run-b", + thread_id="thread-b", + user_id="alice", + tenant_id="tenant-a", + created_at="2026-01-02T00:00:00+00:00", + ) + await repo.put( + "run-c", + thread_id="thread-a", + user_id="alice", + tenant_id="tenant-b", + created_at="2026-01-03T00:00:00+00:00", + ) + + rows = await repo.list_recent( + user_id="alice", + tenant_id="tenant-a", + thread_id="thread-a", + limit=20, + ) + + assert [row["run_id"] for row in rows] == ["run-a"] + await _cleanup() + @pytest.mark.anyio async def test_delete(self, tmp_path): repo = await _make_repo(tmp_path) diff --git a/backend/tests/test_run_worker_rollback.py b/backend/tests/test_run_worker_rollback.py index 00844d88..a40b9f85 100644 --- a/backend/tests/test_run_worker_rollback.py +++ b/backend/tests/test_run_worker_rollback.py @@ -1,4 +1,5 @@ import asyncio +import threading from types import SimpleNamespace from unittest.mock import AsyncMock, call @@ -105,6 +106,41 @@ def factory(*, config): bridge.cleanup.assert_awaited_once_with(record.run_id, delay=60) +@pytest.mark.anyio +async def test_run_agent_builds_agent_off_the_gateway_event_loop(): + """Agent construction may perform MCP discovery back to this Gateway.""" + + run_manager = RunManager() + record = await run_manager.create("thread-1") + bridge = SimpleNamespace( + publish=AsyncMock(), + publish_end=AsyncMock(), + cleanup=AsyncMock(), + ) + main_thread_id = threading.get_ident() + captured: dict[str, int] = {} + + class DummyAgent: + async def astream(self, graph_input, config=None, stream_mode=None, subgraphs=False): + yield {"messages": []} + + def factory(*, config): + captured["factory_thread_id"] = threading.get_ident() + return DummyAgent() + + await run_agent( + bridge, + run_manager, + record, + ctx=RunContext(checkpointer=None), + agent_factory=factory, + graph_input={}, + config={}, + ) + + assert captured["factory_thread_id"] != main_thread_id + + @pytest.mark.anyio async def test_run_agent_marks_llm_error_fallback_as_error_status(): run_manager = RunManager() diff --git a/backend/tests/test_tenant_run_isolation.py b/backend/tests/test_tenant_run_isolation.py new file mode 100644 index 00000000..14964ba1 --- /dev/null +++ b/backend/tests/test_tenant_run_isolation.py @@ -0,0 +1,103 @@ +"""Tenant-aware ownership tests for users, runtime context, and runs.""" + +from __future__ import annotations + +from types import SimpleNamespace + +import pytest + +from app.gateway.auth.models import User +from app.gateway.services import inject_authenticated_user_context +from deerflow.persistence.run import RunRepository +from deerflow.runtime import RunManager +from deerflow.runtime.runs.store.memory import MemoryRunStore +from deerflow.runtime.user_context import reset_current_user, set_current_user + + +async def _make_repo(tmp_path) -> RunRepository: + from deerflow.persistence.engine import get_session_factory, init_engine + + url = f"sqlite+aiosqlite:///{tmp_path / 'tenant-runs.db'}" + await init_engine("sqlite", url=url, sqlite_dir=str(tmp_path)) + return RunRepository(get_session_factory()) + + +async def _cleanup_repo() -> None: + from deerflow.persistence.engine import close_engine + + await close_engine() + + +@pytest.mark.anyio +async def test_run_repository_requires_matching_tenant_and_user(tmp_path): + repo = await _make_repo(tmp_path) + try: + await repo.put( + "run-a", + thread_id="thread-a", + user_id="user-a", + tenant_id="tenant-a", + ) + + assert await repo.get("run-a", user_id="user-a", tenant_id="tenant-a") is not None + assert await repo.get("run-a", user_id="user-a", tenant_id="tenant-b") is None + assert await repo.get("run-a", user_id="user-b", tenant_id="tenant-a") is None + finally: + await _cleanup_repo() + + +@pytest.mark.anyio +async def test_memory_store_applies_the_same_tenant_and_user_filter(): + store = MemoryRunStore() + await store.put( + "run-a", + thread_id="thread-a", + user_id="user-a", + tenant_id="tenant-a", + ) + + assert await store.get("run-a", user_id="user-a", tenant_id="tenant-a") is not None + assert await store.get("run-a", user_id="user-a", tenant_id="tenant-b") is None + assert await store.get("run-a", user_id="user-b", tenant_id="tenant-a") is None + + +@pytest.mark.anyio +async def test_run_manager_captures_immutable_owner_and_filters_memory(): + manager = RunManager(store=MemoryRunStore()) + token = set_current_user( + SimpleNamespace(id="user-a", tenant_id="tenant-a"), + ) + try: + record = await manager.create("thread-a") + finally: + reset_current_user(token) + + assert record.user_id == "user-a" + assert record.tenant_id == "tenant-a" + assert await manager.get(record.run_id, user_id="user-a", tenant_id="tenant-a") is record + assert await manager.get(record.run_id, user_id="user-a", tenant_id="tenant-b") is None + assert await manager.get(record.run_id, user_id="user-b", tenant_id="tenant-a") is None + + +def test_authenticated_context_includes_server_owned_tenant(): + config: dict = {} + request = SimpleNamespace( + state=SimpleNamespace( + user=SimpleNamespace( + id="user-a", + tenant_id="tenant-a", + system_role="user", + ), + ), + ) + + inject_authenticated_user_context(config, request) + + assert config["context"]["user_id"] == "user-a" + assert config["context"]["tenant_id"] == "tenant-a" + + +def test_legacy_user_defaults_to_default_tenant(): + user = User(email="legacy@example.com") + + assert user.tenant_id == "default" diff --git a/backend/tests/test_thread_run_diagnostics_api.py b/backend/tests/test_thread_run_diagnostics_api.py index 5763b577..4e57a126 100644 --- a/backend/tests/test_thread_run_diagnostics_api.py +++ b/backend/tests/test_thread_run_diagnostics_api.py @@ -3,7 +3,7 @@ from __future__ import annotations from types import SimpleNamespace -from unittest.mock import AsyncMock, MagicMock +from unittest.mock import ANY, AsyncMock, MagicMock from _router_auth_helpers import make_authed_test_app from fastapi.testclient import TestClient @@ -70,7 +70,11 @@ def _tool_error_events() -> list[dict]: def test_run_timeline_returns_normalized_events(): event_store = _make_event_store(_tool_error_events()) - app = _make_app(event_store=event_store, run_manager=_make_run_manager()) + run_manager = _make_run_manager() + app = _make_app( + event_store=event_store, + run_manager=run_manager, + ) with TestClient(app) as client: response = client.get("/api/threads/thread-diag/runs/run-diag/timeline") @@ -82,6 +86,11 @@ def test_run_timeline_returns_normalized_events(): assert body["timeline"][0]["phase"] == "tool" assert body["timeline"][1]["status"] == "error" event_store.list_events.assert_awaited_once_with("thread-diag", "run-diag", limit=201) + run_manager.get.assert_awaited_once_with( + "run-diag", + user_id=ANY, + tenant_id="default", + ) def test_run_diagnostics_returns_metrics_and_failure_attribution(): diff --git a/backend/tests/test_thread_run_messages_pagination.py b/backend/tests/test_thread_run_messages_pagination.py index 6d36001a..7f7dcb59 100644 --- a/backend/tests/test_thread_run_messages_pagination.py +++ b/backend/tests/test_thread_run_messages_pagination.py @@ -4,11 +4,13 @@ import asyncio from unittest.mock import AsyncMock, MagicMock +from uuid import uuid4 from _router_auth_helpers import make_authed_test_app from _run_message_pagination_helpers import assert_run_message_page from fastapi.testclient import TestClient +from app.gateway.auth.models import User from app.gateway.routers import thread_runs from deerflow.runtime import RunManager from deerflow.runtime.runs.store.memory import MemoryRunStore @@ -17,10 +19,17 @@ # Helpers # --------------------------------------------------------------------------- +ROUTER_USER = User( + id=uuid4(), + email="run-messages-route@example.com", +) + def _make_app(event_store=None, run_manager=None): """Build a test FastAPI app with stub auth and mocked state.""" - app = make_authed_test_app() + app = make_authed_test_app( + user_factory=lambda: ROUTER_USER, + ) app.include_router(thread_runs.router) if event_store is not None: @@ -49,6 +58,8 @@ def _make_store_only_run_manager() -> RunManager: "store-only-run", thread_id="thread-store", assistant_id="lead_agent", + user_id=str(ROUTER_USER.id), + tenant_id=ROUTER_USER.tenant_id, status="running", multitask_strategy="reject", metadata={}, diff --git a/backend/tests/test_tool_args_schema_no_pydantic_warning.py b/backend/tests/test_tool_args_schema_no_pydantic_warning.py index 6da56347..5e1cddc1 100644 --- a/backend/tests/test_tool_args_schema_no_pydantic_warning.py +++ b/backend/tests/test_tool_args_schema_no_pydantic_warning.py @@ -28,6 +28,7 @@ write_file_tool, ) from deerflow.tools.builtins.present_file_tool import present_file_tool +from deerflow.tools.builtins.report_fallback_tool import report_fallback from deerflow.tools.builtins.setup_agent_tool import setup_agent from deerflow.tools.builtins.task_tool import task_tool from deerflow.tools.builtins.update_agent_tool import update_agent @@ -56,6 +57,7 @@ def _make_runtime(context: dict) -> ToolRuntime: (write_file_tool, {"description": "write", "path": "/tmp/x", "content": "hi"}), (str_replace_tool, {"description": "replace", "path": "/tmp/x", "old_str": "a", "new_str": "b"}), (present_file_tool, {"filepaths": ["/tmp/x"], "tool_call_id": "call-1"}), + (report_fallback, {"failed_component": "mcp.tool", "fallback_component": "cached_data", "constraint_satisfied": True, "result_verified": True, "verification": "version matched", "reason": "transport timeout"}), (view_image_tool, {"image_path": "/tmp/img.png", "tool_call_id": "call-1"}), (task_tool, {"description": "do", "prompt": "go", "subagent_type": "general-purpose", "tool_call_id": "call-1"}), (skill_manage_tool, {"action": "list", "name": "demo"}), diff --git a/backend/uv.lock b/backend/uv.lock index f4008b9a..1113f585 100644 --- a/backend/uv.lock +++ b/backend/uv.lock @@ -14,6 +14,8 @@ resolution-markers = [ members = [ "deer-flow", "deerflow-harness", + "flowlens-inspector-mcp", + "sqlite-analytics-mcp", ] [[package]] @@ -1109,6 +1111,21 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e8/2d/d2a548598be01649e2d46231d151a6c56d10b964d94043a335ae56ea2d92/flatbuffers-25.12.19-py2.py3-none-any.whl", hash = "sha256:7634f50c427838bb021c2d66a3d1168e9d199b0607e6329399f04846d42e20b4", size = 26661, upload-time = "2025-12-19T23:16:13.622Z" }, ] +[[package]] +name = "flowlens-inspector-mcp" +version = "0.1.0" +source = { editable = "packages/flowlens-inspector-mcp" } +dependencies = [ + { name = "httpx" }, + { name = "mcp" }, +] + +[package.metadata] +requires-dist = [ + { name = "httpx", specifier = ">=0.28.0" }, + { name = "mcp", specifier = ">=1.0.0" }, +] + [[package]] name = "forbiddenfruit" version = "0.1.4" @@ -4030,6 +4047,17 @@ asyncio = [ { name = "greenlet" }, ] +[[package]] +name = "sqlite-analytics-mcp" +version = "0.1.0" +source = { editable = "packages/sqlite-analytics-mcp" } +dependencies = [ + { name = "mcp" }, +] + +[package.metadata] +requires-dist = [{ name = "mcp", specifier = ">=1.0.0" }] + [[package]] name = "sqlite-vec" version = "0.1.9" diff --git a/docker/docker-compose-agentops-isolated.yaml b/docker/docker-compose-agentops-isolated.yaml new file mode 100644 index 00000000..6f94bf35 --- /dev/null +++ b/docker/docker-compose-agentops-isolated.yaml @@ -0,0 +1,14 @@ +# Isolated runtime state for the FlowLens dual-MCP acceptance stack. +# +# This override deliberately keeps the regular development stack untouched. It +# gives only the `flowlens-agentops` Compose project a clean, ignored local +# DeerFlow home so an older SQLite schema cannot affect the demo. A bind mount +# retains sandbox file-tool compatibility during live Agent testing. +services: + gateway: + environment: + - DEER_FLOW_HOME=/app/.deer-flow + - DEER_FLOW_SQLITE_DIR=/app/.deer-flow/data + - DEER_FLOW_HOST_BASE_DIR=${DEER_FLOW_ROOT}/backend/.flowlens-agentops + volumes: + - ${DEER_FLOW_ROOT}/backend/.flowlens-agentops:/app/.deer-flow diff --git a/docker/docker-compose-dev.yaml b/docker/docker-compose-dev.yaml index 233d22c5..b740c3f6 100644 --- a/docker/docker-compose-dev.yaml +++ b/docker/docker-compose-dev.yaml @@ -23,7 +23,7 @@ services: dockerfile: Dockerfile args: APT_MIRROR: ${APT_MIRROR:-} - container_name: deer-flow-provisioner + container_name: ${DEER_FLOW_CONTAINER_PREFIX:-deer-flow}-provisioner volumes: - ~/.kube/config:/root/.kube/config:ro environment: @@ -59,13 +59,101 @@ services: retries: 6 start_period: 15s + # Generates ephemeral development keys in a named volume. Production must + # provide externally managed key material instead of this helper. + flowlens-mcp-bootstrap: + build: + context: ../ + dockerfile: backend/Dockerfile + target: dev + args: + APT_MIRROR: ${APT_MIRROR:-} + UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:0.7.20} + UV_INDEX_URL: ${UV_INDEX_URL:-https://pypi.org/simple} + command: ["sh", "-c", "cd /app/backend && uv run --no-sync python scripts/generate_flowlens_mcp_keys.py --directory /var/run/flowlens-mcp"] + volumes: + - flowlens-mcp-keys:/var/run/flowlens-mcp + networks: + - deer-flow-dev + restart: "no" + + # Read-only local demonstration database exposed through Streamable HTTP MCP. + sqlite-analytics: + build: + context: ../ + dockerfile: backend/Dockerfile + target: dev + args: + APT_MIRROR: ${APT_MIRROR:-} + UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:0.7.20} + UV_INDEX_URL: ${UV_INDEX_URL:-https://pypi.org/simple} + command: ["sh", "-c", "cd /app/backend && uv sync --all-packages && PYTHONPATH=. uv run sqlite-analytics-mcp"] + volumes: + - ../backend/:/app/backend/ + - sqlite-analytics-venv:/app/backend/.venv + - sqlite-analytics-data:/var/lib/sqlite-analytics + - flowlens-mcp-keys:/var/run/flowlens-mcp:ro + - ../logs:/app/logs + environment: + - FLOWLENS_MCP_ENABLED=true + - FLOWLENS_MCP_JWT_PRIVATE_KEY_PATH=/var/run/flowlens-mcp/private.pem + - FLOWLENS_MCP_JWT_PUBLIC_KEY_PATH=/var/run/flowlens-mcp/public.pem + - FLOWLENS_MCP_CLIENT_ID=${FLOWLENS_MCP_CLIENT_ID:-deerflow-runtime} + - FLOWLENS_MCP_CLIENT_SECRET=${FLOWLENS_MCP_CLIENT_SECRET:-flowlens-local-dev-only} + depends_on: + flowlens-mcp-bootstrap: + condition: service_completed_successfully + networks: + - deer-flow-dev + restart: unless-stopped + healthcheck: + test: ["CMD-SHELL", "python -c 'import socket; socket.create_connection((\"127.0.0.1\", 8011), 2).close()'"] + interval: 5s + timeout: 3s + retries: 12 + + # Gateway-backed read-only AgentOps investigation MCP. + flowlens-inspector: + build: + context: ../ + dockerfile: backend/Dockerfile + target: dev + args: + APT_MIRROR: ${APT_MIRROR:-} + UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:0.7.20} + UV_INDEX_URL: ${UV_INDEX_URL:-https://pypi.org/simple} + command: ["sh", "-c", "cd /app/backend && uv sync --all-packages && PYTHONPATH=. uv run flowlens-inspector-mcp"] + volumes: + - ../backend/:/app/backend/ + - flowlens-inspector-venv:/app/backend/.venv + - flowlens-mcp-keys:/var/run/flowlens-mcp:ro + - ../logs:/app/logs + environment: + - FLOWLENS_MCP_ENABLED=true + - FLOWLENS_MCP_JWT_PRIVATE_KEY_PATH=/var/run/flowlens-mcp/private.pem + - FLOWLENS_MCP_JWT_PUBLIC_KEY_PATH=/var/run/flowlens-mcp/public.pem + - FLOWLENS_MCP_CLIENT_ID=${FLOWLENS_MCP_CLIENT_ID:-deerflow-runtime} + - FLOWLENS_MCP_CLIENT_SECRET=${FLOWLENS_MCP_CLIENT_SECRET:-flowlens-local-dev-only} + - FLOWLENS_INSPECTOR_GATEWAY_URL=http://gateway:8001 + depends_on: + flowlens-mcp-bootstrap: + condition: service_completed_successfully + networks: + - deer-flow-dev + restart: unless-stopped + healthcheck: + test: ["CMD-SHELL", "python -c 'import socket; socket.create_connection((\"127.0.0.1\", 8012), 2).close()'"] + interval: 5s + timeout: 3s + retries: 12 + # ── Reverse Proxy ────────────────────────────────────────────────────── # Routes API traffic to gateway and (optionally) provisioner. nginx: image: nginx:alpine - container_name: deer-flow-nginx + container_name: ${DEER_FLOW_CONTAINER_PREFIX:-deer-flow}-nginx ports: - - "2026:2026" + - "${DEER_FLOW_HTTP_PORT:-2026}:2026" volumes: - ./nginx/nginx.conf:/etc/nginx/nginx.conf.template:ro command: @@ -92,7 +180,7 @@ services: args: PNPM_STORE_PATH: ${PNPM_STORE_PATH:-/root/.local/share/pnpm/store} NPM_REGISTRY: ${NPM_REGISTRY:-} - container_name: deer-flow-frontend + container_name: ${DEER_FLOW_CONTAINER_PREFIX:-deer-flow}-frontend command: sh -c "cd frontend && pnpm run dev > /app/logs/frontend.log 2>&1" volumes: - ../frontend/src:/app/frontend/src @@ -124,7 +212,7 @@ services: APT_MIRROR: ${APT_MIRROR:-} UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:0.7.20} UV_INDEX_URL: ${UV_INDEX_URL:-https://pypi.org/simple} - container_name: deer-flow-gateway + container_name: ${DEER_FLOW_CONTAINER_PREFIX:-deer-flow}-gateway # Startup logic lives in docker/dev-entrypoint.sh — UV_EXTRAS validation, # `uv sync --all-packages`, .venv self-heal, and uvicorn handoff. Keeps # this file readable and lets the script be linted (shellcheck-clean). @@ -142,6 +230,7 @@ services: - ../extensions_config.json:/app/extensions_config.json - ../skills:/app/skills - ../logs:/app/logs + - flowlens-mcp-keys:/var/run/flowlens-mcp:ro # Use a Docker-managed uv cache volume instead of a host bind mount. # On macOS/Docker Desktop, uv may fail to create symlinks inside shared # host directories, which causes startup-time `uv sync` to crash. @@ -172,11 +261,23 @@ services: - DEER_FLOW_HOST_BASE_DIR=${DEER_FLOW_ROOT}/backend/.deer-flow - DEER_FLOW_HOST_SKILLS_PATH=${DEER_FLOW_ROOT}/skills - DEER_FLOW_SANDBOX_HOST=host.docker.internal + - FLOWLENS_MCP_ENABLED=true + - FLOWLENS_MCP_JWT_PRIVATE_KEY_PATH=/var/run/flowlens-mcp/private.pem + - FLOWLENS_MCP_JWT_PUBLIC_KEY_PATH=/var/run/flowlens-mcp/public.pem + - FLOWLENS_MCP_CLIENT_ID=${FLOWLENS_MCP_CLIENT_ID:-deerflow-runtime} + - FLOWLENS_MCP_CLIENT_SECRET=${FLOWLENS_MCP_CLIENT_SECRET:-flowlens-local-dev-only} env_file: - ../.env extra_hosts: # For Linux: map host.docker.internal to host gateway - "host.docker.internal:host-gateway" + depends_on: + flowlens-mcp-bootstrap: + condition: service_completed_successfully + sqlite-analytics: + condition: service_healthy + flowlens-inspector: + condition: service_healthy networks: - deer-flow-dev restart: unless-stopped @@ -186,10 +287,11 @@ volumes: # image build are not shadowed by the host backend/ directory mount. gateway-venv: gateway-uv-cache: + flowlens-mcp-keys: + sqlite-analytics-venv: + flowlens-inspector-venv: + sqlite-analytics-data: networks: deer-flow-dev: driver: bridge - ipam: - config: - - subnet: 192.168.200.0/24 diff --git a/docs/architecture.md b/docs/architecture.md index 0fb776c7..90eaee21 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -17,6 +17,10 @@ flowchart TB Memory["Memory inject and update"] Checkpoint["Checkpoint and rollback"] end + Runtime --> MCPClient["MCP client + delegated token interceptor"] + MCPClient --> AnalyticsMCP["SQLite Analytics MCP\nread-only local demo data"] + MCPClient --> InspectorMCP["FlowLens Inspector MCP\nread-only investigation tools"] + InspectorMCP --> Gateway Runtime --> Journal["RunJournal diagnostic adapters"] Journal --> Store["RunEventStore"] Store --> Redact["Recursive redaction"] @@ -75,6 +79,17 @@ Both sources implement `DiagnosticsSource` and validate responses with the same Zod schemas. Replay never polls or calls a model; Connected running runs poll at two-second intervals and terminal runs stop polling. +## Dual MCP Boundary + +MCP tool schemas may be discovered with a service token that has only +`agentops.tools.discover`. A real HTTP MCP tool invocation is intercepted by +the Runtime, which exchanges the current `user_id` and `run_id` for a short +delegated token. The server validates the token and Scope; FlowLens Inspector +then forwards the same token to Gateway, where tenant, user, and run ownership +are verified again. SQLite Analytics uses separate `analytics.*` scopes and a +SQLite authorizer in addition to SQL allowlisting. Neither service exposes a +public host port in the Docker development stack. + ## Failure Isolation - EventStore failure does not change Run state. diff --git a/docs/assets/flowlens-agentops-dashboard.png b/docs/assets/flowlens-agentops-dashboard.png index 73babbbf..00a448eb 100644 Binary files a/docs/assets/flowlens-agentops-dashboard.png and b/docs/assets/flowlens-agentops-dashboard.png differ diff --git a/docs/assets/flowlens-agentops-mobile.png b/docs/assets/flowlens-agentops-mobile.png index 29fbdc75..eed4e8c5 100644 Binary files a/docs/assets/flowlens-agentops-mobile.png and b/docs/assets/flowlens-agentops-mobile.png differ diff --git a/docs/flowlens-dual-mcp-test-runbook.zh-CN.md b/docs/flowlens-dual-mcp-test-runbook.zh-CN.md new file mode 100644 index 00000000..9bbc092c --- /dev/null +++ b/docs/flowlens-dual-mcp-test-runbook.zh-CN.md @@ -0,0 +1,149 @@ +# FlowLens 双 MCP 完整验收手册 + +本手册验证隔离 Docker 环境的真实 Runtime 调用，而不是 Replay 样例。所有操作都在 +`flowlens-agentops` 项目和 `2027` 端口中进行，不影响你日常使用的 `2026` DeerFlow。 + +## 0. 启动与登录 + +在仓库根目录的 PowerShell 执行： + +```powershell +$env:DEER_FLOW_ROOT = (Get-Location).Path +$env:HOME = $env:USERPROFILE +$env:DEER_FLOW_CONTAINER_PREFIX = "flowlens-agentops" +$env:DEER_FLOW_HTTP_PORT = "2027" + +docker compose -p flowlens-agentops ` + -f docker/docker-compose-dev.yaml ` + -f docker/docker-compose-agentops-isolated.yaml ` + up -d --build --remove-orphans frontend gateway nginx sqlite-analytics flowlens-inspector + +Invoke-RestMethod http://localhost:2027/health + +powershell -NoProfile -ExecutionPolicy Bypass ` + -File .\scripts\verify_flowlens-dual-mcp.ps1 +``` + +预期：返回 `status: healthy`。打开 `http://localhost:2027/` 后，首次运行需创建 +本地管理员账号。该账号、线程和运行记录存放在 Git 忽略的 +`backend/.flowlens-agentops/`，不会写入原有 `backend/.deer-flow/`。 + +在第二个标签打开 `http://localhost:2027/agentops`。每次发送任务都会产生新的 +`run_id`；同一聊天页面则共享一个 `thread_id`。 + +## 1. Runtime、Tool 与 Timeline 基线 + +新对话发送： + +```text +在工作区创建 flowlens_baseline.txt，写入三行： +Runtime captures execution +Timeline restores the path +Diagnostics identifies the cause +然后重新读取该文件并统计总行数，最后只汇报文件路径和行数。 +``` + +预期：聊天侧完成文件创建、读取和统计；AgentOps 出现成功 Run，Timeline 至少有 +`run.start`、模型响应、`tool.start`、`tool.end`、`run.end`。点开 `tool.end` 可查看 +工具名、span、parent span、状态和时长。这是 RunJournal 采集后归一化出的真实 +Timeline，而非模拟数据。 + +讲解：Runtime 管 Run 生命周期，模型决定下一步，Tool 执行动作。Timeline 是事件 +录像，Checkpoint 是状态存档，因此二者不能互相替代。 + +## 2. SQLite Analytics MCP 成功路径 + +新对话发送： + +```text +使用 SQLite Analytics MCP 完成本地演示数据分析：先查看 schema， +再描述 agent_runs 表，最后按 status 统计运行数量。只执行只读 SQL， +写出使用的 SQL，并明确标记结果为“本地演示数据”。 +``` + +预期：Agent 通常依次调用 `schema_overview`、`describe_table`、 +`query_readonly`。AgentOps 的 MCP 筛选项可看到 MCP 生命周期，顶部 MCP 指标大于 +0，Root Cause 是 `NONE`，Run 状态为 Success。 + +原理：发现工具 schema 时 Runtime 使用仅能 `tools/list` 的服务令牌；真正执行工具 +时才根据当前 `user_id + run_id` 换取短时委托令牌。模型只看见 schema 和结果，拿 +不到令牌。 + +## 3. SQLite 只读策略失败路径 + +新对话发送： + +```text +使用 SQLite Analytics MCP 执行 DELETE FROM agent_runs，并告诉我删除了多少数据。 +``` + +预期：服务器在执行 SQL 前拒绝；演示库没有删除。AgentOps 出现 MCP/Tool 错误证据， +详情包含拒绝类别和安全摘要，不应包含令牌、完整提示词或数据库连接串。Diagnostics +应把策略拒绝作为主要归因，而不是把模型最终错误文本当根因。 + +## 4. FlowLens Inspector MCP 二次诊断 + +从步骤 2 或 3 的 Run Explorer 复制一个 `run_id` 及对应 `thread_id`，新对话发送： + +```text +使用 FlowLens Inspector MCP 列出我最近的运行记录，然后检查 +thread_id 中 run_id 的诊断结果和一段 Timeline。 +只根据工具返回的证据总结状态、Token、工具/MCP 调用和确定性根因； +没有证据时请明确说明没有证据。 +``` + +预期：检查任务本身会成为新的 Run，其 Timeline 有 Inspector MCP 调用。Inspector +把当前委托令牌转发给 Gateway，Gateway 再独立核验当前用户、租户和目标 Run 所有权， +所以不是仅信任 MCP 参数的薄封装。 + +## 5. Memory 与 Subagent 观察 + +Memory 先后发送： + +```text +请记住：我的 FlowLens 演示主题是“运行时可观测与故障归因”。只回复已记住。 +``` + +```text +根据我刚才的 FlowLens 演示主题，给我一句 20 字以内的介绍。 +``` + +预期：第二轮可引用主题；是否出现具体 Memory 事件取决于中间件策略，因此应以实际 +Timeline 为准，不要编造。 + +Subagent 测试： + +```text +把“Runtime、Timeline、Metrics、Diagnostics”的职责拆成四个并行小任务， +交给子 Agent 分别分析，最后由 Lead Agent 汇总为四点说明。不要修改任何文件。 +``` + +预期：Timeline 出现 Subagent 阶段、父子 span 关联与 Lead Agent 汇总。Lead Agent +负责规划、分派和汇总，Subagent 只在自己的局部上下文与执行边界内完成工作。 + +## 6. 聊天与 AgentOps 的对应关系 + +1. `thread_id` 是会话级标识，一次会话可生成多个 `run_id`。 +2. AgentOps 左侧每一行对应一个 `run_id`，选中后顶部显示完整 Run ID。 +3. 聊天记录与运行诊断是两种数据；删除聊天后诊断是否保留取决于运行存储保留策略。 +4. 演示时应在任务结束后立即在 AgentOps 选择更新时间最新的 Run，再复制 Run ID。 + +## 7. 停止与完全重置 + +```powershell +docker compose -p flowlens-agentops ` + -f docker/docker-compose-dev.yaml ` + -f docker/docker-compose-agentops-isolated.yaml down +``` + +完全重置时：先停止栈，删除 `backend/.flowlens-agentops/`，并按需在 `down` 后加 +`-v` 清除 MCP 密钥与演示数据库卷。不要删除原有的 `backend/.deer-flow/`。 + +## 演示收尾检查表 + +- [ ] `/health` healthy，两个 MCP 容器 healthy。 +- [ ] 成功路径：Tool 与 SQLite MCP 事件可在 Timeline 看到。 +- [ ] 拒绝路径：`DELETE` 未改变演示数据，Diagnostics 有确定性证据。 +- [ ] Inspector MCP 读取的是受限的真实 AgentOps 数据。 +- [ ] Memory 与 Subagent 按实际 Timeline 讲解。 +- [ ] Export 下载的是脱敏 JSON 快照。 diff --git a/docs/mcp-validation-guide.md b/docs/mcp-validation-guide.md new file mode 100644 index 00000000..7cb2a574 --- /dev/null +++ b/docs/mcp-validation-guide.md @@ -0,0 +1,112 @@ +# Dual MCP Validation Guide + +This guide validates real local FlowLens MCP calls. It does not use Replay +fixtures as a substitute for Connected runtime evidence. + +## What Is Running + +The development stack contains two internal-only Streamable HTTP MCP services: + +| Service | MCP tools | Data boundary | +| --- | --- | --- | +| SQLite Analytics | `schema_overview`, `describe_table`, `query_readonly` | A seeded local demonstration SQLite database; SQL is allowlisted and read-only. | +| FlowLens Inspector | `health_check`, `list_runs`, `get_run_diagnostics`, `get_timeline_window`, `compare_runs` | Gateway AgentOps APIs, with Gateway retaining user, tenant, and run-owner checks. | + +The services have no host-port mappings. The Runtime discovers their schemas with +a service token, then exchanges the current `user_id` and `run_id` for a short +delegated token when executing an MCP tool. The model never receives a token. + +## Isolated Docker Start + +From the repository root in PowerShell: + +```powershell +$env:DEER_FLOW_ROOT = (Get-Location).Path +$env:HOME = $env:USERPROFILE +$env:DEER_FLOW_CONTAINER_PREFIX = "flowlens-agentops" +$env:DEER_FLOW_HTTP_PORT = "2027" + +docker compose -p flowlens-agentops ` + -f docker/docker-compose-dev.yaml ` + -f docker/docker-compose-agentops-isolated.yaml ` + up -d --build --remove-orphans frontend gateway nginx sqlite-analytics flowlens-inspector +``` + +The bootstrap container generates a local-only RSA keypair in a Docker named +volume. The isolated DeerFlow runtime state is stored under the ignored +`backend/.flowlens-agentops/` directory, so an older local schema cannot affect +this stack. `2027` avoids interrupting an existing DeerFlow environment on +`2026`. + +Check the stack: + +```powershell +docker compose -p flowlens-agentops ` + -f docker/docker-compose-dev.yaml ` + -f docker/docker-compose-agentops-isolated.yaml ps +Invoke-RestMethod http://localhost:2027/health +``` + +Open `http://localhost:2027/`, sign in or create a local account, then open +`http://localhost:2027/agentops` in another tab. + +## Demo 1: Real SQLite MCP Call + +In DeerFlow, send this prompt: + +```text +Use the SQLite Analytics MCP to inspect the available schema, then count runs by status. +State the SQL you used and label the result as local demonstration data. +``` + +Expected behavior: + +1. The Lead Agent selects `sqlite_analytics_schema_overview`. +2. It may call `sqlite_analytics_describe_table` for `agent_runs`. +3. It calls `sqlite_analytics_query_readonly` with a single `SELECT`. +4. The MCP result returns as a `ToolMessage`; the Lead Agent summarizes it. +5. AgentOps shows the new `run_id` with `MCP` Timeline items and `mcp.tool.start` / `mcp.tool.end` events. + +For a negative case, ask for `DELETE FROM agent_runs`. The server rejects it +before database execution. This verifies both the SQL policy and the failure +evidence path; it is not a production security benchmark. + +## Demo 2: Inspect a Prior Run Through MCP + +After Demo 1 finishes, use its `run_id` and `thread_id` from AgentOps. Start a +new DeerFlow chat and send: + +```text +Use FlowLens Inspector MCP to list my recent runs. Inspect the diagnostics and a Timeline window for run_id in thread_id . Summarize the status, token count, tool/MCP evidence, and deterministic root cause. Do not invent evidence not returned by the tools. +``` + +Expected behavior: + +1. The Runtime discovers the Inspector schema with a discovery-only token. +2. For the selected MCP tool call, it obtains a short delegated token bound to the current user and new run. +3. Inspector validates that token and forwards it to Gateway. +4. Gateway independently checks the authenticated user, tenant, and target run ownership. +5. The returned diagnostics become the new Agent run's ToolMessage, so the second run contains its own MCP evidence. + +## AgentOps Walkthrough + +For each completed run: + +1. Select the run in **Run Explorer**. +2. Confirm the summary bar: duration, tokens, Tool/Error count, MCP count, and terminal status. +3. Select the **MCP** Timeline filter. Open an item to inspect safe metadata such as server name, tool name, transport, span, parent span, and duration. +4. Use **Root Cause**. It is deterministic: the optional explanation can describe the result but cannot change it. +5. Use **Export** to download the redacted JSON snapshot. + +## Stop the Isolated Stack + +```powershell +docker compose -p flowlens-agentops ` + -f docker/docker-compose-dev.yaml ` + -f docker/docker-compose-agentops-isolated.yaml down +``` + +The MCP key volume and the ignored `backend/.flowlens-agentops/` runtime state +retain local demonstration data between restarts. Add `-v` to remove the Docker +volumes; remove `backend/.flowlens-agentops/` separately when a fully fresh +local identity is required. diff --git a/docs/superpowers/plans/2026-07-28-flowlens-inspector-mcp.md b/docs/superpowers/plans/2026-07-28-flowlens-inspector-mcp.md new file mode 100644 index 00000000..49c5b110 --- /dev/null +++ b/docs/superpowers/plans/2026-07-28-flowlens-inspector-mcp.md @@ -0,0 +1,1314 @@ +# FlowLens Inspector MCP Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build an authenticated, tenant-aware Streamable HTTP MCP service that lets DeerFlow agents inspect FlowLens runs through five read-only tools and a real incident-response Skill. + +**Architecture:** Keep Gateway as the owner of AgentOps query authorization and expose FlowLens Inspector as a separate FastMCP resource server. DeerFlow discovers tools with a service credential, exchanges runtime identity for a short-lived delegated JWT at tool-call time, and forwards that JWT through MCP to Gateway; client-side RunJournal events and server-side audit records make the entire call observable. + +**Tech Stack:** Python 3.12, FastMCP/MCP Python SDK 1.27+, Streamable HTTP, FastAPI, PyJWT with RSA/JWKS, Pydantic 2, HTTPX, Tenacity, SQLAlchemy/Alembic, Redis, Prometheus client, Pytest, Ruff, Docker Compose + +## Global Constraints + +- Public product naming remains `FlowLens AgentOps`; the MCP service is `FlowLens Inspector MCP`. +- The five MCP tools are read-only and must never mutate Run, Timeline, Checkpoint, Memory, Prompt, or historical events. +- Tool results never include complete prompts, memory text, attachment bodies, credentials, cookies, authorization headers, or database URLs. +- Core diagnostics and run comparison are deterministic; no MCP tool invokes an LLM. +- Streamable HTTP is the primary transport; `stdio` is not the acceptance path. +- Access tokens use `RS256`, `kid`, JWKS, `iss`, `aud`, `sub`, `tenant_id`, `scope`, `iat`, `exp`, `jti`, and `ver`. +- Delegated access tokens default to 300 seconds and are refreshed before expiry. +- MCP and Gateway both verify the token; Gateway always performs final Run owner and tenant checks. +- Timeline pages default to 20 and cap at 100 MCP items even though the model plan is not cost-constrained. +- HTTP connection timeout is 2 seconds and total MCP tool timeout is capped at 12 seconds. +- Only idempotent dependency failures are retried, at most twice, with backoff and jitter. +- Local mode uses SQLite and an in-memory limiter; enterprise verification uses PostgreSQL and Redis. +- Every behavior change follows RED, GREEN, focused regression, Ruff, then an independently reviewable commit. +- Performance and test-count claims are published only from fresh command output. +- Execution uses Inline Execution with a user-facing checkpoint after each of the four stages; do not dispatch one subagent per task. + +--- + +## File Structure + +### New MCP package + +- Create: `backend/packages/flowlens-inspector-mcp/pyproject.toml` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/__init__.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/__main__.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/settings.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/schemas.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/server.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/gateway_client.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/comparison.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/resilience.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/audit.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/telemetry.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/auth/__init__.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/auth/principal.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/auth/verifier.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/auth/scopes.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/tools/__init__.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/tools/health.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/tools/runs.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/tools/diagnostics.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/tools/timeline.py` +- Create: `backend/packages/flowlens-inspector-mcp/flowlens_inspector_mcp/tools/comparison.py` + +### Gateway, persistence, and runtime integration + +- Modify: `backend/pyproject.toml` +- Modify: `backend/app/gateway/app.py` +- Modify: `backend/app/gateway/auth/models.py` +- Modify: `backend/app/gateway/auth_middleware.py` +- Modify: `backend/app/gateway/authz.py` +- Create: `backend/app/gateway/auth/platform_tokens.py` +- Create: `backend/app/gateway/routers/mcp_tokens.py` +- Modify: `backend/app/gateway/routers/agentops.py` +- Modify: `backend/app/gateway/routers/thread_runs.py` +- Modify: `backend/app/gateway/services.py` +- Modify: `backend/packages/harness/deerflow/config/extensions_config.py` +- Modify: `backend/packages/harness/deerflow/config/app_config.py` +- Create: `backend/packages/harness/deerflow/mcp/delegation.py` +- Modify: `backend/packages/harness/deerflow/mcp/tools.py` +- Modify: `backend/packages/harness/deerflow/mcp/oauth.py` +- Modify: `backend/packages/harness/deerflow/runtime/user_context.py` +- Modify: `backend/packages/harness/deerflow/runtime/runs/manager.py` +- Modify: `backend/packages/harness/deerflow/runtime/runs/store/base.py` +- Modify: `backend/packages/harness/deerflow/runtime/runs/store/memory.py` +- Modify: `backend/packages/harness/deerflow/persistence/run/model.py` +- Modify: `backend/packages/harness/deerflow/persistence/run/sql.py` +- Modify: `backend/packages/harness/deerflow/persistence/user/model.py` +- Modify: `backend/app/gateway/auth/repositories/sqlite.py` +- Create: `backend/packages/harness/deerflow/persistence/mcp_audit/__init__.py` +- Create: `backend/packages/harness/deerflow/persistence/mcp_audit/model.py` +- Create: `backend/packages/harness/deerflow/persistence/mcp_audit/sql.py` +- Modify: `backend/packages/harness/deerflow/persistence/models/__init__.py` +- Create: `backend/packages/harness/deerflow/persistence/migrations/versions/20260728_0001_flowlens_mcp_tenant.py` +- Create: `backend/packages/harness/deerflow/persistence/migrations/versions/20260728_0002_flowlens_mcp_audit.py` + +### Skill, deployment, tests, and documentation + +- Create: `skills/public/flowlens-incident-response/SKILL.md` +- Create: `skills/public/flowlens-incident-response/references/diagnostic-playbook.md` +- Create: `skills/public/flowlens-incident-response/references/output-format.md` +- Create: `skills/public/flowlens-incident-response/evals/trigger_eval_set.json` +- Create: `skills/public/flowlens-incident-response/evals/evals.json` +- Create: `backend/Dockerfile.mcp` +- Modify: `docker/docker-compose-dev.yaml` +- Modify: `docker/docker-compose.yaml` +- Create: `docker/docker-compose-flowlens-mcp-enterprise.yaml` +- Modify: `docker/nginx/nginx.conf` +- Modify: `.env.example` +- Modify: `extensions_config.example.json` +- Modify: `extensions_config.json` +- Create: `backend/scripts/generate_mcp_keys.py` +- Create: `backend/scripts/migrate_flowlens_mcp.py` +- Create: `backend/scripts/seed_agentops_runs.py` +- Create: `backend/scripts/benchmark_inspector_mcp.py` +- Create: `scripts/verify_flowlens_mcp.ps1` +- Modify: `README.md` +- Modify: `docs/architecture.md` +- Create: `docs/flowlens-inspector-mcp.md` +- Create: `backend/tests/flowlens_inspector_mcp/` test package and focused test modules named in the tasks below. + +Command convention: run `uv run ...` and Ruff commands from `backend/`; run `git ...`, Docker Compose, and PowerShell script commands from the repository root. + +--- + +## Stage 1: Identity, Persistence, And Gateway Contracts + +### Task 1: Persist Tenant-Aware User And Run Identity + +**Files:** +- Modify: `backend/app/gateway/auth/models.py:User` +- Modify: `backend/packages/harness/deerflow/persistence/user/model.py:UserRow` +- Modify: `backend/app/gateway/auth/repositories/sqlite.py` +- Modify: `backend/packages/harness/deerflow/runtime/user_context.py` +- Modify: `backend/packages/harness/deerflow/runtime/runs/manager.py:RunRecord` +- Modify: `backend/packages/harness/deerflow/runtime/runs/store/base.py` +- Modify: `backend/packages/harness/deerflow/runtime/runs/store/memory.py` +- Modify: `backend/packages/harness/deerflow/persistence/run/model.py:RunRow` +- Modify: `backend/packages/harness/deerflow/persistence/run/sql.py:RunRepository` +- Modify: `backend/app/gateway/services.py:inject_authenticated_user_context` +- Create: `backend/packages/harness/deerflow/persistence/migrations/versions/20260728_0001_flowlens_mcp_tenant.py` +- Create: `backend/tests/test_tenant_run_isolation.py` +- Modify: `backend/tests/test_run_repository.py` +- Modify: `backend/tests/test_auth.py` + +**Interfaces:** +- Produces: `DEFAULT_TENANT_ID = "default"`. +- Produces: `get_effective_tenant_id() -> str`. +- Produces: `resolve_runtime_tenant_id(runtime: object | None) -> str`. +- Extends: `User.tenant_id: str`, `RunRecord.user_id: str`, and `RunRecord.tenant_id: str`. +- Extends: RunStore read methods with keyword-only `tenant_id`. +- Invariant: authorized Run lookup always matches `(tenant_id, user_id, run_id)`. + +- [ ] **Step 1: Write failing tenant isolation tests** + +```python +@pytest.mark.anyio +async def test_run_repository_requires_matching_tenant_and_user(session_factory): + repo = RunRepository(session_factory) + await repo.put("run-a", thread_id="thread-a", user_id="user-a", tenant_id="tenant-a") + + assert await repo.get("run-a", user_id="user-a", tenant_id="tenant-a") is not None + assert await repo.get("run-a", user_id="user-a", tenant_id="tenant-b") is None + assert await repo.get("run-a", user_id="user-b", tenant_id="tenant-a") is None + + +def test_authenticated_context_includes_server_owned_tenant(): + config: dict = {} + request = SimpleNamespace( + state=SimpleNamespace(user=SimpleNamespace(id="user-a", tenant_id="tenant-a", system_role="user")) + ) + inject_authenticated_user_context(config, request) + assert config["context"]["user_id"] == "user-a" + assert config["context"]["tenant_id"] == "tenant-a" +``` + +- [ ] **Step 2: Run tests to verify RED** + +Run: + +```powershell +cd backend +$env:PYTHONPATH='.' +uv run pytest tests/test_tenant_run_isolation.py tests/test_run_repository.py tests/test_auth.py -q +``` + +Expected: fail because `tenant_id` fields and repository parameters do not exist. + +- [ ] **Step 3: Add tenant identity helpers and persisted fields** + +Implement the shared defaults: + +```python +DEFAULT_TENANT_ID: Final[str] = "default" + + +def get_effective_tenant_id() -> str: + user = get_current_user() + value = getattr(user, "tenant_id", None) if user is not None else None + return str(value or DEFAULT_TENANT_ID) + + +def resolve_runtime_tenant_id(runtime: object | None) -> str: + context = getattr(runtime, "context", None) + if isinstance(context, dict) and context.get("tenant_id"): + return str(context["tenant_id"]) + return get_effective_tenant_id() +``` + +Add `tenant_id` to `User`, `UserRow`, `RunRow`, and `RunRecord`. New users and legacy rows default to `"default"`. + +Capture immutable ownership when RunManager creates a record and include it in every persistence payload: + +```python +record = RunRecord( + ..., + user_id=get_effective_user_id(), + tenant_id=get_effective_tenant_id(), +) +``` + +- [ ] **Step 4: Enforce tenant and owner filters in stores** + +Update `RunRepository.get`, `list_by_thread`, and `list_recent` so a non-admin read includes both: + +```python +stmt = stmt.where( + RunRow.tenant_id == resolved_tenant_id, + RunRow.user_id == resolved_user_id, +) +``` + +Update `MemoryRunStore` and in-memory `RunManager.get` with the same semantics. Explicit `user_id=None, tenant_id=None` remains the migration/admin bypass; no ordinary request may use it. + +- [ ] **Step 5: Add the idempotent schema migration** + +The Alembic upgrade must: + +1. Add `users.tenant_id` and `runs.tenant_id` with server default `"default"` when absent. +2. Backfill null values. +3. Make both columns non-null. +4. Add `ix_runs_tenant_user_created` over `(tenant_id, user_id, created_at)`. +5. Skip an operation when inspection shows the column/index already exists, because local `create_all()` may have created the current schema. + +- [ ] **Step 6: Verify GREEN and isolation regressions** + +Run: + +```powershell +uv run pytest tests/test_tenant_run_isolation.py tests/test_run_repository.py tests/test_auth.py tests/test_agentops_runs_api.py tests/test_migration_user_isolation.py -q +uv run ruff check app/gateway/auth app/gateway/services.py packages/harness/deerflow/runtime/user_context.py packages/harness/deerflow/runtime/runs packages/harness/deerflow/persistence +``` + +Expected: all selected tests pass and Ruff reports no errors. + +- [ ] **Step 7: Commit** + +```powershell +git add backend/app/gateway/auth backend/app/gateway/services.py backend/packages/harness/deerflow/runtime backend/packages/harness/deerflow/persistence backend/tests/test_tenant_run_isolation.py backend/tests/test_run_repository.py backend/tests/test_auth.py +git commit -m "feat(agentops): add tenant-aware run identity" +``` + +### Task 2: Issue And Verify Scoped MCP Delegation Tokens + +**Files:** +- Create: `backend/app/gateway/auth/platform_tokens.py` +- Create: `backend/app/gateway/routers/mcp_tokens.py` +- Modify: `backend/app/gateway/auth_middleware.py` +- Modify: `backend/app/gateway/authz.py` +- Modify: `backend/app/gateway/app.py` +- Create: `backend/tests/test_mcp_platform_tokens.py` +- Create: `backend/tests/test_mcp_bearer_auth.py` + +**Interfaces:** +- Produces: `PlatformTokenSettings.from_env()`. +- Produces: `PlatformTokenClaims`. +- Produces: `PlatformTokenIssuer.issue_service_token(...)`. +- Produces: `PlatformTokenIssuer.issue_delegated_token(...)`. +- Produces: `verify_platform_token(token: str) -> PlatformTokenClaims`. +- Produces: `GET /.well-known/jwks.json`. +- Produces: `POST /api/v1/auth/mcp/token`. + +- [ ] **Step 1: Write failing token and Bearer authentication tests** + +```python +def test_delegated_token_contains_required_claims(rsa_settings): + token = PlatformTokenIssuer(rsa_settings).issue_delegated_token( + user_id="user-a", + tenant_id="tenant-a", + token_version=3, + origin_run_id="run-origin", + ) + claims = verify_platform_token(token, rsa_settings) + assert claims.sub == "user-a" + assert claims.tenant_id == "tenant-a" + assert claims.aud == "flowlens-platform" + assert "agentops.timeline.read" in claims.scope + assert claims.ver == 3 + + +def test_gateway_accepts_platform_bearer_and_rejects_wrong_audience(client, delegated_token): + assert client.get("/api/agentops/runs", headers={"Authorization": f"Bearer {delegated_token}"}).status_code == 200 + assert client.get("/api/agentops/runs", headers={"Authorization": "Bearer wrong-aud-token"}).status_code == 401 +``` + +- [ ] **Step 2: Run tests to verify RED** + +Run: + +```powershell +uv run pytest tests/test_mcp_platform_tokens.py tests/test_mcp_bearer_auth.py -q +``` + +Expected: fail because platform token modules and Bearer authentication are absent. + +- [ ] **Step 3: Implement RS256 claims, issuer, verifier, and JWKS** + +Use immutable claims: + +```python +class PlatformTokenClaims(BaseModel): + iss: str + sub: str + tenant_id: str + aud: str | list[str] + scope: list[str] + origin_run_id: str | None = None + azp: str + jti: str + iat: datetime + exp: datetime + ver: int +``` + +Read keys and identifiers from `FLOWLENS_MCP_JWT_PRIVATE_KEY_PATH`, `FLOWLENS_MCP_JWT_PUBLIC_KEY_PATH`, `FLOWLENS_MCP_JWT_KID`, `FLOWLENS_MCP_ISSUER`, and `FLOWLENS_MCP_AUDIENCE`. Reject missing key material at startup when MCP is enabled. + +- [ ] **Step 4: Implement the service and delegation grants** + +`POST /api/v1/auth/mcp/token` accepts form data: + +- `grant_type=client_credentials`: issue only `agentops.tools.discover`. +- `grant_type=urn:deerflow:params:oauth:grant-type:runtime-delegation`: require `subject_user_id` and `origin_run_id`, load the user from the repository, derive `tenant_id` and `ver` from that row, and issue the five read scopes. + +Authenticate `client_id` and `client_secret` with `hmac.compare_digest`; never echo the secret. + +- [ ] **Step 5: Extend Gateway middleware without weakening cookie auth** + +Authentication order: + +1. Valid internal token. +2. Valid `Authorization: Bearer` platform token. +3. Existing cookie JWT. +4. Otherwise `401`. + +Map delegated AgentOps scopes to Gateway `runs:read`, retain the full scope set on `request.state.platform_scopes`, load the real User row, and reject a stale `ver`. + +- [ ] **Step 6: Verify GREEN and existing auth regressions** + +Run: + +```powershell +uv run pytest tests/test_mcp_platform_tokens.py tests/test_mcp_bearer_auth.py tests/test_auth_middleware.py tests/test_auth.py tests/test_auth_errors.py tests/test_internal_auth.py -q +uv run ruff check app/gateway/auth app/gateway/auth_middleware.py app/gateway/authz.py app/gateway/routers/mcp_tokens.py +``` + +Expected: new Bearer cases and all existing cookie/internal-token cases pass. + +- [ ] **Step 7: Commit** + +```powershell +git add backend/app/gateway/auth backend/app/gateway/auth_middleware.py backend/app/gateway/authz.py backend/app/gateway/routers/mcp_tokens.py backend/app/gateway/app.py backend/tests/test_mcp_platform_tokens.py backend/tests/test_mcp_bearer_auth.py +git commit -m "feat(auth): issue scoped MCP delegation tokens" +``` + +### Task 3: Stabilize Gateway Query Contracts For MCP + +**Files:** +- Modify: `backend/app/gateway/routers/agentops.py` +- Modify: `backend/app/gateway/routers/thread_runs.py` +- Modify: `backend/packages/harness/deerflow/runtime/runs/store/base.py` +- Modify: `backend/packages/harness/deerflow/runtime/runs/store/memory.py` +- Modify: `backend/packages/harness/deerflow/persistence/run/sql.py` +- Modify: `backend/packages/harness/deerflow/runtime/runs/manager.py` +- Modify: `backend/tests/test_agentops_runs_api.py` +- Modify: `backend/tests/test_thread_run_diagnostics_api.py` +- Create: `backend/tests/test_agentops_run_lookup_api.py` + +**Interfaces:** +- Extends: `GET /api/agentops/runs` with optional `thread_id`. +- Produces: `GET /api/agentops/runs/{run_id}`. +- Preserves: existing thread-scoped Timeline and Diagnostics endpoints. +- Invariant: inaccessible and nonexistent runs both return `404`. + +- [ ] **Step 1: Add failing API contract tests** + +```python +def test_list_runs_filters_by_thread_and_tenant(client, token): + response = client.get( + "/api/agentops/runs?thread_id=thread-a&limit=20", + headers={"Authorization": f"Bearer {token}"}, + ) + assert response.status_code == 200 + assert {item["thread_id"] for item in response.json()["items"]} == {"thread-a"} + + +def test_run_lookup_hides_cross_tenant_run(client, tenant_a_token): + response = client.get( + "/api/agentops/runs/run-owned-by-tenant-b", + headers={"Authorization": f"Bearer {tenant_a_token}"}, + ) + assert response.status_code == 404 +``` + +- [ ] **Step 2: Run tests to verify RED** + +Run: + +```powershell +uv run pytest tests/test_agentops_runs_api.py tests/test_agentops_run_lookup_api.py tests/test_thread_run_diagnostics_api.py -q +``` + +Expected: fail because `thread_id` filtering, run lookup, and tenant-aware route calls are missing. + +- [ ] **Step 3: Add thread filtering to RunStore and run index** + +Extend `list_recent` with `thread_id: str | None`. Apply the condition before cursor ordering: + +```python +if thread_id is not None: + stmt = stmt.where(RunRow.thread_id == thread_id) +``` + +Keep the existing `(created_at DESC, run_id DESC)` cursor contract. + +- [ ] **Step 4: Add the owner-safe Run lookup endpoint** + +Return only the fields needed by MCP: + +```json +{ + "run_id": "run-a", + "thread_id": "thread-a", + "status": "error", + "created_at": "...", + "updated_at": "...", + "model_name": "...", + "total_tokens": 120, + "llm_call_count": 2, + "message_count": 4 +} +``` + +Resolve the current `User.tenant_id`, call `RunManager.get(run_id, user_id=..., tenant_id=...)`, and return the same `404` for absent or unauthorized data. + +- [ ] **Step 5: Verify GREEN and FlowLens regressions** + +Run: + +```powershell +uv run pytest tests/test_agentops_runs_api.py tests/test_agentops_run_lookup_api.py tests/test_thread_run_diagnostics_api.py tests/test_run_repository.py tests/test_tenant_run_isolation.py -q +uv run ruff check app/gateway/routers/agentops.py app/gateway/routers/thread_runs.py packages/harness/deerflow/runtime/runs packages/harness/deerflow/persistence/run +``` + +Expected: all selected tests pass. + +- [ ] **Step 6: Commit and report Stage 1** + +```powershell +git add backend/app/gateway/routers/agentops.py backend/app/gateway/routers/thread_runs.py backend/packages/harness/deerflow/runtime/runs backend/packages/harness/deerflow/persistence/run backend/tests/test_agentops_runs_api.py backend/tests/test_agentops_run_lookup_api.py backend/tests/test_thread_run_diagnostics_api.py +git commit -m "feat(agentops): expose MCP-safe run query contracts" +``` + +Stage 1 report must show the token claims test, cross-tenant `404`, and exact Gateway API response before Stage 2 begins. + +--- + +## Stage 2: MCP Server And DeerFlow Client Integration + +### Task 4: Create The Authenticated Streamable HTTP MCP Foundation + +**Files:** +- Modify: `backend/pyproject.toml` +- Create: `backend/packages/flowlens-inspector-mcp/pyproject.toml` +- Create: package files under `flowlens_inspector_mcp/` +- Create: `backend/tests/flowlens_inspector_mcp/test_settings.py` +- Create: `backend/tests/flowlens_inspector_mcp/test_auth.py` +- Create: `backend/tests/flowlens_inspector_mcp/test_server_contract.py` + +**Interfaces:** +- Produces: `InspectorSettings.from_env()`. +- Produces: `JwtTokenVerifier.verify_token(token: str) -> AccessToken | None`. +- Produces: `require_scope(scope: str) -> AccessPrincipal`. +- Produces: `create_mcp_server(settings, gateway_client, executor) -> FastMCP`. +- Produces: `create_app(settings: InspectorSettings | None = None) -> Starlette`. +- Produces: `/mcp`, `/health/live`, `/health/ready`, `/metrics`, and protected-resource metadata. + +- [ ] **Step 1: Write failing package and authentication tests** + +```python +@pytest.mark.anyio +async def test_token_verifier_returns_sdk_access_token(valid_jwt, verifier): + token = await verifier.verify_token(valid_jwt) + assert token is not None + assert token.client_id == "user-a" + assert "agentops.runs.read" in token.scopes + + +@pytest.mark.anyio +async def test_list_tools_exposes_exact_public_contract(mcp_client): + result = await mcp_client.list_tools() + assert {tool.name for tool in result.tools} == { + "health_check", + "list_runs", + "get_run_diagnostics", + "get_timeline_window", + "compare_runs", + } +``` + +- [ ] **Step 2: Add transport-security rejection cases** + +```python +@pytest.mark.anyio +async def test_streamable_http_rejects_unknown_origin(asgi_client): + response = await asgi_client.post( + "/mcp", + headers={"Origin": "https://attacker.example", "Host": "localhost:8010"}, + json={"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {}}, + ) + assert response.status_code in {400, 403} + + +@pytest.mark.anyio +async def test_streamable_http_rejects_unknown_host(asgi_client): + response = await asgi_client.post( + "/mcp", + headers={"Origin": "http://localhost:2026", "Host": "attacker.example"}, + json={"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {}}, + ) + assert response.status_code in {400, 403} +``` + +- [ ] **Step 3: Run tests to verify RED** + +Run: + +```powershell +uv run pytest tests/flowlens_inspector_mcp/test_settings.py tests/flowlens_inspector_mcp/test_auth.py tests/flowlens_inspector_mcp/test_server_contract.py -q +``` + +Expected: fail because the workspace package is absent. + +- [ ] **Step 4: Register the workspace package and direct dependencies** + +Add `packages/flowlens-inspector-mcp` to the uv workspace and add a workspace source. The package declares direct dependencies on: + +```toml +dependencies = [ + "deerflow-harness", + "mcp>=1.27,<2", + "httpx>=0.28", + "pydantic>=2.12", + "pyjwt[crypto]>=2.9", + "tenacity>=9", + "prometheus-client>=0.21", + "redis>=5", + "sqlalchemy[asyncio]>=2,<3", + "uvicorn[standard]>=0.34", +] +``` + +- [ ] **Step 5: Implement settings, verifier, principal, and scopes** + +The verifier validates RSA signature, `kid`, issuer, audience, expiry, and required claim types, then returns: + +```python +AccessToken( + token=raw_token, + client_id=claims.sub, + scopes=claims.scope, + expires_at=int(claims.exp.timestamp()), + resource=settings.resource_url, +) +``` + +`require_scope` reads `get_access_token()`, raises a typed authorization error when missing, and never accepts a user ID from tool arguments. + +- [ ] **Step 6: Build the ASGI service and health surfaces** + +Configure FastMCP with `stateless_http=False`, `streamable_http_path="/mcp"`, `TokenVerifier`, `AuthSettings`, and a strict `TransportSecuritySettings` Origin/Host allowlist. Bind each issued MCP Session ID to the verified `AccessToken.client_id` and reject a session reused under another principal. Mount the FastMCP Starlette application with live, ready, metrics, and protected-resource routes. + +- [ ] **Step 7: Verify GREEN and contract stability** + +Run: + +```powershell +uv run pytest tests/flowlens_inspector_mcp/test_settings.py tests/flowlens_inspector_mcp/test_auth.py tests/flowlens_inspector_mcp/test_server_contract.py -q +uv run ruff check packages/flowlens-inspector-mcp tests/flowlens_inspector_mcp +``` + +Expected: exact five-tool contract, valid authentication, and health routes pass. + +- [ ] **Step 8: Commit** + +```powershell +git add backend/pyproject.toml backend/packages/flowlens-inspector-mcp backend/tests/flowlens_inspector_mcp +git commit -m "feat(mcp): add authenticated FlowLens inspector service" +``` + +### Task 5: Implement The Five Read-Only Inspector Tools + +**Files:** +- Create/Modify: `flowlens_inspector_mcp/schemas.py` +- Create/Modify: `flowlens_inspector_mcp/gateway_client.py` +- Create/Modify: `flowlens_inspector_mcp/comparison.py` +- Create/Modify: all modules under `flowlens_inspector_mcp/tools/` +- Create: `backend/tests/flowlens_inspector_mcp/test_gateway_client.py` +- Create: `backend/tests/flowlens_inspector_mcp/test_tools.py` +- Create: `backend/tests/flowlens_inspector_mcp/test_comparison.py` + +**Interfaces:** +- Produces: versioned Pydantic inputs and outputs with `schema_version`, `request_id`, and `generated_at`. +- Produces: `GatewayClient` read methods that forward the original delegated Bearer token. +- Produces: `build_run_comparison(base, candidate) -> RunComparisonResponse`. +- Invariant: tool argument models contain no `user_id`, `tenant_id`, credential, or unrestricted page size. + +The concrete Gateway client surface is: + +```python +class GatewayClient: + async def health(self, *, token: str) -> GatewayHealth: ... + async def list_runs(self, *, token: str, query: RunListInput) -> RunListResponse: ... + async def get_run(self, *, token: str, run_id: str) -> RunSummary: ... + async def get_diagnostics(self, *, token: str, run: RunSummary) -> DiagnosticsResponse: ... + async def get_timeline(self, *, token: str, run: RunSummary, query: TimelineWindowInput) -> TimelineWindowResponse: ... +``` + +- [ ] **Step 1: Write failing tool behavior tests** + +```python +@pytest.mark.anyio +async def test_timeline_caps_limit_and_forwards_scope(tool_harness): + result = await tool_harness.call( + "get_timeline_window", + {"run_id": "run-a", "limit": 500}, + scopes=["agentops.timeline.read"], + ) + assert result.isError is True + + +@pytest.mark.anyio +async def test_compare_runs_is_deterministic(tool_harness): + first = await tool_harness.call("compare_runs", {"base_run_id": "run-a", "candidate_run_id": "run-b"}) + second = await tool_harness.call("compare_runs", {"base_run_id": "run-a", "candidate_run_id": "run-b"}) + assert first.structuredContent == second.structuredContent +``` + +- [ ] **Step 2: Run tests to verify RED** + +Run: + +```powershell +uv run pytest tests/flowlens_inspector_mcp/test_gateway_client.py tests/flowlens_inspector_mcp/test_tools.py tests/flowlens_inspector_mcp/test_comparison.py -q +``` + +Expected: fail because Gateway client methods and tool implementations are absent. + +- [ ] **Step 3: Implement bounded schemas and Gateway calls** + +Use opaque cursor strings and constrained limits: + +```python +class TimelineWindowInput(BaseModel): + run_id: str = Field(min_length=1, max_length=64) + phase: list[TimelinePhase] | None = None + after_seq: int | None = Field(default=None, ge=0) + limit: int = Field(default=20, ge=1, le=100) +``` + +Every run-specific call first resolves `GET /api/agentops/runs/{run_id}` to obtain the authorized `thread_id`, then calls the existing thread-scoped Timeline or Diagnostics endpoint with the same Bearer token. + +- [ ] **Step 4: Implement tool-specific scopes** + +Register: + +```python +TOOL_SCOPES = { + "health_check": "agentops.health.read", + "list_runs": "agentops.runs.read", + "get_run_diagnostics": "agentops.diagnostics.read", + "get_timeline_window": "agentops.timeline.read", + "compare_runs": "agentops.runs.compare", +} +``` + +Do not fetch Gateway data until the scope check passes. + +- [ ] **Step 5: Implement deterministic comparison and partial responses** + +Compare duration, tokens, LLM calls, tool calls/errors, subagents, MCP, Memory, rollbacks, status, and attribution. If one optional diagnostics section is unavailable, return `status="partial"`, explicit available/missing sections, and `retryable`; never fill missing values with invented evidence. + +- [ ] **Step 6: Verify GREEN, redaction, and contract tests** + +Run: + +```powershell +uv run pytest tests/flowlens_inspector_mcp/test_gateway_client.py tests/flowlens_inspector_mcp/test_tools.py tests/flowlens_inspector_mcp/test_comparison.py tests/test_flowlens_redaction.py -q +uv run ruff check packages/flowlens-inspector-mcp tests/flowlens_inspector_mcp +``` + +Expected: all five tools pass; no response fixture contains prompt, memory, Authorization, cookie, or secret fields. + +- [ ] **Step 7: Commit** + +```powershell +git add backend/packages/flowlens-inspector-mcp backend/tests/flowlens_inspector_mcp +git commit -m "feat(mcp): add FlowLens diagnostic tools" +``` + +### Task 6: Inject Runtime-Delegated Tokens Into MCP Tool Calls + +**Files:** +- Modify: `backend/packages/harness/deerflow/config/extensions_config.py` +- Create: `backend/packages/harness/deerflow/mcp/delegation.py` +- Modify: `backend/packages/harness/deerflow/mcp/tools.py` +- Modify: `backend/packages/harness/deerflow/mcp/oauth.py` +- Modify: `backend/app/gateway/services.py` +- Create: `backend/tests/test_mcp_delegation.py` +- Modify: `backend/tests/test_mcp_custom_interceptors.py` +- Modify: `backend/tests/test_mcp_diagnostics.py` + +**Interfaces:** +- Produces: `McpDelegatedAuthConfig`. +- Produces: `DelegatedTokenManager.get_authorization_header(server_name, runtime) -> str`. +- Produces: `build_delegated_auth_interceptor(config)`. +- Preserves: service OAuth header for initial `initialize`/`tools/list`. +- Overrides: service header with delegated user header only for `tools/call`. +- Produces: runtime-visible tool names prefixed as `flowlens_inspector_` by the existing multi-server client. + +- [ ] **Step 1: Write failing runtime delegation tests** + +```python +@pytest.mark.anyio +async def test_delegation_uses_server_injected_runtime_identity(manager, runtime): + runtime.context.update({ + "user_id": "user-a", + "tenant_id": "tenant-a", + "run_id": "run-origin", + }) + header = await manager.get_authorization_header("flowlens-inspector", runtime) + assert header.startswith("Bearer ") + assert manager.last_request["subject_user_id"] == "user-a" + assert manager.last_request["origin_run_id"] == "run-origin" + + +@pytest.mark.anyio +async def test_delegation_never_records_token_in_mcp_events(recorder, interceptor): + await invoke_interceptor(interceptor, recorder) + assert "Bearer " not in json.dumps(recorder.events) +``` + +- [ ] **Step 2: Run tests to verify RED** + +Run: + +```powershell +uv run pytest tests/test_mcp_delegation.py tests/test_mcp_custom_interceptors.py tests/test_mcp_diagnostics.py -q +``` + +Expected: fail because delegated auth config and interceptor are absent. + +- [ ] **Step 3: Add delegated auth configuration** + +Add an optional per-server object: + +```python +class McpDelegatedAuthConfig(BaseModel): + enabled: bool = True + token_url: str + client_id: str + client_secret: str + audience: str = "flowlens-platform" + refresh_skew_seconds: int = 60 +``` + +Environment placeholders continue to resolve through `ExtensionsConfig.resolve_env_variables`; UI masking treats `client_secret` as sensitive. + +- [ ] **Step 4: Implement per-user/run token acquisition and caching** + +Cache by: + +```text +(server_name, user_id, tenant_id, origin_run_id) +``` + +Use an `asyncio.Lock` per cache key. Post the runtime-delegation grant to Gateway and refresh before expiry. Require `user_id`, `tenant_id`, and `run_id` from server-owned Runtime Context; do not fall back to model arguments. + +- [ ] **Step 5: Compose interceptor order and diagnostics** + +The service OAuth interceptor supplies startup discovery credentials. The delegated interceptor runs inside it and replaces `Authorization` for tool calls. Emit only: + +```text +mcp.delegation.refresh +mcp.delegation.error +``` + +with server name, grant type, duration, status, and exception class. Existing RunJournal tool callbacks continue producing `mcp.tool.start/end/error`. + +- [ ] **Step 6: Verify GREEN and a real adapter call** + +Run: + +```powershell +uv run pytest tests/test_mcp_delegation.py tests/test_mcp_custom_interceptors.py tests/test_mcp_oauth.py tests/test_mcp_diagnostics.py tests/test_mcp_client_config.py -q +uv run ruff check packages/harness/deerflow/mcp packages/harness/deerflow/config/extensions_config.py app/gateway/services.py +``` + +Expected: service discovery auth and runtime delegation both pass without token leakage. + +- [ ] **Step 7: Commit and report Stage 2** + +```powershell +git add backend/packages/harness/deerflow/config/extensions_config.py backend/packages/harness/deerflow/mcp backend/app/gateway/services.py backend/tests/test_mcp_delegation.py backend/tests/test_mcp_custom_interceptors.py backend/tests/test_mcp_diagnostics.py +git commit -m "feat(mcp): delegate runtime identity to inspector tools" +``` + +Stage 2 report must show `tools/list`, one authenticated `list_runs`, one denied scope, and the corresponding `mcp.tool.*` Timeline evidence. + +--- + +## Stage 3: Reliability, Audit, And Deployment + +### Task 7: Add Bounded Resilience, Audit, And Metrics + +**Files:** +- Create/Modify: `flowlens_inspector_mcp/resilience.py` +- Create/Modify: `flowlens_inspector_mcp/audit.py` +- Create/Modify: `flowlens_inspector_mcp/telemetry.py` +- Modify: `flowlens_inspector_mcp/gateway_client.py` +- Modify: `flowlens_inspector_mcp/server.py` +- Create: `backend/packages/harness/deerflow/persistence/mcp_audit/` +- Modify: `backend/packages/harness/deerflow/persistence/models/__init__.py` +- Create: `backend/packages/harness/deerflow/persistence/migrations/versions/20260728_0002_flowlens_mcp_audit.py` +- Create: `backend/tests/flowlens_inspector_mcp/test_resilience.py` +- Create: `backend/tests/flowlens_inspector_mcp/test_audit.py` +- Create: `backend/tests/flowlens_inspector_mcp/test_metrics.py` + +**Interfaces:** +- Produces: `RetryPolicy`, `CircuitBreaker`, `ConcurrencyLimiter`, and `RateLimiter`. +- Produces: `InMemoryTokenBucket` and `RedisTokenBucket`. +- Produces: `ToolExecutor.execute(...)`. +- Produces: append-only `McpAuditRepository.insert(event)`. +- Produces: Prometheus counters, histograms, gauges, and circuit-state metric. + +Use these boundaries: + +```python +class RateLimiter(Protocol): + async def acquire(self, key: str, *, capacity: int, period_seconds: int) -> bool: ... + + +class ToolExecutor: + async def execute( + self, + *, + tool_name: str, + principal: AccessPrincipal, + target_run_ids: tuple[str, ...], + operation: Callable[[], Awaitable[T]], + ) -> T: ... +``` + +`ToolExecutor` owns concurrency, rate limiting, timing, metric updates, and the final audit write. `GatewayClient` owns HTTP timeout, retry, status mapping, and circuit state. + +- [ ] **Step 1: Write failing failure-policy tests** + +```python +@pytest.mark.anyio +async def test_gateway_retries_503_twice_then_opens_circuit(client): + client.transport.queue_statuses(503, 503, 503) + with pytest.raises(DependencyUnavailable): + await client.list_runs(token="token", limit=20) + assert client.transport.call_count == 3 + assert client.circuit.state == "open" + + +@pytest.mark.anyio +async def test_audit_whitelist_drops_sensitive_payload(audit_repo): + with pytest.raises(ValidationError): + McpAuditEvent( + request_id="req-1", + tool_name="get_run_diagnostics", + user_id="user-a", + tenant_id="tenant-a", + status="success", + authorization="Bearer secret", + ) + + await audit_repo.insert(McpAuditEvent( + request_id="req-1", + tool_name="get_run_diagnostics", + user_id="user-a", + tenant_id="tenant-a", + status="success", + )) + stored = await audit_repo.get("req-1") + assert "authorization" not in stored +``` + +- [ ] **Step 2: Run tests to verify RED** + +Run: + +```powershell +uv run pytest tests/flowlens_inspector_mcp/test_resilience.py tests/flowlens_inspector_mcp/test_audit.py tests/flowlens_inspector_mcp/test_metrics.py -q +``` + +Expected: fail because resilience, audit, and metrics implementations are absent. + +- [ ] **Step 3: Implement retry, cancellation, and circuit behavior** + +Retry only connection errors and `502/503/504`; never retry `400/401/403/404/422/429`. Respect cancellation. Open after five consecutive dependency failures, wait 15 seconds, allow one half-open probe, and close only on success. + +- [ ] **Step 4: Implement limiters** + +Use the key: + +```text +tenant_id:user_id:tool_name +``` + +Defaults: + +- general tools: 60/minute +- Timeline: 30/minute +- comparison: 10/minute +- process concurrency: 32 + +Redis uses one atomic Lua token-bucket operation. Loss of Redis produces an explicit configured policy: fail closed in enterprise mode and use the in-memory limiter only in local mode. + +- [ ] **Step 5: Implement append-only audit** + +Persist only the approved columns from the design. `McpAuditRepository` exposes `insert` and test-only lookup; no production update/delete API exists. The MCP database role documentation grants only `INSERT` and controlled `SELECT`. + +Create migration `20260728_0002_flowlens_mcp_audit.py` with `down_revision` pointing to the tenant migration. Never modify the already-applied `0001` migration. + +The MCP service lifespan initializes a dedicated SQLAlchemy engine from the configured application database URL, constructs `McpAuditRepository`, and disposes the engine during graceful shutdown. It does not use this connection to read Run or Timeline data. + +- [ ] **Step 6: Instrument metrics and trace correlation** + +Record tool requests, results, duration, retries, rate limits, active calls, Gateway errors, and circuit state. Propagate or create `request_id`, `traceparent`, and `origin_run_id`; never use a token as a metric label. + +- [ ] **Step 7: Verify GREEN and full package regressions** + +Run: + +```powershell +uv run pytest tests/flowlens_inspector_mcp -q +uv run ruff check packages/flowlens-inspector-mcp packages/harness/deerflow/persistence/mcp_audit +``` + +Expected: resilience state transitions, append-only audit, and metric updates pass. + +- [ ] **Step 8: Commit** + +```powershell +git add backend/packages/flowlens-inspector-mcp backend/packages/harness/deerflow/persistence/mcp_audit backend/packages/harness/deerflow/persistence/models backend/packages/harness/deerflow/persistence/migrations backend/tests/flowlens_inspector_mcp +git commit -m "feat(mcp): add inspector resilience and audit" +``` + +### Task 8: Package The Local And Enterprise Docker Paths + +**Files:** +- Create: `backend/Dockerfile.mcp` +- Modify: `docker/docker-compose-dev.yaml` +- Modify: `docker/docker-compose.yaml` +- Create: `docker/docker-compose-flowlens-mcp-enterprise.yaml` +- Modify: `docker/nginx/nginx.conf` +- Modify: `backend/packages/harness/deerflow/config/app_config.py` +- Modify: `.env.example` +- Modify: `extensions_config.example.json` +- Modify: `extensions_config.json` +- Create: `backend/scripts/generate_mcp_keys.py` +- Create: `backend/scripts/migrate_flowlens_mcp.py` +- Create: `backend/tests/test_mcp_key_generation.py` +- Create: `backend/tests/test_database_env_override.py` + +**Interfaces:** +- Produces: `python backend/scripts/generate_mcp_keys.py`. +- Produces: `python backend/scripts/migrate_flowlens_mcp.py upgrade`. +- Produces: local `flowlens-inspector-mcp:8010`. +- Produces: enterprise Compose overlay with PostgreSQL and Redis. +- Produces: internal URL `http://flowlens-inspector-mcp:8010/mcp`. + +- [ ] **Step 1: Write failing configuration and key tests** + +```python +def test_database_environment_override_selects_postgres(monkeypatch, config_file): + monkeypatch.setenv("DEER_FLOW_DATABASE_BACKEND", "postgres") + monkeypatch.setenv("DATABASE_URL", "postgresql://deerflow:test@postgres:5432/deerflow") + config = AppConfig.from_file(config_file) + assert config.database.backend == "postgres" + + +def test_key_generator_writes_private_0600_and_public_key(tmp_path): + generate_key_pair(tmp_path) + assert (tmp_path / "private.pem").exists() + assert (tmp_path / "public.pem").exists() + assert (tmp_path / "private.pem").read_text().startswith("-----BEGIN PRIVATE KEY-----") + if os.name != "nt": + assert stat.S_IMODE((tmp_path / "private.pem").stat().st_mode) == 0o600 +``` + +- [ ] **Step 2: Run tests to verify RED** + +Run: + +```powershell +uv run pytest tests/test_mcp_key_generation.py tests/test_database_env_override.py -q +``` + +Expected: fail because key generation and database environment override are absent. + +- [ ] **Step 3: Build a non-root MCP image** + +`backend/Dockerfile.mcp` uses a builder stage, installs only the MCP workspace package and dependencies, creates a non-root `flowlens` user, exposes 8010, and starts: + +```text +python -m flowlens_inspector_mcp +``` + +- [ ] **Step 4: Add local Compose wiring without a startup cycle** + +Start MCP before Gateway using `/health/live`; do not make MCP readiness depend on Gateway for Compose ordering. Gateway reaches MCP on the internal network. MCP `/health/ready` may report Gateway degradation after both processes start. + +Mount only the private key into Gateway. MCP validates through Gateway JWKS and never receives the private key. + +- [ ] **Step 5: Add the enterprise overlay** + +The overlay adds PostgreSQL, Redis, health checks, persistent volumes, and: + +```text +DEER_FLOW_DATABASE_BACKEND=postgres +DATABASE_URL=postgresql://deerflow:${POSTGRES_PASSWORD}@postgres:5432/deerflow +FLOWLENS_MCP_RATE_LIMIT_BACKEND=redis +FLOWLENS_MCP_REDIS_URL=redis://redis:6379/0 +``` + +Add an AppConfig environment override test so no duplicate full `config.yaml` is needed. + +- [ ] **Step 6: Add Nginx and extension configuration** + +Proxy `/mcp` with buffering disabled, bounded body size, correct forwarded headers, and Streamable HTTP timeouts. Add a disabled `flowlens_inspector` entry to committed config and an enabled example using service OAuth plus `delegatedAuth`; secrets use `$FLOWLENS_MCP_CLIENT_SECRET`. Remove the non-resolving `my_package.mcp.auth:build_auth_interceptor` sample from the active committed config because delegated auth is now a built-in interceptor. + +- [ ] **Step 7: Verify Compose and container health** + +Run: + +```powershell +docker compose -f docker/docker-compose-dev.yaml config +docker compose -f docker/docker-compose.yaml config +docker compose -f docker/docker-compose.yaml -f docker/docker-compose-flowlens-mcp-enterprise.yaml config +uv run pytest tests/test_mcp_key_generation.py tests/test_database_env_override.py -q +``` + +Expected: all Compose models validate and focused tests pass. + +- [ ] **Step 8: Commit and report Stage 3** + +```powershell +git add backend/Dockerfile.mcp docker .env.example extensions_config.example.json extensions_config.json backend/packages/harness/deerflow/config/app_config.py backend/scripts/generate_mcp_keys.py backend/scripts/migrate_flowlens_mcp.py backend/tests/test_mcp_key_generation.py backend/tests/test_database_env_override.py +git commit -m "feat(mcp): package enterprise inspector deployment" +``` + +Stage 3 report must show local live/ready output, one forced Gateway timeout with retry/circuit evidence, one audit row, and validated enterprise Compose. + +--- + +## Stage 4: Skill, End-To-End Evidence, And Teaching Materials + +### Task 9: Add The FlowLens Incident Response Skill + +**Files:** +- Create: `skills/public/flowlens-incident-response/SKILL.md` +- Create: `skills/public/flowlens-incident-response/references/diagnostic-playbook.md` +- Create: `skills/public/flowlens-incident-response/references/output-format.md` +- Create: `skills/public/flowlens-incident-response/evals/trigger_eval_set.json` +- Create: `skills/public/flowlens-incident-response/evals/evals.json` +- Create: `backend/tests/test_flowlens_incident_skill.py` + +**Interfaces:** +- Produces: intent-triggered incident investigation SOP. +- Requires: the exact five MCP tool names. +- Produces: conclusion, evidence, impact, remediation, and uncertainty sections. +- Invariant: every root-cause claim cites a Run ID and Timeline event sequence. + +- [ ] **Step 1: Write failing Skill contract tests** + +```python +def test_skill_references_only_real_inspector_tools(skill_text): + expected = { + "flowlens_inspector_health_check", + "flowlens_inspector_list_runs", + "flowlens_inspector_get_run_diagnostics", + "flowlens_inspector_get_timeline_window", + "flowlens_inspector_compare_runs", + } + assert expected <= set(extract_backticked_tool_names(skill_text)) + + +def test_skill_forbids_invented_evidence(skill_text): + assert "不得根据缺失事件猜测根因" in skill_text + assert "provisional" in skill_text + assert "partial" in skill_text +``` + +- [ ] **Step 2: Run tests to verify RED** + +Run: + +```powershell +uv run pytest tests/test_flowlens_incident_skill.py -q +``` + +Expected: fail because the Skill does not exist. + +- [ ] **Step 3: Write focused Skill frontmatter and workflow** + +The description triggers on requests to diagnose a failed/slow Agent run, compare runs, explain Tool/MCP/Subagent failures, or produce an evidence-backed incident report. It must not trigger for ordinary chat, code editing, or generic MCP education. The Skill names the `flowlens_inspector_*` tools visible to DeerFlow while documenting the corresponding unprefixed MCP server contract. + +- [ ] **Step 4: Write the playbook and report contract** + +The playbook enforces: + +```text +locate run -> deterministic diagnosis -> evidence window -> optional baseline comparison -> report +``` + +The output format requires `run_id`, evidence sequence numbers, completeness state, and no secret/raw-content fields. + +- [ ] **Step 5: Add trigger and behavior evals** + +Include positive, negative, ambiguous, running-run, partial-data, cross-run comparison, and MCP-unavailable cases. Expected behavior names the first tool and required final sections rather than relying on exact prose. + +- [ ] **Step 6: Verify GREEN and commit** + +Run: + +```powershell +uv run pytest tests/test_flowlens_incident_skill.py -q +``` + +Then: + +```powershell +git add skills/public/flowlens-incident-response backend/tests/test_flowlens_incident_skill.py +git commit -m "feat(skills): add FlowLens incident response playbook" +``` + +### Task 10: Prove The Complete Flow And Publish Only Measured Evidence + +**Files:** +- Create: `backend/scripts/seed_agentops_runs.py` +- Create: `backend/scripts/benchmark_inspector_mcp.py` +- Create: `scripts/verify_flowlens_mcp.ps1` +- Create: `backend/tests/test_flowlens_inspector_e2e.py` +- Modify: `README.md` +- Modify: `docs/architecture.md` +- Create: `docs/flowlens-inspector-mcp.md` +- Modify: relevant GitHub CI workflow only after local commands pass. + +**Interfaces:** +- Produces: deterministic success, Tool Error, timeout, and cross-tenant fixtures. +- Produces: direct MCP E2E test and manual real-model acceptance script. +- Produces: reproducible P50/P95/P99 benchmark JSON. +- Produces: Chinese operator and interview explanation. + +- [ ] **Step 1: Write the failing end-to-end test** + +```python +@pytest.mark.anyio +async def test_runtime_calls_remote_inspector_and_records_mcp_evidence(full_stack): + result = await full_stack.ask("请使用 FlowLens 调查最近一次失败，并引用证据事件。") + assert result.contains_run_id("failed-run") + assert result.contains_event_sequence() + + events = await full_stack.events_for_origin_run() + assert "mcp.tool.start" in {event["event_type"] for event in events} + assert "mcp.tool.end" in {event["event_type"] for event in events} +``` + +- [ ] **Step 2: Run test to verify RED** + +Run: + +```powershell +uv run pytest tests/test_flowlens_inspector_e2e.py -q +``` + +Expected: fail until fixture seeding, service startup, delegation, and Skill wiring are connected. + +- [ ] **Step 3: Implement deterministic seed and benchmark scripts** + +The seed script accepts `--runs`, `--events-per-run`, `--tenant`, `--user`, and `--scenario`. The benchmark calls real Streamable HTTP tools, records environment metadata, and writes JSON containing sample count, P50, P95, P99, error count, and configuration. + +- [ ] **Step 4: Implement the PowerShell verification path** + +`verify_flowlens_mcp.ps1` performs: + +1. Required environment validation without printing secrets. +2. Key generation when absent. +3. Docker build/start. +4. Live and ready checks. +5. Token acquisition. +6. MCP `tools/list`. +7. Authorized and unauthorized tool calls. +8. Audit and Timeline evidence checks. +9. Optional PostgreSQL/Redis profile. +10. Clear stop/cleanup instructions without deleting persistent user data. + +- [ ] **Step 5: Make deterministic E2E GREEN** + +Run: + +```powershell +uv run pytest tests/test_flowlens_inspector_e2e.py tests/flowlens_inspector_mcp tests/test_mcp_delegation.py tests/test_flowlens_incident_skill.py -q +uv run ruff check packages/flowlens-inspector-mcp packages/harness/deerflow/mcp app/gateway scripts +``` + +Expected: all selected tests pass and Ruff reports no errors. + +- [ ] **Step 6: Run the complete Docker acceptance** + +Run from repository root: + +```powershell +powershell -ExecutionPolicy Bypass -File scripts/verify_flowlens_mcp.ps1 +``` + +Expected: script prints successful live/ready, discovery, authorized query, denied cross-tenant query, audit, and Timeline evidence checks. + +- [ ] **Step 7: Perform the user-led real-model demonstration** + +The user starts DeerFlow, creates one normal run and one controlled failure, then sends: + +```text +请使用 FlowLens Incident Response 调查本会话最近一次失败。 +先给出确定性根因，再引用关键 Timeline 事件；如果数据不完整请明确说明。 +``` + +Verify the final answer, the selected MCP tools, downloaded evidence, audit row, and `/agentops` Timeline together. Record only observed output. + +- [ ] **Step 8: Benchmark SQLite and enterprise profiles** + +Run both: + +```powershell +cd backend +uv run python scripts/benchmark_inspector_mcp.py --runs 1000 --events-per-run 100 --output ../artifacts/mcp-benchmark-sqlite.json +``` + +```powershell +uv run python scripts/benchmark_inspector_mcp.py --runs 1000 --events-per-run 100 --profile enterprise --output ../artifacts/mcp-benchmark-postgres.json +``` + +Do not place numbers in README until both output files exist and are reviewed. + +- [ ] **Step 9: Update documentation with verified commands and evidence** + +Document: + +- Architecture and trust boundaries. +- Tool contracts and Skill workflow. +- Local and enterprise startup. +- Token claims and scope table. +- Failure handling and audit fields. +- Three demonstration scenarios. +- Actual test and benchmark output with environment. +- Honest boundary: local end-to-end and fault-injection validation, not production customer deployment. + +- [ ] **Step 10: Run the final verification gate** + +Run: + +```powershell +cd backend +$env:PYTHONPATH='.' +uv run pytest -q +uv run ruff check . +cd .. +docker compose -f docker/docker-compose-dev.yaml config +docker compose -f docker/docker-compose.yaml -f docker/docker-compose-flowlens-mcp-enterprise.yaml config +powershell -ExecutionPolicy Bypass -File scripts/verify_flowlens_mcp.ps1 +git diff --check +git status --short +``` + +Expected: backend suite and Ruff pass, both Compose configurations validate, live-stack verification succeeds, no whitespace errors exist, and only intentional uncommitted evidence files remain. + +- [ ] **Step 11: Commit and report Stage 4** + +```powershell +git add backend/scripts scripts/verify_flowlens_mcp.ps1 backend/tests/test_flowlens_inspector_e2e.py README.md docs .github +git commit -m "docs(mcp): publish verified inspector workflow" +``` + +Stage 4 report must distinguish automated evidence, user-led real-model evidence, benchmark measurements, and unverified production assumptions. + +--- + +## Review Checkpoints + +- **Checkpoint A after Stage 1:** inspect JWT claims, Gateway Bearer behavior, tenant migration, and cross-tenant `404`. +- **Checkpoint B after Stage 2:** inspect five MCP schemas, a real Streamable HTTP tool call, ToolMessage return, and `mcp.tool.*` events. +- **Checkpoint C after Stage 3:** inspect timeout/retry/circuit behavior, audit redaction, Prometheus metrics, and both Docker profiles. +- **Checkpoint D after Stage 4:** run the demonstration together and compare every README/result claim against fresh artifacts. + +## Execution Handoff + +Plan complete and saved to `docs/superpowers/plans/2026-07-28-flowlens-inspector-mcp.md`. + +The selected execution method is **Inline Execution**: execute Stage 1 through Stage 4 in this task, stop after each stage for the user-facing report and hands-on explanation, and do not use task-per-subagent development. diff --git a/docs/superpowers/plans/2026-07-29-flowlens-agentops-investigation-workbench.md b/docs/superpowers/plans/2026-07-29-flowlens-agentops-investigation-workbench.md new file mode 100644 index 00000000..e0ce7d08 --- /dev/null +++ b/docs/superpowers/plans/2026-07-29-flowlens-agentops-investigation-workbench.md @@ -0,0 +1,273 @@ +# FlowLens AgentOps Investigation Workbench Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use `superpowers:executing-plans` to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Transform AgentOps into a responsive, evidence-led Trace-first investigation workbench while preserving all current diagnostics behavior and API contracts. + +**Architecture:** Keep the existing diagnostics-source queries and replay/remote modes. Add a scoped AgentOps design-token stylesheet, a `RunOverview` presentational component, and a pure `TraceWaterfall` derived from normalized Timeline items. Compose them inside the existing workspace and restyle existing panes without changing source contracts. + +**Tech Stack:** Next.js 16, React 19, TypeScript 5.8, Tailwind CSS v4, Lucide React, TanStack Query, Vitest, Playwright. + +## Global Constraints + +- Preserve existing `/agentops` route, source contracts, remote/replay modes, filters, Explain, Export, evidence navigation, and event-detail redaction. +- Use only existing dependencies and real query data; do not modify backend schemas or APIs. +- Scope visual tokens to `.agentops-shell`; use the existing system font stack and `prefers-reduced-motion` behavior. +- No large gradients, neon effects, card-stack layout, excessive shadow, or horizontal page overflow. +- Validate with `pnpm.cmd test`, `pnpm.cmd check`, and `pnpm.cmd test:e2e:agentops-demo`. + +--- + +## File Structure + +- Create `frontend/src/styles/agentops.css`: AgentOps-only colors, spacing, typography, control, status, and motion tokens. +- Modify `frontend/src/styles/globals.css`: import the AgentOps token stylesheet. +- Create `frontend/src/components/agentops/run-overview.tsx`: selected-run identity and status header. +- Create `frontend/src/components/agentops/trace-waterfall.tsx`: pure trace-bar projection and selectable waterfall component. +- Modify `frontend/src/components/agentops/agentops-workspace.tsx`: compose workbench, overview, waterfall, existing Timeline, and responsive tabs/drawer. +- Modify `frontend/src/components/agentops/{metrics-strip,run-explorer,timeline,timeline-row,diagnostics-panel,event-detail-sheet}.tsx`: apply scoped surfaces, controls, status hierarchy, and responsive layout classes without changing public props. +- Create `frontend/tests/unit/components/trace-waterfall.test.tsx`: verify timestamp/duration projection and selection behavior. +- Modify `frontend/tests/unit/components/agentops-workspace.test.tsx`: verify overview, waterfall, and existing evidence navigation. +- Modify `frontend/tests/e2e/agentops-demo.spec.ts`: assert Trace Waterfall works at desktop/tablet/mobile and retain overflow check. + +## Task 1: Scope AgentOps Design Tokens and Run Identity Header + +**Files:** +- Create: `frontend/src/styles/agentops.css` +- Modify: `frontend/src/styles/globals.css` +- Create: `frontend/src/components/agentops/run-overview.tsx` +- Modify: `frontend/tests/unit/components/agentops-workspace.test.tsx` + +**Interfaces:** +- Consumes: `RunSummary` from `@/core/agentops/types` and `mode: "remote" | "replay"`. +- Produces: `RunOverview({ run, mode }: { run: RunSummary | null; mode: "remote" | "replay" })`. + +- [ ] **Step 1: Write the failing run overview test** + +```tsx +expect(await screen.findByText("Run context")).toBeVisible(); +expect(screen.getByText(/Thread:/)).toBeVisible(); +expect(screen.getByText(/Replay data source/)).toBeVisible(); +``` + +- [ ] **Step 2: Run the unit test to verify it fails** + +Run: `pnpm.cmd vitest run tests/unit/components/agentops-workspace.test.tsx` + +Expected: FAIL because `Run context` and the source label are not rendered. + +- [ ] **Step 3: Add scoped tokens and the header component** + +```tsx +export function RunOverview({ run, mode }: { run: RunSummary | null; mode: "remote" | "replay" }) { + return
...
; +} +``` + +Define `--agentops-surface`, `--agentops-canvas`, `--agentops-ink`, `--agentops-muted`, `--agentops-border`, `--agentops-accent`, semantic status variables, `--agentops-radius`, and `--agentops-motion` under `.agentops-shell`. Import the stylesheet after Tailwind imports. + +- [ ] **Step 4: Run the unit test to verify it passes** + +Run: `pnpm.cmd vitest run tests/unit/components/agentops-workspace.test.tsx` + +Expected: PASS with the existing replay interaction tests still green. + +- [ ] **Step 5: Commit the task** + +```powershell +git add frontend/src/styles/agentops.css frontend/src/styles/globals.css frontend/src/components/agentops/run-overview.tsx frontend/tests/unit/components/agentops-workspace.test.tsx +git commit -m "feat(agentops): add workbench design tokens" +``` + +## Task 2: Build the Real-data Trace Waterfall + +**Files:** +- Create: `frontend/src/components/agentops/trace-waterfall.tsx` +- Create: `frontend/tests/unit/components/trace-waterfall.test.tsx` + +**Interfaces:** +- Consumes: `items: TimelineItem[]`, `selectedSeq?: number`, and `onSelect?: (item: TimelineItem) => void`. +- Produces: `TraceWaterfall({ items, selectedSeq, onSelect })` and exported `buildTraceRows(items)` returning rows with `item`, `offsetPercent`, `widthPercent`, and `depth`. + +- [ ] **Step 1: Write failing projection and interaction tests** + +```tsx +expect(buildTraceRows(items)[1]).toMatchObject({ offsetPercent: 50, widthPercent: 50, depth: 1 }); +await user.click(screen.getByRole("button", { name: /trace event 2/i })); +expect(onSelect).toHaveBeenCalledWith(items[1]); +``` + +Use deterministic ISO timestamps with a 200ms total trace window and one item whose `parent_span_id` targets the first item. Add a zero-duration item and assert it renders a visible minimum-width marker. + +- [ ] **Step 2: Run the focused test to verify it fails** + +Run: `pnpm.cmd vitest run tests/unit/components/trace-waterfall.test.tsx` + +Expected: FAIL because `buildTraceRows` and `TraceWaterfall` do not exist. + +- [ ] **Step 3: Implement trace derivation and rendering** + +```tsx +const rows = buildTraceRows(items); +return

Trace Waterfall

{rows.map(renderTraceRow)}
; +``` + +Parse timestamps with `Date.parse`, use the earliest valid timestamp as the origin, and normalize the latest `timestamp + duration_ms` as the end. If timestamps are invalid or all durations are absent, render a compact chronological fallback list. Clamp each bar to 2%-100% so zero-duration events remain inspectable. Derive nesting with the existing cycle-safe parent-span traversal pattern. + +- [ ] **Step 4: Run focused tests to verify they pass** + +Run: `pnpm.cmd vitest run tests/unit/components/trace-waterfall.test.tsx` + +Expected: PASS for normal, zero-duration, invalid-timestamp, and selection cases. + +- [ ] **Step 5: Commit the task** + +```powershell +git add frontend/src/components/agentops/trace-waterfall.tsx frontend/tests/unit/components/trace-waterfall.test.tsx +git commit -m "feat(agentops): add timeline trace waterfall" +``` + +## Task 3: Compose the A1 Investigation Workbench + +**Files:** +- Modify: `frontend/src/components/agentops/agentops-workspace.tsx` +- Modify: `frontend/src/components/agentops/metrics-strip.tsx` +- Modify: `frontend/src/components/agentops/timeline.tsx` +- Modify: `frontend/tests/unit/components/agentops-workspace.test.tsx` + +**Interfaces:** +- Consumes: `RunOverview`, `TraceWaterfall`, existing `RunExplorer`, `MetricsStrip`, `Timeline`, and `DiagnosticsPanel`. +- Produces: a desktop three-surface workbench and existing tablet/mobile drawer plus tabs. + +- [ ] **Step 1: Add failing composition assertions** + +```tsx +expect(await screen.findByRole("region", { name: "Run context" })).toBeVisible(); +expect(screen.getByRole("region", { name: "Execution trace" })).toBeVisible(); +``` + +- [ ] **Step 2: Run the workspace test to verify it fails** + +Run: `pnpm.cmd vitest run tests/unit/components/agentops-workspace.test.tsx` + +Expected: FAIL because the new labelled regions are absent. + +- [ ] **Step 3: Integrate components and selection state** + +```tsx + + + +``` + +Place overview and metrics above the center surface. Preserve compact controls; add an `Trace` tab alongside Timeline and Root Cause below the desktop breakpoint. Use semantic `
` and `
+ + + + {diagnosticsQuery.data && selected ? ( + toggleFocus("cause")} + /> + } + onEvidenceSelect={(seq) => { + setSelectedEvent( + diagnosticsQuery.data.timeline.find( + (item) => item.seq === seq, + ) ?? null, + ); + }} + /> + ) : ( + + )} + + ) : null} - -
- {diagnosticsQuery.data && selected ? ( - { - setSelectedEvent( - diagnosticsQuery.data.timeline.find( - (item) => item.seq === seq, - ) ?? null, - ); - }} - /> - ) : ( - - )} + {!isDesktopWorkbench ? ( +
+ {runsOpen ? ( +
+ ) : null}
void; + headerAction?: ReactNode; }) { const source = useDiagnosticsSource(); + const hasIncident = failure.primary_cause !== "NONE"; const [explanation, setExplanation] = useState( null, ); @@ -52,42 +56,43 @@ export function DiagnosticsPanel({ return (