diff --git a/ai/mcp-server.mdx b/ai/mcp-server.mdx new file mode 100644 index 00000000..96165eee --- /dev/null +++ b/ai/mcp-server.mdx @@ -0,0 +1,102 @@ +--- +title: 'Checkly MCP Server' +description: 'Connect MCP-compatible AI clients to Checkly to inspect checks, test sessions, RCA, result assets, status pages, incidents, and account context.' +sidebarTitle: 'Overview' +--- + +The Checkly MCP Server lets MCP-compatible AI clients connect to Checkly over Streamable HTTP and use Checkly tools from your conversation. + +Use the production MCP endpoint in your client: + +```text +https://api.checklyhq.com/mcp +``` + +Use it when your agent needs live Checkly account context, check status, check results, test sessions, root cause analyses, result assets, status pages, incidents, account environment variables, or when it needs to trigger existing checks. + + +Use [Checkly Skills](/ai/skills) and the [Checkly CLI](/cli/overview) when your agent needs to create, edit, test, bundle, or deploy check code. The MCP server runs remotely and cannot access your local filesystem. + + +## What you can do + +- Inspect account membership, plan, and feature entitlements. +- Check which checks are passing, failing, muted, or deactivated. +- Read compact check results and test-session results. +- Fetch normalized result asset manifests and bounded text assets. +- Start or read Rocky AI root cause analyses. +- Trigger existing deployed checks on demand. +- Manage account-level environment variables and secrets. +- Read status pages and manage status page incidents. + +## Quick start + + +Before connecting an MCP client, ensure you have: + +- A Checkly user account. +- Access to the Checkly account you want the MCP client to use. +- An MCP-compatible client that supports Streamable HTTP and OAuth. + + +Add the Checkly MCP endpoint to your client, complete the OAuth flow, then ask your client to verify the connection: + +```text title="Prompt" wrap +Use Checkly to show which accounts I can access. +``` + +If you belong to more than one Checkly account, set the account in your client configuration with the `X-Checkly-Account` header when your client supports headers, or pass `accountId` in tool calls. + +See [Set up the MCP server](/ai/mcp-server/setup) for client configuration examples. + +## Example prompts + +Basic prompts: + +```text title="Prompt" wrap +Show me the current status of my Checkly checks. +``` + +```text title="Prompt" wrap +List open status page incidents. +``` + +```text title="Prompt" wrap +What features are available on my current Checkly plan? +``` + +Advanced prompts: + +```text title="Prompt" wrap +Investigate the latest failed result for the checkout API check. If there is an error group, check whether an RCA already exists before triggering a new one. +``` + +```text title="Prompt" wrap +Trigger the checks tagged production-smoke, then follow the test session until results are available. +``` + +```text title="Prompt" wrap +Create a major status page incident for the API outage, but do not notify subscribers until I confirm the message. +``` + +## MCP, Skills, and CLI + +MCP, Checkly Skills, and the Checkly CLI are complementary. The CLI can also read live Checkly data and remains the most capable path for scripted workflows and local Monitoring as Code projects. + +Use the MCP Server when you want a quick, OAuth-based connection from an MCP-compatible client without setting up the Checkly CLI in that environment. Use Checkly Skills with the CLI when your agent needs to create, edit, test, or deploy code from your local project. + +| Use case | Recommended path | +| --- | --- | +| Ask quick, ad hoc questions from an MCP-compatible client | MCP Server | +| Inspect live Checkly data without local CLI setup | MCP Server | +| Trigger existing deployed checks from chat | MCP Server or [Checkly CLI](/cli/overview) | +| Create or edit check code | [Checkly Skills](/ai/skills) with the [Checkly CLI](/cli/overview) | +| Test, deploy, or automate local Monitoring as Code projects | [Checkly CLI](/cli/overview) | +| Add reusable Checkly best practices to an agent | [Checkly Skills](/ai/skills) | + +## Learn more + +- [Set up the MCP server](/ai/mcp-server/setup) +- [MCP tools reference](/ai/mcp-server/tools) +- [Security and permissions](/ai/mcp-server/security-and-permissions) +- [Troubleshooting](/ai/mcp-server/troubleshooting) diff --git a/ai/mcp-server/security-and-permissions.mdx b/ai/mcp-server/security-and-permissions.mdx new file mode 100644 index 00000000..fa145d50 --- /dev/null +++ b/ai/mcp-server/security-and-permissions.mdx @@ -0,0 +1,76 @@ +--- +title: 'MCP security and permissions' +description: 'Understand how the Checkly MCP Server authenticates users, filters tools by permission, resolves account context, and handles write actions.' +sidebarTitle: 'Security & permissions' +--- + +The Checkly MCP Server uses OAuth and Checkly account authorization together. OAuth permissions decide which tools are visible to your MCP client. Checkly account membership, role, and feature entitlements decide whether a specific tool call can operate on an account. + +## Authentication + +The MCP Server accepts Auth0-issued bearer tokens for `https://api.checklyhq.com/mcp`. Your MCP client completes the OAuth flow and sends the token with requests to the MCP endpoint. + +Checkly maps the token subject to a Checkly user, then loads that user's account memberships and account context for tool calls. + +## OAuth permissions + +| Permission | Description | +| --- | --- | +| `checkly:account:read` | Read your Checkly account membership and status | +| `checkly:account:invite` | Invite members to your Checkly account | +| `checkly:checks:read` | List checks, their status and results | +| `checkly:checks:run` | Trigger Checkly checks and on-demand test sessions | +| `checkly:incidents:read` | Read your Checkly incidents | +| `checkly:incidents:write` | Create and update your Checkly incidents | +| `checkly:environment-variables:read` | Read your Checkly account environment variables (secret values excluded) | +| `checkly:environment-variables:write` | Create, update and delete your Checkly account environment variables | +| `checkly:status-pages:read` | Read your Checkly status pages | +| `checkly:rca:read` | Read your Checkly root cause analyses | +| `checkly:rca:run` | Run Checkly root cause analysis for your account | +| `checkly:test-sessions:read` | Read your Checkly test sessions | +| `checkly:assets:read` | Read your Checkly assets | + +Tools are filtered from `tools/list` when the MCP session does not include the required permission. Tool calls are also rejected if the session lacks the required permission. + +## Account resolution + +Most tools operate on one Checkly account. Checkly resolves the account in this order: + +1. `accountId` in the tool arguments. +2. `X-Checkly-Account` on the MCP connection. +3. Your sole account, if your user belongs to exactly one account. + +If you belong to multiple accounts and no account can be resolved, the tool returns a message with the accounts available to your session. + +Accounts that require mTLS are not available through the MCP Server. + +## Role checks + +Some tools require both an OAuth permission and a Checkly account role: + +| Tool or action | Additional account access required | +| --- | --- | +| `invite-account-member` | Owner or Admin | +| `create-account-environment-variable` | Write access | +| `update-account-environment-variable` | Write access | +| `trigger-checks` | Run access | +| `trigger-root-cause-analysis` | Run access | +| Status page incident writes | Access required by the underlying incident action | + +## Write-action safety + +Some MCP tools create side effects: + +- `invite-account-member` sends an invite email and is not idempotent. +- `trigger-checks` consumes check-run execution quota. +- `trigger-root-cause-analysis` consumes RCA invocation quota. +- `create-status-page-incident`, `update-status-page-incident`, and `resolve-status-page-incident` can notify subscribers and are not idempotent. +- Environment variable write tools can create or replace account-level variables and secrets. + +Review write tool calls in your MCP client before approving them. + +## Secrets + +MCP read tools never reveal Checkly secret values. Secret values are returned as `null`. + +When you create or update a secret through MCP, Checkly encrypts the value and does not echo it back in the tool response. diff --git a/ai/mcp-server/setup.mdx b/ai/mcp-server/setup.mdx new file mode 100644 index 00000000..e2838aa7 --- /dev/null +++ b/ai/mcp-server/setup.mdx @@ -0,0 +1,127 @@ +--- +title: 'Set up the Checkly MCP Server' +description: 'Configure an MCP-compatible client to connect to the Checkly MCP Server over Streamable HTTP.' +sidebarTitle: 'Setup' +--- + +Use the production MCP endpoint in your client: + +```text +https://api.checklyhq.com/mcp +``` + + +Before setting up the Checkly MCP Server, ensure you have: + +- A Checkly user account. +- Access to the Checkly account you want your MCP client to use. +- An MCP-compatible client that supports Streamable HTTP and OAuth. + + +The Checkly MCP Server uses OAuth. Your MCP client connects to the endpoint, opens a browser-based login flow, and receives access to the Checkly tools it can use. + +The server exposes protected-resource metadata for OAuth discovery. Most MCP clients discover this automatically when they connect to the endpoint. + +## Codex + +Add the server: + +```bash Terminal +codex mcp add checkly --url https://api.checklyhq.com/mcp +``` + +Then log in: + +```bash Terminal +codex mcp login checkly +``` + +Restart Codex after adding the server. + +## Claude Code + +Add the server with the HTTP transport: + +```bash Terminal +claude mcp add --transport http checkly https://api.checklyhq.com/mcp +``` + +Start a Claude Code session and complete the OAuth flow when prompted. + +## Cursor + +Add the server to `.cursor/mcp.json` in your project, or to `~/.cursor/mcp.json` for all projects: + +```json mcp.json +{ + "mcpServers": { + "checkly": { + "url": "https://api.checklyhq.com/mcp" + } + } +} +``` + +Restart Cursor after changing the configuration. + +## VS Code + +Add the server to `.vscode/mcp.json` in your workspace, or to your VS Code user profile: + +```json mcp.json +{ + "servers": { + "checkly": { + "type": "http", + "url": "https://api.checklyhq.com/mcp" + } + } +} +``` + +Restart VS Code after changing the configuration. + +## Other MCP clients + +For clients that accept a generic `mcpServers` configuration, use: + +```json mcp.json +{ + "mcpServers": { + "checkly": { + "url": "https://api.checklyhq.com/mcp", + "headers": { + "X-Checkly-Account": "" + } + } + } +} +``` + +Omit `headers` if you want Checkly to resolve the account from tool arguments or your user membership. + +## Choose an account + +Most tools need an account context. Checkly resolves the account in this order: + +1. The `accountId` argument passed to the tool. +2. The `X-Checkly-Account` header on the MCP connection. +3. Your sole account, if your user belongs to exactly one account. + +If you belong to multiple accounts and do not provide an account, the tool returns a message with the accounts you can access. + +## Verify the connection + +After authentication, ask your MCP client: + +```text title="Prompt" wrap +Use Checkly to show which accounts I can access. +``` + +Then verify tool access: + +```text title="Prompt" wrap +Use Checkly to list the tools available to this session. +``` + +The visible tools depend on the OAuth permissions granted to your MCP session. See [Security and permissions](/ai/mcp-server/security-and-permissions) for details. diff --git a/ai/mcp-server/tools.mdx b/ai/mcp-server/tools.mdx new file mode 100644 index 00000000..819652e8 --- /dev/null +++ b/ai/mcp-server/tools.mdx @@ -0,0 +1,144 @@ +--- +title: 'Checkly MCP tools' +description: 'Reference for the tools exposed by the Checkly MCP Server, grouped by workflow.' +sidebarTitle: 'Tools' +--- + +This reference reflects the current MCP Server v1 tool surface. It is not a roadmap or a commitment to future tools. + +Each tool is shown only when the MCP session has the required OAuth permission. Some write tools also require a matching Checkly account role, such as Admin or Read & Write. + +## Account tools + +| Tool | Type | Description | +| --- | --- | --- | +| `whoami` | Read | Return the authenticated user and account memberships visible to the MCP session. | +| `get-account-entitlements` | Read | Return the resolved account plan and feature entitlement state. | +| `list-account-members` | Read | List account members and pending invites. | +| `invite-account-member` | Write | Invite a user to the resolved account. Requires Owner or Admin access, sends an invite email, and is not idempotent. | +| `prepare-local-check-authoring` | Read | Return an account-aware runbook for creating, testing, and deploying check code with the local Checkly CLI. | + +Example prompts: + +```text title="Prompt" wrap +Use Checkly to show which accounts I can access. +``` + +```text title="Prompt" wrap +Prepare the local Checkly CLI steps to create a browser check for checkout. +``` + +## Check tools + +| Tool | Type | Description | +| --- | --- | --- | +| `get-check-status` | Read | Return current status for checks in the account, including check type, activation state, and passing or failing state. | +| `list-check-results` | Read | List recent compact results for one check. Raw logs, traces, screenshots, videos, and download URLs are omitted. | +| `get-check-result` | Read | Return compact detail for one check result. | +| `get-check-performance-summary` | Read | Return average, P50, P95, and P99 response-time metrics for one check over a date range when data is available. | +| `trigger-checks` | Write | Run existing deployed checks on demand and record the run as a test session. Consumes check-run execution quota. | + +`trigger-checks` only runs checks that already exist on the account. Use `prepare-local-check-authoring` and the Checkly CLI when you need to create, test, or deploy local check code. + +Example prompts: + +```text title="Prompt" wrap +Show me failing checks and summarize what changed in their latest results. +``` + +```text title="Prompt" wrap +Trigger the checks tagged production-smoke, then poll the test session until results are available. +``` + +## Test session tools + +| Tool | Type | Description | +| --- | --- | --- | +| `list-test-sessions` | Read | List recent test sessions with filters and cursor ordering. | +| `get-test-session` | Read | Return one test session and compact result rows. | +| `get-test-session-result` | Read | Return compact detail for one test-session result, including useful run metadata and small summaries. | + +Example prompt: + +```text title="Prompt" wrap +Find the latest failed test session and show me the failed result IDs. +``` + +## Result asset tools + +| Tool | Type | Description | +| --- | --- | --- | +| `list-check-result-assets` | Read | Return a normalized asset manifest for a check result. | +| `fetch-check-result-asset` | Read | Fetch one selected check-result asset. Text assets may be returned inline with a byte limit; binary or large assets return a download link. | +| `list-test-session-result-assets` | Read | Return a normalized asset manifest for a test-session result. | +| `fetch-test-session-result-asset` | Read | Fetch one selected test-session-result asset. Text assets may be returned inline with a byte limit; binary or large assets return a download link. | + +Example prompt: + +```text title="Prompt" wrap +For the failed check result, list available assets and fetch the text log if one exists. +``` + +## Root cause analysis tools + +| Tool | Type | Description | +| --- | --- | --- | +| `get-error-group-root-cause-analyses` | Read | Return a check or test-session error group and any existing root cause analyses. | +| `get-root-cause-analysis` | Read | Return one root cause analysis by RCA ID. Pending analyses return compact polling status. | +| `trigger-root-cause-analysis` | Write | Trigger a new RCA for a check or test-session error group. Requires run access and consumes RCA invocation quota. | + +Call `get-error-group-root-cause-analyses` before triggering a new RCA so your agent can reuse existing analysis when available. + +Example prompt: + +```text title="Prompt" wrap +Check whether this error group already has an RCA. If it does not, ask me before triggering a new one. +``` + +## Environment variable tools + +| Tool | Type | Description | +| --- | --- | --- | +| `list-account-environment-variables` | Read | List account-level variables and secrets. Secret values are returned as `null`. | +| `get-account-environment-variable` | Read | Get one account-level variable by key. Secret values are returned as `null`. | +| `create-account-environment-variable` | Write | Create an account-level variable or secret. Secret values are encrypted and not echoed back. | +| `update-account-environment-variable` | Write | Update an account-level variable or secret by key. To update a secret, pass `secret: true`. | + +Example prompt: + +```text title="Prompt" wrap +List account environment variables and tell me whether API_TOKEN is stored as a secret. +``` + +## Status page tools + +| Tool | Type | Description | +| --- | --- | --- | +| `list-status-pages` | Read | List status pages, including cards and services needed for incident targeting. | +| `get-status-page` | Read | Get one status page with cards and services. | + +Example prompt: + +```text title="Prompt" wrap +List my status pages and show the service IDs for the production page. +``` + +## Incident tools + +| Tool | Type | Description | +| --- | --- | --- | +| `list-status-page-incidents` | Read | List status page incidents, optionally filtered by page and incident status. | +| `get-status-page-incident` | Read | Get one status page incident with services and incident updates. | +| `create-status-page-incident` | Write | Create a new status page incident and optionally notify subscribers. Not idempotent. | +| `update-status-page-incident` | Write | Post a progress update to an existing incident and optionally notify subscribers. Not idempotent. | +| `resolve-status-page-incident` | Write | Resolve an incident by posting a final resolved update and optionally notifying subscribers. Not idempotent. | + +Example prompts: + +```text title="Prompt" wrap +Show open status page incidents and summarize their latest updates. +``` + +```text title="Prompt" wrap +Draft a major status page incident for the API outage and wait for confirmation before notifying subscribers. +``` diff --git a/ai/mcp-server/troubleshooting.mdx b/ai/mcp-server/troubleshooting.mdx new file mode 100644 index 00000000..3a8f6500 --- /dev/null +++ b/ai/mcp-server/troubleshooting.mdx @@ -0,0 +1,90 @@ +--- +title: 'Troubleshoot the Checkly MCP Server' +description: 'Resolve common Checkly MCP Server setup, authentication, account selection, and tool access issues.' +sidebarTitle: 'Troubleshooting' +--- + + +Before troubleshooting, confirm: + +- Your MCP client is configured with `https://api.checklyhq.com/mcp`. +- You completed the OAuth login flow. +- You restarted your MCP client after editing configuration. +- Your Checkly user has access to the account you want to use. + + +## Authentication fails + +If your client reports that authentication is required or invalid: + +1. Re-run your client's MCP login flow. +2. Confirm the endpoint is exactly `https://api.checklyhq.com/mcp`. +3. Remove stale Checkly MCP credentials from the client if it keeps reusing an old token. +4. Reconnect and call `whoami`. + +For Codex, run: + +```bash Terminal +codex mcp login checkly +``` + +## No tools are listed + +Visible tools depend on the OAuth permissions granted to the MCP session. If no tools appear: + +1. Use your MCP client's refresh tools option. +2. Restart the client if the tools list still does not update. +3. Reconnect and complete OAuth again. + +After the tools list refreshes, call `whoami` or ask Checkly which accounts you can access to verify the connection. + +## A specific tool is missing + +Each tool requires an OAuth permission. For example: + +- Check status and results require permission to list checks, their status and results. +- Test sessions require permission to read your Checkly test sessions. +- Asset tools require permission to read Checkly assets plus the related result permission. +- Incident write tools require permission to create and update your Checkly incidents. + +See [MCP tools](/ai/mcp-server/tools) for the full tool list. + +## The tool asks for an account + +If you belong to multiple Checkly accounts, pass `accountId` in the tool call or configure the `X-Checkly-Account` header when your client supports headers. + +Ask your client: + +```text title="Prompt" wrap +Use Checkly to show the accounts I can access, then use accountId for the next Checkly request. +``` + +## The MCP server cannot create or deploy check code + +The MCP server runs remotely and cannot access your local project files. It cannot author, bundle, test, or deploy check code. + +Ask for a CLI handoff instead: + +```text title="Prompt" wrap +Use Checkly to prepare the local check authoring runbook for a browser check. +``` + +Then run the returned Checkly CLI steps in your local project. + +## Write tools did more than expected + +Some write tools are not idempotent. Retrying a call can create another invite, another incident, or another incident update. + +Before approving write tool calls, check: + +- The target account ID. +- The target status page or check ID. +- Whether subscribers will be notified. +- Whether the action consumes check-run or RCA quota. + +## Browser clients or web IDEs cannot connect + +Some browser-based clients require CORS support and may handle OAuth differently from command-line clients. If a browser-based client cannot connect: + +1. Try a command-line client such as Codex or Claude Code. +2. Confirm the client supports Streamable HTTP MCP servers with OAuth. diff --git a/ai/overview.mdx b/ai/overview.mdx index bc4f0455..f98c26b5 100644 --- a/ai/overview.mdx +++ b/ai/overview.mdx @@ -1,6 +1,6 @@ --- title: 'AI Agents & Coding Assistants' -description: 'Use AI agents and coding assistants to create, update, and deploy your Checkly monitoring setup with skills and rules.' +description: 'Use AI agents and coding assistants with Checkly Skills, rules, and the Checkly MCP Server.' sidebarTitle: 'Agents & Checkly' keywords: ['mcp', 'ai', 'rules', 'generate checks', 'agents', 'context'] --- @@ -44,35 +44,41 @@ Provide the necessary Checkly context and let your AI agent of choice do the res ## Add Checkly context to your AI agent conversation -Install [Checkly Skills](/ai/skills) or add the [Checkly Rules](/ai/rules) to your AI conversation to give your AI agent enough context to perform Checkly-related tasks. +Install [Checkly Skills](/ai/skills), add [Checkly Rules](/ai/rules), or connect the [Checkly MCP Server](/ai/mcp-server) to give your AI agent enough context to perform Checkly-related tasks. - + Let your agents pull in all required Checkly context on demand. + +Give your agents live access to Checkly account data and remote actions. + + Include the entire Checkly context in commands or documentation. -### Skills vs Rules +### Skills, MCP, and Rules **Use Skills** when your AI agent supports the [Agent Skills](https://agentskills.io) standard. Skills load context on demand, keeping your agent's context window lean until Checkly-related tasks arise. This is the recommended approach for compatible agents. +**Use the MCP Server** when your agent needs live Checkly data or remote account actions, such as reading check status, inspecting test sessions, triggering existing checks, reading RCA, or managing status page incidents. + **Use Rules** when your agent doesn't support skills or when you want the Checkly context always available. Rules files are loaded at session start and provide consistent context throughout your conversation. -## Why is there no Checkly MCP server (yet)? +## When to use the Checkly MCP Server The MCP concept is often used to enable LLMs to interact with external systems. It acts as a bridge between the AI model and the target system, translating natural language commands into actionable API calls or code snippets. -**With Monitoring as Code and agent-friendly CLI commands, Checkly already provides a native way to control your monitoring infrastructure via code and access monitoring results via the command line.** +The [Checkly MCP Server](/ai/mcp-server) connects MCP-compatible clients to Checkly over Streamable HTTP. It exposes tools for account context, check status and results, test sessions, result assets, root cause analyses, environment variables, status pages, incidents, and triggering existing checks. -Whether you need to configure your monitoring setup, access check data or declare incidents, AI can write and update the necessary construct files and execute the Checkly CLI commands autonomously. +**Use the MCP Server for live Checkly account operations. Use Skills and the Checkly CLI for Monitoring as Code authoring.** -We are researching additional AI-native workflows. [Let us know in the public roadmap](https://feedback.checklyhq.com/p/checkly-mcp) if you are interested in more agent-friendly integrations. +The MCP server runs remotely and cannot access your local filesystem. When you need to create, edit, test, or deploy check code, use [Checkly Skills](/ai/skills) with the [Checkly CLI](/cli/overview). ## Markdown access @@ -131,7 +137,7 @@ Based on this, how do I set up a browser check with a custom user agent? ``` -## LLMs.txt +## LLMs.txt The [llms.txt standard](https://llmstxt.org/) provides a machine-readable index of all available documentation pages. Checkly publishes an `llms.txt` file at [`checklyhq.com/llms.txt`](https://www.checklyhq.com/llms.txt) that lists every documentation page with its markdown URL and a short description. @@ -162,4 +168,4 @@ curl https://www.checklyhq.com/llms.txt # Fetch a specific page from the index curl https://checklyhq.com/docs/detect/synthetic-monitoring/browser-checks/overview.md -``` \ No newline at end of file +``` diff --git a/docs.json b/docs.json index 8e42572b..b76090ba 100644 --- a/docs.json +++ b/docs.json @@ -350,6 +350,16 @@ "pages": [ "ai/rocky-ai", "ai/overview", + { + "group": "MCP Server", + "pages": [ + "ai/mcp-server", + "ai/mcp-server/setup", + "ai/mcp-server/tools", + "ai/mcp-server/security-and-permissions", + "ai/mcp-server/troubleshooting" + ] + }, "ai/skills", "ai/rules" ]