Skip to content

fix: Auto monitor setup alignments #159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
OzBenSimhonTraceloop merged 6 commits into from
Mar 8, 2026
Merged

fix: Auto monitor setup alignments #159

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f2deb86
wip
OzBenSimhonTraceloop Mar 4, 2026
8f34a33
wip
OzBenSimhonTraceloop Mar 5, 2026
2b84269
wip
OzBenSimhonTraceloop Mar 5, 2026
8d40fa4
wip
OzBenSimhonTraceloop Mar 5, 2026
c592c52
wip
OzBenSimhonTraceloop Mar 5, 2026
848d9ea
wip
OzBenSimhonTraceloop Mar 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 53 additions & 23 deletions api-reference/auto-monitor-setups/create-an-auto-monitor-setup.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,19 +27,20 @@ An auto monitor setup is an asynchronous process that attempts to create a [Trac
See the full list of available slugs in the [Evaluator Slugs](/evaluators/evaluator-slugs) reference.
</ParamField>

<ParamField body="selector" type="object">
A map of span attribute key-value pairs used to filter which spans this monitor applies to. Only spans matching **all** provided key-value pairs will be evaluated.
<ParamField body="selector" type="object[]">
An array of filter rules used to match spans. Each rule specifies an attribute key, a value to match, and a source indicating where the attribute lives. Only spans matching **all** provided rules will be evaluated.

Keys are span attribute names and values can be strings, numbers, or booleans.
Each rule has the following fields:

**Example selector keys:**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `key` | string | Yes | The attribute key to filter on (e.g., `gen_ai.system`, `service.name`) |
| `value` | string | Conditional | The value to match against. **Required** for `equals`, `not_equals`, `contains`, `not_contains` operators |
| `values` | string[] | Conditional | List of values to match against. **Required** for `in`, `not_in` operators |
| `source` | string | Yes | Where the attribute lives: `span_attributes` or `resource_attributes` |
| `operator` | string | No | Comparison operator. Defaults to `equals`. One of: `equals`, `not_equals`, `contains`, `not_contains`, `exists`, `not_exists`, `in`, `not_in` |

| Key | Type | Example |
|-----|------|---------|
| `llm.request.model` | string | `"gpt-4o"`, `"claude-3-5-sonnet"` |
| `llm.request.max_tokens` | number | `1000` |

All available span attributes can be found in your [Traceloop traces page](https://app.traceloop.com).
All available span and resource attributes can be found in your [Traceloop traces page](https://app.traceloop.com).
</ParamField>

## Example Request
Expand All @@ -51,38 +52,67 @@ curl -X POST https://api.traceloop.com/v2/auto-monitor-setups \
-d '{
"external_id": "my-agent-monitor-1",
"evaluators": ["answer-relevancy", "toxicity-detector"],
"selector": {
"llm.vendor": "openai",
"llm.request.model": "gpt-4o"
}
"selector": [
{"key": "gen_ai.system", "value": "openai", "source": "span_attributes"},
{"key": "gen_ai.request.model", "value": "gpt-4o", "source": "span_attributes"}
]
}'
```

## Response

### 201 Created

Returns the created auto monitor setup object.
Returns the created auto monitor setup object. The `init_rules` array reflects the stored selector — same shape as the input `selector`.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix init_rules “same shape” wording mismatch.

Line 66 says init_rules has the same shape as input selector, but the response example includes explicit operator defaults that input can omit. This can confuse integrators.

🛠️ Suggested wording update 
-Returns the created auto monitor setup object. The `init_rules` array reflects the stored selector — same shape as the input `selector`.
+Returns the created auto monitor setup object. The `init_rules` array reflects the stored selector and may include defaulted fields (for example, `operator: "equals"`).
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@api-reference/auto-monitor-setups/create-an-auto-monitor-setup.mdx` at line
66, The doc text incorrectly claims init_rules is “same shape as the input
selector”; instead state that init_rules reflects the stored selector with any
defaulted fields (e.g., operator) expanded — i.e., the response may include
explicit operator defaults that callers can omit in the request. Update the
single sentence referencing init_rules/selector to note that server-side
defaults (such as operator) are populated in the response so integrators know
the returned shape can include those defaulted fields.


```json
{
"id": "ams_01j9...",
"id": "cmm...",
"external_id": "my-agent-monitor-1",
"org_id": "c108269c-...",
"project_id": "cm9v2g95l...",
"env_project_id": "cm9v2ga9i...",
"init_rules": [
{
"key": "gen_ai.system",
"value": "openai",
"source": "span_attributes",
"operator": "equals"
},
{
"key": "gen_ai.request.model",
"value": "gpt-4o",
"source": "span_attributes",
"operator": "equals"
}
],
"evaluators": [
{
"evaluator_id": "...",
"evaluator_type": "answer-relevancy",
"status": "active",
"binding_id": "..."
"input_schema": [
{ "type": "string", "name": "completion", "description": "The LLM response to evaluate" },
{ "type": "string", "name": "context", "description": "The context for the answer" },
{ "type": "string", "name": "question", "description": "The original question" }
],
"output_schema": [
{ "type": "float", "name": "answer_relevancy_score", "description": "Relevancy score (0-1)" }
],
"status": "pending"
},
{
"evaluator_id": "...",
"evaluator_type": "toxicity-detector",
"status": "active",
"binding_id": "..."
"input_schema": [
{ "type": "string", "name": "text", "description": "The text to analyze for toxicity" }
],
"output_schema": [
{ "type": "boolean", "name": "is_toxic", "description": "Whether the text is toxic" }
],
"status": "pending"
}
],
"created_at": "2025-01-15T10:30:00Z"
"status": "pending",
"created_at": "2026-01-15T10:30:00Z",
"updated_at": "2026-01-15T10:30:00Z"
}
```

Expand Down
Loading