feat(k8s): support promql historical data queries by dangaiden · Pull Request #82 · sysdiglabs/sysdig-mcp-server

dangaiden · 2026-04-27T09:17:14Z

Enable users the ability to query historical data (if provided a timeline) from metric tools that use Sysdig Monitor with no data restrictions.

Adds optional start/end historical query parameters to all Sysdig Monitor k8s_list_* tools, enabling LLMs to query Kubernetes metrics over a past time window instead of only the current snapshot.
When provided, the underlying PromQL is wrapped in the appropriate aggregation for each tool:
- CPU/memory/pod count → avg_over_time
- Restarted pods → increase
- Unavailable pods → min_over_time >= 1
- HTTP/network errors → sum_over_time / N (rate per second)
- Inventory tools (clusters, nodes, workloads, etc.) → max_over_time > 0
When omitted, tools behave as before (instant snapshot)

…ools

Copilot

Pull request overview

Adds first-class historical querying support to the Sysdig Monitor k8s_list_* MCP tools by introducing shared start/end RFC3339 parameters, validating/resolving the time window, and wrapping PromQL appropriately for “windowed” queries while preserving legacy instant-snapshot behavior when omitted.

Changes:

Introduce shared time-window parsing/validation utilities (TimeWindow, ParseTimeWindow, WithTimeWindowParams) and apply them across k8s_list_* Monitor tools.
Update affected tools to pass GetQueryV1Params.Time (eval at end) and set a 60s timeout for windowed queries; deprecate interval for HTTP/network error tools with precedence rules + warnings.
Expand test coverage to include windowed query construction using an injected clock; document the behavior in tool docs and top-level README.

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
internal/infra/mcp/tools/utils.go	Adds shared time-window types/helpers and request arg presence detection.
internal/infra/mcp/tools/tools_suite_test.go	Adds shared test helpers for expected windowed query params and limits.
internal/infra/mcp/tools/tool_k8s_list_workloads.go	Adds start/end support, eval time + timeout, and windowed PromQL wrapping logic.
internal/infra/mcp/tools/tool_k8s_list_workloads_test.go	Adds mock clock + windowed query expectation cases.
internal/infra/mcp/tools/tool_k8s_list_underutilized_pods_memory_quota.go	Adds start/end support; wraps usage/limit in `avg_over_time` when windowed.
internal/infra/mcp/tools/tool_k8s_list_underutilized_pods_memory_quota_test.go	Adds mock clock + windowed query expectation.
internal/infra/mcp/tools/tool_k8s_list_underutilized_pods_cpu_quota.go	Adds start/end support; wraps usage/quota in `avg_over_time` when windowed.
internal/infra/mcp/tools/tool_k8s_list_underutilized_pods_cpu_quota_test.go	Adds mock clock + windowed query expectation.
internal/infra/mcp/tools/tool_k8s_list_top_unavailable_pods.go	Adds start/end support and Sysdig-canonical windowed unavailable semantics (`min_over_time >= 1`).
internal/infra/mcp/tools/tool_k8s_list_top_unavailable_pods_test.go	Adds mock clock + windowed semantics tests.
internal/infra/mcp/tools/tool_k8s_list_top_restarted_pods.go	Adds start/end support; uses `increase()` when windowed.
internal/infra/mcp/tools/tool_k8s_list_top_restarted_pods_test.go	Adds mock clock + windowed increase() expectations.
internal/infra/mcp/tools/tool_k8s_list_top_network_errors_in_pods.go	Adds start/end support; deprecates interval with precedence and windowed `sum_over_time/N` behavior.
internal/infra/mcp/tools/tool_k8s_list_top_network_errors_in_pods_test.go	Adds mock clock + windowed query expectations and clarifies legacy cases.
internal/infra/mcp/tools/tool_k8s_list_top_http_errors_in_pods.go	Adds start/end support; deprecates interval with precedence and windowed `sum_over_time/N` behavior.
internal/infra/mcp/tools/tool_k8s_list_top_http_errors_in_pods_test.go	Adds mock clock + windowed query expectations (including precedence over interval).
internal/infra/mcp/tools/tool_k8s_list_top_memory_consumed_workload.go	Adds start/end support; uses `avg_over_time` for windowed.
internal/infra/mcp/tools/tool_k8s_list_top_memory_consumed_workload_test.go	Adds mock clock + windowed query expectation.
internal/infra/mcp/tools/tool_k8s_list_top_memory_consumed_container.go	Adds start/end support; uses `avg_over_time` for windowed.
internal/infra/mcp/tools/tool_k8s_list_top_memory_consumed_container_test.go	Adds mock clock + windowed query expectation.
internal/infra/mcp/tools/tool_k8s_list_top_cpu_consumed_workload.go	Adds start/end support; uses `avg_over_time` for windowed.
internal/infra/mcp/tools/tool_k8s_list_top_cpu_consumed_workload_test.go	Adds mock clock + windowed query expectations + invalid window tests.
internal/infra/mcp/tools/tool_k8s_list_top_cpu_consumed_container.go	Adds start/end support; uses `avg_over_time` for windowed.
internal/infra/mcp/tools/tool_k8s_list_top_cpu_consumed_container_test.go	Adds mock clock + windowed query expectation.
internal/infra/mcp/tools/tool_k8s_list_pod_containers.go	Adds start/end support; uses `max_over_time(...) > 0` inventory semantics when windowed.
internal/infra/mcp/tools/tool_k8s_list_pod_containers_test.go	Adds mock clock + windowed inventory semantics test.
internal/infra/mcp/tools/tool_k8s_list_nodes.go	Adds start/end support; uses `max_over_time(...) > 0` inventory semantics when windowed.
internal/infra/mcp/tools/tool_k8s_list_nodes_test.go	Adds mock clock + windowed inventory semantics test.
internal/infra/mcp/tools/tool_k8s_list_cronjobs.go	Adds start/end support; uses `max_over_time(...) > 0` inventory semantics when windowed.
internal/infra/mcp/tools/tool_k8s_list_cronjobs_test.go	Adds mock clock + windowed inventory semantics test.
internal/infra/mcp/tools/tool_k8s_list_count_pods_per_cluster.go	Adds start/end support; uses `avg_over_time` for windowed pod counts.
internal/infra/mcp/tools/tool_k8s_list_count_pods_per_cluster_test.go	Adds mock clock + windowed query expectation.
internal/infra/mcp/tools/tool_k8s_list_clusters.go	Adds start/end support; uses `max_over_time(...) > 0` inventory semantics when windowed.
internal/infra/mcp/tools/tool_k8s_list_clusters_test.go	Adds mock clock + windowed inventory semantics tests.
internal/infra/mcp/tools/README.md	Documents start/end semantics and the per-tool PromQL wrapping table + deprecation behavior.
cmd/server/main.go	Injects a system clock into all window-aware tools at registration time.
README.md	Adds a note pointing to the tool docs for windowed aggregation semantics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-27T09:21:23Z

+// RangeSelector returns the PromQL range-selector literal for this window, e.g. "[3600s]".
+// The duration is rounded down to whole seconds so the selector is stable and debuggable.
+func (w TimeWindow) RangeSelector() string {
+	return fmt.Sprintf("[%ds]", int64(w.End.Sub(w.Start).Seconds()))
+}


TimeWindow.RangeSelector() can emit a "[0s]" range when the resolved window is <1s. This can happen in the supported "start without end" case because end defaults to clk.Now() (sub-second precision) while start is second precision; if start is within the current second, End.Sub(Start) truncates to 0 seconds. PromQL range selectors do not accept 0s, and for HTTP/network error tools the derived windowSeconds can become 0, causing a division-by-zero in the query. Consider normalizing end to whole seconds (or rounding up) and/or rejecting windows shorter than 1s in ParseTimeWindow before returning the TimeWindow.

tembleking

Nit: pre-existing typo in utils.go:16 — "exampes" should be "examples". Not from this PR but you're touching the file, easy drive-by fix.

…erplate, and add missing ParseTimeWindow unit tests

…sArg helper

Copilot

Pull request overview

Copilot reviewed 38 out of 38 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

internal/infra/mcp/tools/utils.go:16

Examples writes to schema["exampes"], which looks like a typo and will prevent MCP JSON schema examples from being emitted/recognized. Rename the key to "examples" so tooling can surface these examples correctly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-27T15:13:20Z

+// newWindowedQueryParams constructs the GetQueryV1Params value that a windowed tool
+// invocation is expected to produce: Query string, Time = end.Unix() via FromQueryTime1,
+// and a 60s Timeout.
+func newWindowedQueryParams(query string, end time.Time) sysdig.GetQueryV1Params {
+	var qt sysdig.Time
+	Expect(qt.FromQueryTime1(end.Unix())).To(Succeed())
+	timeout := sysdig.Timeout("60s")
+	return sysdig.GetQueryV1Params{
+		Query:   query,
+		Time:    &qt,
+		Timeout: &timeout,
+	}


newWindowedQueryParams calls Expect(...) during helper execution. Since this helper is used while building DescribeTable Entry(...) values, it can run during package init / spec construction (before RegisterFailHandler / a running spec exists), which can panic or behave unpredictably. Prefer returning an error, or handle the FromQueryTime1 error without Gomega (e.g., panic with a clear message) and keep assertions inside It/table bodies.

Copilot · 2026-04-27T15:13:21Z

+// RangeSelector returns the PromQL range-selector literal for this window, e.g. "[3600s]".
+func (w TimeWindow) RangeSelector() string {
+	return fmt.Sprintf("[%ds]", int64(w.End.Sub(w.Start).Seconds()))
+}
+
+// WindowSeconds returns the duration of the window in whole seconds.
+func (w TimeWindow) WindowSeconds() int64 {
+	return int64(w.End.Sub(w.Start).Seconds())
+}


RangeSelector / WindowSeconds compute whole seconds via Duration.Seconds() (float64) and then cast to int64, which truncates and can yield 0 for sub-second windows. That can produce invalid PromQL like [0s] and even / 0 in the HTTP/network error tools. Compute whole seconds using integer duration arithmetic (e.g., w.End.Sub(w.Start)/time.Second) and ensure the result is >= 1 for non-zero windows.

dangaiden · 2026-04-27T17:07:56Z

All done, not taking care of the Copilot comments as per this PR.

tembleking · 2026-05-04T15:35:19Z

+func (w TimeWindow) RangeSelector() string {
+	return fmt.Sprintf("[%ds]", int64(w.End.Sub(w.Start).Seconds()))
+}


If someone calls RangeSelector() on a zero TimeWindow, this produces [0s] — invalid PromQL.

All callers currently guard with !tw.IsZero(), so not a live bug, but the method itself has no guard. A future caller could easily miss the precondition.

Suggested change

func (w TimeWindow) RangeSelector() string {

return fmt.Sprintf("[%ds]", int64(w.End.Sub(w.Start).Seconds()))

}

func (w TimeWindow) RangeSelector() string {

if w.IsZero() {

panic("RangeSelector called on zero TimeWindow")

}

return fmt.Sprintf("[%ds]", int64(w.End.Sub(w.Start).Seconds()))

}

Or return (string, error) if you prefer not to panic. Same applies to WindowSeconds() below.

tembleking · 2026-05-04T15:35:20Z

+func (w TimeWindow) ApplyToParams(params *sysdig.GetQueryV1Params) error {
+	evalTime, err := w.EvalTime()
+	if err != nil {
+		return err
+	}
+	params.Time = evalTime
+	if !w.IsZero() {
+		timeout := sysdig.Timeout(windowedQueryTimeout)
+		params.Timeout = &timeout
+	}
+	return nil
+}


On the zero-window path, this still sets params.Time = nil, overwriting whatever was there before. Not a bug today since callers don't pre-populate it, but it's a surprising side-effect for a "no-op" case.

Suggested change

func (w TimeWindow) ApplyToParams(params *sysdig.GetQueryV1Params) error {

evalTime, err := w.EvalTime()

if err != nil {

return err

}

params.Time = evalTime

if !w.IsZero() {

timeout := sysdig.Timeout(windowedQueryTimeout)

params.Timeout = &timeout

}

return nil

}

func (w TimeWindow) ApplyToParams(params *sysdig.GetQueryV1Params) error {

if w.IsZero() {

return nil

}

evalTime, err := w.EvalTime()

if err != nil {

return err

}

params.Time = evalTime

timeout := sysdig.Timeout(windowedQueryTimeout)

params.Timeout = &timeout

return nil

}

tembleking · 2026-05-04T15:35:20Z

+// Reads "start" and "end" from the request, validates them, and return the resolved TimeWindow.
+
+func ParseTimeWindow(request mcp.CallToolRequest, clk clock.Clock) (TimeWindow, error) {


nit: Blank line between comment and function disconnects them — this won't be treated as godoc.

Suggested change

// Reads "start" and "end" from the request, validates them, and return the resolved TimeWindow.

func ParseTimeWindow(request mcp.CallToolRequest, clk clock.Clock) (TimeWindow, error) {

// Reads "start" and "end" from the request, validates them, and return the resolved TimeWindow.

func ParseTimeWindow(request mcp.CallToolRequest, clk clock.Clock) (TimeWindow, error) {

tembleking · 2026-05-04T15:35:20Z

+		return TimeWindow{}, fmt.Errorf("end (%s) must be after start (%s)", end.Format(time.RFC3339), start.Format(time.RFC3339))
+	}
+
+	return TimeWindow{Start: start, End: end}, nil


No upper bound on window duration. A user could request start=2020-01-01T00:00:00Z, producing a multi-year range that will likely OOM Prometheus.

Worth adding a max window guard here (30d? 90d?), or at minimum documenting in the param description that there's a practical limit.

tembleking · 2026-05-04T15:35:20Z

+	It("truncates sub-second precision from now so RangeSelector never emits [0s]", func() {
+		mockClock.EXPECT().Now().Return(now)
+		_, err := tools.ParseTimeWindow(makeRequest(map[string]any{
+			"start": "2026-04-16T12:00:00Z",
+		}), mockClock)
+		Expect(err).To(HaveOccurred())
+	})
+})


Good coverage of ParseTimeWindow error paths, but missing direct unit tests for RangeSelector(), WindowSeconds(), EvalTime(), and ApplyToParams().

Also, this last test's description says "RangeSelector never emits [0s]" but it never calls RangeSelector() — it actually tests that ParseTimeWindow errors when start == truncated now. The description is misleading.

Suggested additions:

RangeSelector() returns correct string for a known 1h window → [3600s]

ApplyToParams() sets Time and Timeout correctly on non-zero window

WindowSeconds() on a 1h window returns 3600

tembleking · 2026-05-04T15:35:20Z

 		)
 	})
+
+	When("the time window is invalid", func() {


This is the only tool with error-path tests for invalid time windows. The other 15 tools skip them entirely.

Since ParseTimeWindow is shared, it's acceptable, but consider adding the same block to at least one more tool from a different category (e.g. http_errors, which has the legacy interval fallback) to catch tool-specific pre/post-processing bugs around the ParseTimeWindow call.

dangaiden added 6 commits April 27, 2026 10:51

feat: add start/end parameters for historical queries on k8s_list_* t…

485e6ce

…ools

refactor(tools): inline time-window helpers into utils.go

66063c7

end in the future → clamped to now, query proceeds normally

5583dc7

remove 7-day cap on historical query window

8c54225

Fix Readme mention of old MCP_MAX_INTERVAL

09dd09f

Concise wording

f6d6f07

dangaiden requested review from Copilot and tembleking April 27, 2026 09:17

dangaiden requested a review from a team as a code owner April 27, 2026 09:17

Copilot started reviewing on behalf of dangaiden April 27, 2026 09:17 View session

dangaiden requested a review from alecron April 27, 2026 09:21

Copilot AI reviewed Apr 27, 2026

View reviewed changes

tembleking changed the title ~~feat:monitor historical data~~ feat(k8s): support promql historical data queries Apr 27, 2026

tembleking reviewed Apr 27, 2026

View reviewed changes

Comment thread internal/infra/mcp/tools/utils.go

tembleking reviewed Apr 27, 2026

View reviewed changes

Comment thread internal/infra/mcp/tools/utils.go

dangaiden added 2 commits April 27, 2026 16:58

prevent PromQL range issues, add ApplyToParams to remove handler boil…

05346d3

…erplate, and add missing ParseTimeWindow unit tests

chore(tools): remove interval deprecation warnings and dead requestHa…

1d8dce6

…sArg helper

dangaiden requested a review from Copilot April 27, 2026 15:05

Copilot started reviewing on behalf of dangaiden April 27, 2026 15:05 View session

Copilot AI reviewed Apr 27, 2026

View reviewed changes

dangaiden added 2 commits April 27, 2026 17:17

fix: correct pre-existing typo in Examples function schema key

e4e7c00

add happy path test for ParseTimeWindow with valid start and end

36f4674

dangaiden requested a review from tembleking April 27, 2026 17:07

Clean up...

7548f2a

tembleking reviewed May 4, 2026

View reviewed changes

		// Reads "start" and "end" from the request, validates them, and return the resolved TimeWindow.

		func ParseTimeWindow(request mcp.CallToolRequest, clk clock.Clock) (TimeWindow, error) {

Conversation

dangaiden commented Apr 27, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

tembleking left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

dangaiden commented Apr 27, 2026

Uh oh!

tembleking May 4, 2026

Choose a reason for hiding this comment

Uh oh!

tembleking May 4, 2026

Choose a reason for hiding this comment

Uh oh!

tembleking May 4, 2026

Choose a reason for hiding this comment

Uh oh!

tembleking May 4, 2026

Choose a reason for hiding this comment

Uh oh!

tembleking May 4, 2026

Choose a reason for hiding this comment

Uh oh!

tembleking May 4, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants