Skip to content

RTECO-1017 - Detect agent + CI invocation context, enrich metrics wit…#1555

Open
fluxxBot wants to merge 4 commits into
masterfrom
RTECO-1017-CIdetect
Open

RTECO-1017 - Detect agent + CI invocation context, enrich metrics wit…#1555
fluxxBot wants to merge 4 commits into
masterfrom
RTECO-1017-CIdetect

Conversation

@fluxxBot
Copy link
Copy Markdown
Contributor

@fluxxBot fluxxBot commented May 4, 2026

…h agent/is_agent/is_interactive

  • All tests passed. If this feature is not already covered by the tests, I added new tests.
  • All static analysis checks passed.
  • This pull request is on the master branch.
  • I used gofmt for formatting the code before submitting the pull request.

Summary

Adds detection of CLI invocation context — AI agent (Claude, Cursor, Gemini, Goose, Copilot, Kilocode, Roo Code, Codex) and interactive-terminal presence — and
enriches the existing jfcli_commands_count metric and outbound User-Agent header with the new dimensions. Also extends the existing CI provider list with
Harness.

Motivation

RTECO-1017: differentiate human / CI / agent invocations of jf. Backend currently can't tell whether a
command came from a developer's desktop, a CI runner, or an AI agent. Without this, we can't measure agent-driven usage or separate human vs automated traffic.

Changes

  • New common/commands/execution_context.go — invocation-context detection.
    • ExecutionContext struct: Agent, IsAgent, IsInteractive, TraceID.
    • DetectExecutionContext() populates the struct from env vars + terminal probe.
    • EnrichUserAgent(base string) string — returns the base User-Agent appended with (agent; ci=<system>) context. Used by jfrog-cli's SetUserAgent call.
    • Reuses clientlog.IsStdOutTerminal() from jfrog-client-go/utils/log for cached + test-overridable TTY detection. No direct dependency on
      golang.org/x/term.
  • New common/commands/execution_context_test.go — table-driven tests sourced from agentEnvDetectors (single source of truth), plus trace-ID gating,
    no-env fallback, and EnrichUserAgent combinations.
  • Extended utils/metrics/metrics.goMetricsData gains IsAgent, Agent, IsInteractive (additive, omitempty).
  • Updated common/commands/metrics_collector.go:
    • CollectMetrics populates the new fields via DetectExecutionContext.
    • Detection runs outside the mutex; lock now scopes only the map write.
    • Added HARNESS_BUILD_IDharness to the CI provider list.

Agent detection

First match wins. Generic AGENT env collapses to the literal unknown to keep label cardinality bounded.

Agent Env vars
claude CLAUDECODE, CLAUDE_CODE_ENTRYPOINT
gemini GEMINI_CLI
goose GOOSE_TERMINAL
cursor CURSOR_AGENT, CURSOR_CLI, CURSOR_TRACE_ID
copilot COPILOT_CLI
kilocode KILO_IPC_SOCKET_PATH, KILO_SERVER_PASSWORD
roo_code ROO_CODE_IPC_SOCKET_PATH
codex CODEX_CI

Dropped from the initial draft (no verified per-invocation subprocess marker): windsurf, aider, replit.

Trace-ID propagation

TraceID is gated on agent identity — only Cursor's CURSOR_TRACE_ID is honored, and only when the detected agent is cursor. Prevents stale values leaked
from an outer shell (e.g. a Cursor env present while the real invoker is Claude Code) from being reused.

User-Agent enrichment

EnrichUserAgent(base) appends (agent), (ci=<system>), or (agent; ci=<system>) to the base. Returns the base unchanged when neither is detected. The
jfrog-cli main.go change (separate PR) calls this once at startup.

@fluxxBot fluxxBot added the new feature Automatically generated release notes label May 4, 2026
Comment thread common/commands/execution_context.go Outdated
Comment thread common/commands/execution_context.go Outdated
Comment thread common/commands/execution_context.go Outdated
Comment thread common/commands/execution_context_test.go Outdated
Comment thread common/commands/metrics_collector.go
Comment thread common/commands/execution_context.go Outdated
Comment thread common/commands/execution_context.go Outdated
Comment thread common/commands/execution_context.go Outdated
Comment thread common/commands/execution_context.go
{"goose", []string{"GOOSE_TERMINAL"}},
{"cursor", []string{"CURSOR_AGENT", "CURSOR_CLI", "CURSOR_TRACE_ID"}},
{"copilot", []string{"COPILOT_CLI"}},
{"kilocode", []string{"KILO_IPC_SOCKET_PATH", "KILO_SERVER_PASSWORD"}},
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also in description table I see replit, windsurf, and aider entries but here I am not seeing them implemented

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah needed to drop it, no official documentation regarding any env from them

Comment thread common/commands/execution_context.go
… memoization for execution context to prevent divergence when environment variables change. Introduced resetExecutionContextForTest helper to facilitate isolated tests. Updated relevant test cases to utilize the new helper for consistent state management.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new feature Automatically generated release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants