Skip to content

Stop local Cockpit from requiring hosted Observe identity configuration #458

Description

Regression

A freshly initialized workspace with a valid Foundry project endpoint cannot start the local Cockpit.

Reproduction:

  1. Install the Cockpit extra from release v0.13.0.
  2. Run agentops init.
  3. Configure a valid Foundry project endpoint and agent target.
  4. Run agentops cockpit.

Preflight succeeds, but application creation crashes with:

ValueError: missing required Observe hosted configuration:
AGENTOPS_TENANT_ID, AGENTOPS_APPLICATION_CLIENT_ID, AGENTOPS_UAMI_CLIENT_ID

Root cause

create_app(..., mode="local") uses WorkspaceProjectResolver to auto-discover an Observe project scope after init writes AZURE_AI_FOUNDRY_PROJECT_ENDPOINT.

When a project is discovered, the local branch calls create_observe_facade(). That factory is explicitly designed for hosted Observe and requires:

  • an Entra application client ID;
  • a user-assigned managed identity client ID;
  • hosted Easy Auth / OBO identity configuration.

Those requirements are valid for the hosted multi-user Cockpit but not for a developer running agentops cockpit locally with Azure CLI / DefaultAzureCredential authentication.

The existing tests miss the production path: basic local tests have no discoverable project, while local scoped tests inject a fake observe_service. No test exercises automatic workspace scope discovery followed by construction of the real local Observe service.

Expected behavior

  • agentops cockpit must start locally after a normal agentops init with a valid Foundry project endpoint.
  • Local mode must never require AGENTOPS_APPLICATION_CLIENT_ID, AGENTOPS_UAMI_CLIENT_ID, Easy Auth headers, or hosted OBO configuration.
  • Hosted mode must retain the current fail-fast validation for its required Entra application and UAMI settings.
  • Local Observe should authenticate through the existing developer credential convention, using DefaultAzureCredential(process_timeout=30) / Azure CLI credentials.
  • Project scope auto-discovery should remain available locally.
  • Local queries should use aggregate developer identity access. User-delegated attribution views that require an authenticated hosted end user must be disabled or reported as unavailable locally rather than emulated with fabricated identity.
  • Missing SDKs, insufficient RBAC, unavailable Azure CLI credentials, or discovery/query failures must produce actionable diagnostics in Cockpit without crashing application startup.
  • Do not ask users to define fake hosted identity environment variables as a workaround.

Design direction

Introduce a local Observe construction path instead of reusing the hosted factory unchanged. For example:

  • create_local_observe_facade(scope, credential=...) for aggregate discovery/query access; and
  • the existing hosted factory for Easy Auth, OBO, UAMI, and delegated attribution.

Shared discovery, query, cache, cost, and scope behavior should remain reusable, but authentication requirements must be mode-specific.

Acceptance criteria

  • The exact agentops initagentops cockpit reproduction starts successfully with no hosted identity variables.
  • Local automatic project-scope discovery creates a working local Observe service.
  • Local mode uses Azure CLI / DefaultAzureCredential rather than ManagedIdentityCredential tied to AGENTOPS_UAMI_CLIENT_ID.
  • Local mode does not construct an OBO credential or require an Entra application client ID.
  • Hosted mode still requires and validates tenant, application client, UAMI client, and Easy Auth configuration.
  • Aggregate Observe views remain available locally when the developer has read access.
  • Delegated user/department views are explicitly unavailable locally when they require hosted user context; the UI explains why without crashing.
  • Azure authentication, RBAC, SDK, discovery, and query failures degrade to diagnostics instead of preventing Cockpit startup.
  • No credentials, access tokens, or identity claims are written to logs or responses.
  • A regression test uses a workspace whose endpoint resolves to a project scope, does not inject observe_service, leaves all hosted identity environment variables absent, and proves local app creation succeeds.
  • Mode tests prove local startup does not consume hosted identity settings and hosted startup still fails fast when they are missing.
  • Documentation distinguishes local developer authentication from hosted Easy Auth/OBO/UAMI requirements.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions