Regression
A freshly initialized workspace with a valid Foundry project endpoint cannot start the local Cockpit.
Reproduction:
- Install the Cockpit extra from release v0.13.0.
- Run
agentops init.
- Configure a valid Foundry project endpoint and agent target.
- Run
agentops cockpit.
Preflight succeeds, but application creation crashes with:
ValueError: missing required Observe hosted configuration:
AGENTOPS_TENANT_ID, AGENTOPS_APPLICATION_CLIENT_ID, AGENTOPS_UAMI_CLIENT_ID
Root cause
create_app(..., mode="local") uses WorkspaceProjectResolver to auto-discover an Observe project scope after init writes AZURE_AI_FOUNDRY_PROJECT_ENDPOINT.
When a project is discovered, the local branch calls create_observe_facade(). That factory is explicitly designed for hosted Observe and requires:
- an Entra application client ID;
- a user-assigned managed identity client ID;
- hosted Easy Auth / OBO identity configuration.
Those requirements are valid for the hosted multi-user Cockpit but not for a developer running agentops cockpit locally with Azure CLI / DefaultAzureCredential authentication.
The existing tests miss the production path: basic local tests have no discoverable project, while local scoped tests inject a fake observe_service. No test exercises automatic workspace scope discovery followed by construction of the real local Observe service.
Expected behavior
agentops cockpit must start locally after a normal agentops init with a valid Foundry project endpoint.
- Local mode must never require
AGENTOPS_APPLICATION_CLIENT_ID, AGENTOPS_UAMI_CLIENT_ID, Easy Auth headers, or hosted OBO configuration.
- Hosted mode must retain the current fail-fast validation for its required Entra application and UAMI settings.
- Local Observe should authenticate through the existing developer credential convention, using
DefaultAzureCredential(process_timeout=30) / Azure CLI credentials.
- Project scope auto-discovery should remain available locally.
- Local queries should use aggregate developer identity access. User-delegated attribution views that require an authenticated hosted end user must be disabled or reported as unavailable locally rather than emulated with fabricated identity.
- Missing SDKs, insufficient RBAC, unavailable Azure CLI credentials, or discovery/query failures must produce actionable diagnostics in Cockpit without crashing application startup.
- Do not ask users to define fake hosted identity environment variables as a workaround.
Design direction
Introduce a local Observe construction path instead of reusing the hosted factory unchanged. For example:
create_local_observe_facade(scope, credential=...) for aggregate discovery/query access; and
- the existing hosted factory for Easy Auth, OBO, UAMI, and delegated attribution.
Shared discovery, query, cache, cost, and scope behavior should remain reusable, but authentication requirements must be mode-specific.
Acceptance criteria
- The exact
agentops init → agentops cockpit reproduction starts successfully with no hosted identity variables.
- Local automatic project-scope discovery creates a working local Observe service.
- Local mode uses Azure CLI /
DefaultAzureCredential rather than ManagedIdentityCredential tied to AGENTOPS_UAMI_CLIENT_ID.
- Local mode does not construct an OBO credential or require an Entra application client ID.
- Hosted mode still requires and validates tenant, application client, UAMI client, and Easy Auth configuration.
- Aggregate Observe views remain available locally when the developer has read access.
- Delegated user/department views are explicitly unavailable locally when they require hosted user context; the UI explains why without crashing.
- Azure authentication, RBAC, SDK, discovery, and query failures degrade to diagnostics instead of preventing Cockpit startup.
- No credentials, access tokens, or identity claims are written to logs or responses.
- A regression test uses a workspace whose endpoint resolves to a project scope, does not inject
observe_service, leaves all hosted identity environment variables absent, and proves local app creation succeeds.
- Mode tests prove local startup does not consume hosted identity settings and hosted startup still fails fast when they are missing.
- Documentation distinguishes local developer authentication from hosted Easy Auth/OBO/UAMI requirements.
Regression
A freshly initialized workspace with a valid Foundry project endpoint cannot start the local Cockpit.
Reproduction:
agentops init.agentops cockpit.Preflight succeeds, but application creation crashes with:
Root cause
create_app(..., mode="local")usesWorkspaceProjectResolverto auto-discover an Observe project scope afterinitwritesAZURE_AI_FOUNDRY_PROJECT_ENDPOINT.When a project is discovered, the local branch calls
create_observe_facade(). That factory is explicitly designed for hosted Observe and requires:Those requirements are valid for the hosted multi-user Cockpit but not for a developer running
agentops cockpitlocally with Azure CLI /DefaultAzureCredentialauthentication.The existing tests miss the production path: basic local tests have no discoverable project, while local scoped tests inject a fake
observe_service. No test exercises automatic workspace scope discovery followed by construction of the real local Observe service.Expected behavior
agentops cockpitmust start locally after a normalagentops initwith a valid Foundry project endpoint.AGENTOPS_APPLICATION_CLIENT_ID,AGENTOPS_UAMI_CLIENT_ID, Easy Auth headers, or hosted OBO configuration.DefaultAzureCredential(process_timeout=30)/ Azure CLI credentials.Design direction
Introduce a local Observe construction path instead of reusing the hosted factory unchanged. For example:
create_local_observe_facade(scope, credential=...)for aggregate discovery/query access; andShared discovery, query, cache, cost, and scope behavior should remain reusable, but authentication requirements must be mode-specific.
Acceptance criteria
agentops init→agentops cockpitreproduction starts successfully with no hosted identity variables.DefaultAzureCredentialrather thanManagedIdentityCredentialtied toAGENTOPS_UAMI_CLIENT_ID.observe_service, leaves all hosted identity environment variables absent, and proves local app creation succeeds.