Problem
When many tests fail in the same project (batch test run --all, regression reruns, MCP execute), agents and humans see a flat list of 20–50 separate failures. In practice, those failures often share one underlying root cause:
- Authentication / token expired
- Staging environment down (
network_timeout, infra)
- Broken navigation (
routing_404)
- Shared code defect (same
recommendedFixTarget.reference)
- Corrupted test data / producer failure cascading to consumers
Today the CLI only supports per-test analysis (test failure get, test failure summary). An agent must download many bundles or guess which test to investigate first — wasting AI tokens, testing credits, and debug time.
Proposed solution (CLI Phase-0)
Add testsprite test failure triage --project <id>:
- List all failed tests (
GET /tests?status=failed)
- Fetch lightweight
failure/summary per test (no screenshots/video)
- Group client-side using deterministic heuristics over existing M2.1 fields:
- shared
recommendedFixTarget.reference
- env-wide
failureKind (infra, network, network_timeout, routing_404)
- normalized
rootCauseHypothesis prefix
- singleton fallback
- Return clusters with
representativeTestId, memberTestIds, confidence, fixPriority
Why CLI-first
- Uses only existing public APIs — no backend changes required
- Immediately reduces duplicate investigation and bundle downloads
- Natural read surface when native backend clustering ships later
Acceptance criteria
Future (backend)
Native clustering API with semantic embeddings, wave/cascade graph, and --rerun-representatives orchestration.
Problem
When many tests fail in the same project (batch
test run --all, regression reruns, MCP execute), agents and humans see a flat list of 20–50 separate failures. In practice, those failures often share one underlying root cause:network_timeout,infra)routing_404)recommendedFixTarget.reference)Today the CLI only supports per-test analysis (
test failure get,test failure summary). An agent must download many bundles or guess which test to investigate first — wasting AI tokens, testing credits, and debug time.Proposed solution (CLI Phase-0)
Add
testsprite test failure triage --project <id>:GET /tests?status=failed)failure/summaryper test (no screenshots/video)recommendedFixTarget.referencefailureKind(infra,network,network_timeout,routing_404)rootCauseHypothesisprefixrepresentativeTestId,memberTestIds,confidence,fixPriorityWhy CLI-first
Acceptance criteria
testsprite test failure triage --project <id> --output jsonreturns clustered output--type,--filter,--max-concurrencysupported--dry-runreturns canned sampleFuture (backend)
Native clustering API with semantic embeddings, wave/cascade graph, and
--rerun-representativesorchestration.