Skip to content

Commit ced8f66

Browse files
authored
feat: add /debug-test slash command for analyzing test failures (#855)
Signed-off-by: emmaaroche <eroche@redhat.com>
1 parent 1396503 commit ced8f66

1 file changed

Lines changed: 69 additions & 0 deletions

File tree

.claude/commands/debug-test.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
---
2+
description: Analyze test failures and suggest fixes
3+
---
4+
5+
# Debug Test Failures
6+
7+
Analyze test failures and suggest fixes.
8+
9+
## Instructions
10+
11+
Follow these steps to analyze the test failure:
12+
13+
1. **Extract file path from traceback**: The error shows the failing file path (e.g., `testsuite/tests/.../test_basic.py:39`)
14+
15+
2. **Read test file and trace fixtures**:
16+
- Read the test file at the path from the traceback
17+
- Look at fixtures (function parameters) and imports that the test uses
18+
- Read `conftest.py` in same directory, then parent directories if needed
19+
- Understand what the test expects vs what's actually happening
20+
21+
3. **Categorize the failure**:
22+
- Identify the pattern (see `Common Failure Patterns` below)
23+
- Determine if this is a test/code issue or an environment/cluster state issue
24+
25+
4. **Identify file authors**:
26+
- For the failing test file and relevant conftest.py files, use this command to find recent contributors:
27+
```bash
28+
git log --format='%an' <file> | sort | uniq -c | sort -rn | head -5
29+
```
30+
- Include these authors in the output so users know who to contact for deeper context if needed
31+
32+
5. **Provide appropriate response**:
33+
- **If investigation needed**: Suggest kubectl commands to check policy status, conditions, events, logs, and verify resources exist and are ready
34+
- **IMPORTANT**: kubectl commands often require resources still in the cluster. Since tests cleanup after finishing, instruct to:
35+
- Set a breakpoint (outline where to set this)
36+
- Re-run the failing test locally
37+
- **If cluster/environment issue**: Provide kubectl/cleanup commands to fix cluster state
38+
- **If test/code issue**: Suggest code changes to fixtures or tests
39+
- **Mixed approach**: Combine multiple approaches as needed (e.g., investigate first, then apply fixes based on what you find)
40+
41+
## Common Failure Patterns
42+
43+
- **Policy not enforced**: Expected 302/401/429 but got 200 (policy not attached, wrong route/gateway, wrong host, or enforcement not yet propagated)
44+
- **Policy targeting**: `is_affected_by()` returns `False` (wrong `targetRef`, `parentRef`, namespace, labels, or resource mismatch)
45+
- **Reconciliation / propagation delay**: Async reconciliation not finished, config/policy not pushed, `observedGeneration` not catching up, race conditions
46+
- **DNS resolution / propagation**: `NXDOMAIN`, stale records (external DNS delay or caching, record deletion not fully propagated)
47+
- **DNS provider / credentials**: `DNSProviderError` (missing/misnamed/wrong-namespace secret, provider throttling or rate limits)
48+
- **Dependency missing or not found**: Policy `Accepted=False`, `reason=MissingDependency`, or selector found no objects (`selected 0`)
49+
(required operator/backend not installed, wrong namespace, or wrong labels, e.g., Authorino, Limitador)
50+
- **Component not ready**: Dependency installed but not `Ready` (slow rollout or readiness probe delays)
51+
- **Connectivity / request timeouts**: Gateway or Listener not ready, LoadBalancer provisioning delay, DNS resolves but traffic does not route, client `httpx` connect/read timeouts
52+
- **Wrong endpoint hit**: Test traffic bypasses the intended Gateway or Route (wrong URL, scheme, port, or hostname)
53+
- **Stale state / cleanup issues**: Leftover resources from previous tests (policies, routes, DNS records, namespaces not fully deleted)
54+
- **Invalid or rejected configuration**: Policy `Accepted=False` with reasons like `Invalid` or `TargetNotFound` (bad references, missing targets, wrong issuer/secret)
55+
- **Status / model mismatch**: `MissingModelBranch` (testsuite expects status/conditions not present, CRD or controller output drift)
56+
57+
## Output Format
58+
59+
**Issue**: [Summary of the problem]
60+
61+
**Root cause**: [Summary of why it's failing, or "needs more investigation" if unclear]
62+
63+
**Recommended steps**: [One or more steps - can be investigation, code changes, environment fixes, or a mix]
64+
65+
**File authors (Contact these contributors if you need deeper context beyond Claude's analysis)**:
66+
67+
- `<file_path>`: Author Name (N commits), Another Author (M commits), ...
68+
69+
_NOTE:_ Do NOT auto-run bash/kubectl commands or web searches before providing direct analysis and recommendations.

0 commit comments

Comments
 (0)