|
| 1 | +--- |
| 2 | +description: Analyze test failures and suggest fixes |
| 3 | +--- |
| 4 | + |
| 5 | +# Debug Test Failures |
| 6 | + |
| 7 | +Analyze test failures and suggest fixes. |
| 8 | + |
| 9 | +## Instructions |
| 10 | + |
| 11 | +Follow these steps to analyze the test failure: |
| 12 | + |
| 13 | +1. **Extract file path from traceback**: The error shows the failing file path (e.g., `testsuite/tests/.../test_basic.py:39`) |
| 14 | + |
| 15 | +2. **Read test file and trace fixtures**: |
| 16 | + - Read the test file at the path from the traceback |
| 17 | + - Look at fixtures (function parameters) and imports that the test uses |
| 18 | + - Read `conftest.py` in same directory, then parent directories if needed |
| 19 | + - Understand what the test expects vs what's actually happening |
| 20 | + |
| 21 | +3. **Categorize the failure**: |
| 22 | + - Identify the pattern (see `Common Failure Patterns` below) |
| 23 | + - Determine if this is a test/code issue or an environment/cluster state issue |
| 24 | + |
| 25 | +4. **Identify file authors**: |
| 26 | + - For the failing test file and relevant conftest.py files, use this command to find recent contributors: |
| 27 | + ```bash |
| 28 | + git log --format='%an' <file> | sort | uniq -c | sort -rn | head -5 |
| 29 | + ``` |
| 30 | + - Include these authors in the output so users know who to contact for deeper context if needed |
| 31 | + |
| 32 | +5. **Provide appropriate response**: |
| 33 | + - **If investigation needed**: Suggest kubectl commands to check policy status, conditions, events, logs, and verify resources exist and are ready |
| 34 | + - **IMPORTANT**: kubectl commands often require resources still in the cluster. Since tests cleanup after finishing, instruct to: |
| 35 | + - Set a breakpoint (outline where to set this) |
| 36 | + - Re-run the failing test locally |
| 37 | + - **If cluster/environment issue**: Provide kubectl/cleanup commands to fix cluster state |
| 38 | + - **If test/code issue**: Suggest code changes to fixtures or tests |
| 39 | + - **Mixed approach**: Combine multiple approaches as needed (e.g., investigate first, then apply fixes based on what you find) |
| 40 | + |
| 41 | +## Common Failure Patterns |
| 42 | + |
| 43 | +- **Policy not enforced**: Expected 302/401/429 but got 200 (policy not attached, wrong route/gateway, wrong host, or enforcement not yet propagated) |
| 44 | +- **Policy targeting**: `is_affected_by()` returns `False` (wrong `targetRef`, `parentRef`, namespace, labels, or resource mismatch) |
| 45 | +- **Reconciliation / propagation delay**: Async reconciliation not finished, config/policy not pushed, `observedGeneration` not catching up, race conditions |
| 46 | +- **DNS resolution / propagation**: `NXDOMAIN`, stale records (external DNS delay or caching, record deletion not fully propagated) |
| 47 | +- **DNS provider / credentials**: `DNSProviderError` (missing/misnamed/wrong-namespace secret, provider throttling or rate limits) |
| 48 | +- **Dependency missing or not found**: Policy `Accepted=False`, `reason=MissingDependency`, or selector found no objects (`selected 0`) |
| 49 | + (required operator/backend not installed, wrong namespace, or wrong labels, e.g., Authorino, Limitador) |
| 50 | +- **Component not ready**: Dependency installed but not `Ready` (slow rollout or readiness probe delays) |
| 51 | +- **Connectivity / request timeouts**: Gateway or Listener not ready, LoadBalancer provisioning delay, DNS resolves but traffic does not route, client `httpx` connect/read timeouts |
| 52 | +- **Wrong endpoint hit**: Test traffic bypasses the intended Gateway or Route (wrong URL, scheme, port, or hostname) |
| 53 | +- **Stale state / cleanup issues**: Leftover resources from previous tests (policies, routes, DNS records, namespaces not fully deleted) |
| 54 | +- **Invalid or rejected configuration**: Policy `Accepted=False` with reasons like `Invalid` or `TargetNotFound` (bad references, missing targets, wrong issuer/secret) |
| 55 | +- **Status / model mismatch**: `MissingModelBranch` (testsuite expects status/conditions not present, CRD or controller output drift) |
| 56 | +
|
| 57 | +## Output Format |
| 58 | +
|
| 59 | +**Issue**: [Summary of the problem] |
| 60 | +
|
| 61 | +**Root cause**: [Summary of why it's failing, or "needs more investigation" if unclear] |
| 62 | +
|
| 63 | +**Recommended steps**: [One or more steps - can be investigation, code changes, environment fixes, or a mix] |
| 64 | +
|
| 65 | +**File authors (Contact these contributors if you need deeper context beyond Claude's analysis)**: |
| 66 | +
|
| 67 | +- `<file_path>`: Author Name (N commits), Another Author (M commits), ... |
| 68 | +
|
| 69 | +_NOTE:_ Do NOT auto-run bash/kubectl commands or web searches before providing direct analysis and recommendations. |
0 commit comments