Skip to content

test(licensing): make the outage-is-not-a-cancellation test actually test that - #78

Merged
Coding-Dev-Tools merged 1 commit into
mainfrom
fix/transport-failure-test-teeth
Jul 27, 2026
Merged

test(licensing): make the outage-is-not-a-cancellation test actually test that#78
Coding-Dev-Tools merged 1 commit into
mainfrom
fix/transport-failure-test-teeth

Conversation

@Coding-Dev-Tools

Copy link
Copy Markdown
Owner

The defect

test_a_transport_failure_is_not_mistaken_for_a_billing_denial raised:

cloud_session.CloudSessionError("...", status=503, transient=True)

CloudSessionError.__init__ accepts only message and statustransient belongs to CloudFeatureError. So the construction raised TypeError, _fetch_authoritative_entitlement's broad except Exception swallowed it, and getattr(exc, "status", None) returned None instead of 503.

The assertion passed while never exercising a transport failure at all. It was proving "a TypeError doesn't revoke access" — not "a 503 doesn't".

This is mine: I introduced it earlier in this session while adding the 402 billing-denial handling.

Why it matters

That test guards a revenue-visible behaviour: only a 402 may clear a paying customer's cached access; an outage must not. The gate is:

if getattr(exc, "status", None) == 402:
    record_billing_denial()
    _deny_entitlement_cache()

If that were ever loosened to if getattr(exc, "status", None):, a 503 would revoke a paying customer mid-outage. The test existed to catch exactly that — and could not, because TypeError carries no status attribute at all, so the mutation was invisible to it.

Verification

Mutation-tested both ways:

  • Gate loosened to any-truthy-status, after this fix → test fails: AssertionError: an outage revoked a paying customer, with cloud_access_active: False.
  • Same mutation, before this fix → test passed.

Full offline gate on this branch: 1,397 passed / 0 failed (1,411 collected, 14 skipped), ruff check . clean.

One-line change plus a comment recording why, so the conflation is not reintroduced.

🤖 Generated with Claude Code

…test that

The test raised CloudSessionError(..., status=503, transient=True), but that
class takes only status -- transient belongs to CloudFeatureError. So
the construction raised TypeError, the caller's broad except Exception
swallowed it, and getattr(exc, "status", None) was None rather than 503.

The assertion therefore passed while never exercising a transport failure at
all, and could not have caught the regression it exists to prevent: loosening
the 402 gate so any error clears a paying customer's access.

Verified by mutation -- with the gate changed to any-truthy-status the test now
fails with "an outage revoked a paying customer"; before this change it passed
even with that mutation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coding-Dev-Tools has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@Coding-Dev-Tools Coding-Dev-Tools left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — `test(licensing): make the outage-is-not-a-cancellation test actually test that`

LGTM

Good catch on the `CloudSessionError`/`CloudFeatureError` conflation. The mutation-test evidence is exactly the right proof shape: before the fix the test passed even with the gate loosened to any-truthy-status, which is the regression this assertion was meant to prevent. A test that cannot fail on the regression it guards against is worse than no test — it creates the appearance of coverage while providing none.

The comment recording why `transient` is wrong here is worth its weight: the next person to touch this will not make the same mistake, and the reason is specific enough to be actionable (`transient belongs to CloudFeatureError`, not a generic note).

CI green across 3.10/3.11/3.12 + core floor + browser smoke. +6/-1 is proportional to the fix.

One minor note for the future: consider a lint rule or type narrowing that flags `CloudSessionError(..., transient=...)` at static-analysis time rather than relying on runtime `TypeError` discovery. Not blocking — the comment and mutation test close the loop for now.

@Coding-Dev-Tools

Copy link
Copy Markdown
Owner Author

🔍 Pre-PR Code Review — ✅ LGTM (approve blocked: own PR)

Reviewer: Pre-PR Code Analyzer (automated)
Scope: +6/-1 in 1 file

Test Quality

Critical fix: the test was previously a no-op (TypeError from invalid transient=True swallowed by broad except). Now correctly raises CloudSessionError(status=503) and actually exercises the outage code path. Mutation testing confirms the test now catches the regression it was designed to prevent.

Verdict: LGTM — ready to merge

Minimal, surgical fix that transforms a false-assurance test into a real guard. CI green.

@Coding-Dev-Tools
Coding-Dev-Tools merged commit 73aa244 into main Jul 27, 2026
11 checks passed
@Coding-Dev-Tools
Coding-Dev-Tools deleted the fix/transport-failure-test-teeth branch July 27, 2026 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant