Skip to content

Require a valid proof of work for a token, not just a good score - #29

Merged
cport1 merged 1 commit into
mainfrom
fix/pow-precondition
Aug 19, 2026
Merged

Require a valid proof of work for a token, not just a good score#29
cport1 merged 1 commit into
mainfrom
fix/pow-precondition

Conversation

@cport1

@cport1 cport1 commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

A bare curl was issued a valid token — ten times out of ten, on all three servers:

$ curl -X POST /api/verify -d '{"siteKey":"demo-site-key","signals":{}}'
tokens minted: 10 / 10

No browser, no proof of work, a curl User-Agent, empty signals. Every token passed siteverify. Reproduced against v1.21.0, so it predates the siteverify work.

Why the detectors did not stop it

They fired. A forged solution referencing a challenge that was never issued produced seven detections and still scored 0.4003, under the 0.5 success threshold.

calculateFinalScore is a weighted sum, so a category contributes at most its own weight however conclusive it is. bot is weighted 0.13; every PoW failure firing at once reaches 0.9986 under noisy-OR, which is 0.1298 of the final score — 0.37 short, permanently. dispositiveFloor's own doc comment diagnoses this exact failure, but the mark had only been applied to navigator.webdriver and the ChromeDriver/Puppeteer globals, never to the proof of work — whose own code comment reads "hard fail".

The fix

  • A proof of work is a precondition, checked outside the score: success = score < 0.5 && hostnameAllowed && powSatisfied. Gating here means no future reweighting can reopen it, and it holds even if the dispositive floor is lowered.
  • The three PoW failures are marked Dispositive (absent, unverifiable, nonce-unbound), so the reported score is honest too — 0.9 rather than 0.4.
  • Refusals name the failed precondition (pow_not_satisfied / hostname_not_allowed) via a reason field.

No effect on legitimate traffic

The widget already solves a challenge on every path and aborts rather than submit without one. Measured on the bench harness, which completes the real handshake:

before after
human FPR 0.00% (0/126) 0.00% (0/126)
human median score 0.097 0.097
agent TPR 97.33% 97.33%
curl bulk-mint 10/10 tokens 0/10 tokens

Bench gate exits 0.

Tests

E2E suite's eight "gets low score" assertions posted no PoW and so floored at 0.9; they now complete the real handshake via a helper that reuses the bench solver rather than duplicating the scheme. Detection-only assertions still skip it — whether a payload trips a detector does not depend on the handshake. 104 → 107 assertions, all passing on Node.

Go and Python regression tests cover a withheld token, the dispositive floor, a forged solution, the reported reason, and — pinning why the gate must exist — that the bot category alone cannot reach the threshold.

The 5 Go / 13 Python E2E failures are the documented pre-existing divergences, verified identical against the pre-change build.

Breaking

Anyone calling /api/verify directly from a custom client must now complete the handshake. There is no flag to restore the old behaviour — it was a bypass, not a feature. See Upgrading to 1.23.0.

…ood score

A bare `curl` was issued a valid token. Ten times out of ten, on all three
servers:

    $ curl -X POST /api/verify -d '{"siteKey":"demo-site-key","signals":{}}'
    tokens minted: 10 / 10

No browser, no proof of work, a curl User-Agent, empty signals. Every token
passed siteverify. Confirmed against v1.21.0 too, so this predates the
siteverify work rather than being introduced by it.

The detectors were never the problem — they all fired. A forged solution
referencing a challenge that was never issued produced seven detections
(`PoW verification failed`, `User-Agent indicates bot/automation tool`, `Zero
mouse, touch, or keyboard events recorded`, `No mouse movement detected before
click`, ...) and still scored 0.4003, under the 0.5 success threshold.

The aggregation discarded them. calculateFinalScore is a weighted sum, so a
category contributes at most its own weight no matter how conclusive it is. The
bot category is weighted 0.13; every PoW failure firing at once reaches 0.9986
under noisy-OR, which is 0.1298 of the final score, 0.37 short of the threshold
and always will be. dispositiveFloor's own doc comment diagnoses exactly this,
but the mark had only ever been applied to navigator.webdriver and the
ChromeDriver/Puppeteer globals — never to the proof of work, whose own code
comment reads "hard fail".

Two changes, deliberately both:

  - A proof of work is now a precondition, checked outside the score:
    `success = score < 0.5 && hostnameAllowed && powSatisfied`. A score
    threshold answers "how suspicious is this visitor", which is the wrong
    question to ask of someone who never completed the challenge. Gating here
    means no future reweighting can reopen this, and it holds even if the
    dispositive floor is lowered or removed.
  - The three PoW failures (absent, unverifiable, nonce-unbound) are marked
    Dispositive, so the reported score is honest too — an integrator
    risk-banding on the score now sees 0.9 rather than 0.4.

Refusals name the failed precondition (`pow_not_satisfied`,
`hostname_not_allowed`) via a new `reason` field, because a caller whose score is
comfortably under the threshold otherwise has no way to tell why no token came
back. Go's VerificationResult gained the field; Node and Python already reported
one for the hostname case.

No effect on legitimate traffic. The widget already solves a challenge on every
path and aborts rather than submit without one, so nothing about its flow
changes. Measured on the bench harness, which completes the real handshake:
human false-positive rate 0.00% (0/126), human median score unchanged at 0.097,
agent TPR 97.33%, gate exits 0.

The E2E suite posted synthetic signals with no PoW, so its eight "gets low score"
assertions floored at 0.9. Those now complete the real handshake via a
makeVerifiedRequest helper that reuses the bench solver rather than carrying a
second copy of the scheme. Detection-only assertions still use makeRequest: they
ask whether a payload trips a detector, which does not depend on the handshake.
104 -> 107 assertions, all passing on Node; the five Go and thirteen Python
failures are the documented pre-existing divergences, verified identical against
the pre-change build.

Regression tests in Go and Python cover a withheld token, the dispositive floor,
a forged solution, the reported reason, and — pinning why the gate must exist at
all — that the bot category alone cannot reach the threshold.
@cport1
cport1 merged commit 2b54bb7 into main Aug 19, 2026
6 checks passed
@cport1
cport1 deleted the fix/pow-precondition branch August 19, 2026 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant