Skip to content

compute: reject a negative record count in accumulable reduce - #38010

Open
def- wants to merge 1 commit into
MaterializeInc:mainfrom
def-:compute-negative-record-count
Open

compute: reject a negative record count in accumulable reduce#38010
def- wants to merge 1 commit into
MaterializeInc:mainfrom
def-:compute-negative-record-count

Conversation

@def-

@def- def- commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

AccumulableErrorCheck accumulates a record count per key, and finalize_accum returns that count directly for count(*). The check only ever compared the count against zero: one arm caught a net-zero count paired with a non-zero accumulation, and another caught negative accumulation for unsigned sums. A NEGATIVE count passed both, so it reached finalize_accum unexamined and surfaced as a user-visible negative number with no error and no log line. A materialized view over such an aggregate then persisted the nonsense value with a positive multiplicity, making it durable in a derived shard and indistinguishable from a real result.

A well formed collection has non-negative multiplicities, so a negative count means the input is already corrupt. Report it as an error instead. The check is per key rather than per aggregate, because the record count is a property of the input records, not of any one aggregate.

This is defense in depth, not a root cause fix: it only fires once something upstream has already driven a collection negative. Its value is turning silently wrong answers into a diagnosable error, which is exactly what has been missing when invalid retractions show up in production.

Note the check is inherently per key. A global aggregate accumulates every record under one empty key, so an unmatched retraction there is absorbed into a positive total and stays invisible. Only grouped aggregates are covered.

@def-
def- requested a review from a team as a code owner August 3, 2026 07:53
@def-
def- requested a review from petrosagg August 3, 2026 07:54
`AccumulableErrorCheck` accumulates a record count per key, and
`finalize_accum` returns that count directly for `count(*)`. The check
only ever compared the count against zero: one arm caught a net-zero
count paired with a non-zero accumulation, and another caught negative
accumulation for unsigned sums. A NEGATIVE count passed both, so it
reached `finalize_accum` unexamined and surfaced as a user-visible
negative number with no error and no log line. A materialized view over
such an aggregate then persisted the nonsense value with a positive
multiplicity, making it durable in a derived shard and indistinguishable
from a real result.

A well formed collection has non-negative multiplicities, so a negative
count means the input is already corrupt. Report it as an error instead.
The check is per key rather than per aggregate, because the record count
is a property of the input records, not of any one aggregate.

This is defense in depth, not a root cause fix: it only fires once
something upstream has already driven a collection negative. Its value is
turning silently wrong answers into a diagnosable error, which is exactly
what has been missing when invalid retractions show up in production.

Note the check is inherently per key. A global aggregate accumulates
every record under one empty key, so an unmatched retraction there is
absorbed into a positive total and stays invisible. Only grouped
aggregates are covered.

Extends test/testdrive/avro-cdcv2.td with a regression case that injects
an unmatched retraction through `ENVELOPE MATERIALIZE`, the only source
envelope that passes upstream diffs through verbatim, and asserts both
that a grouped count errors and that the error propagates into a
materialized view over it. A positive control covers real retractions, so
the new arm cannot fire on well formed input.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@def-
def- force-pushed the compute-negative-record-count branch from c2450cd to 69dba7f Compare August 3, 2026 08:37
@def-
def- requested a review from a team as a code owner August 3, 2026 08:37
@frankmcsherry

Copy link
Copy Markdown
Contributor

Fwiw, worth taking some care here as this is also the mechanism that we use to diagnose flawed data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants