Fix the remaining latent bugs found by the type checker - #760
Open
wbarnha wants to merge 6 commits into
Open
Conversation
`SSLCredentials.__init__` defaulted `purpose` to None and passed it straight
into `ssl.create_default_context(purpose=...)`, which begins
if not isinstance(purpose, _ASN1Object):
raise TypeError(purpose)
So `SSLCredentials()` and `SSLCredentials(cafile=...)` -- any call that does not
supply an explicit `context` -- raised `TypeError: None`. The class could only
ever be constructed by handing it a context built elsewhere, which defeats the
cafile/capath/cadata parameters entirely.
Default to `ssl.Purpose.SERVER_AUTH`: the same default `create_default_context()`
itself applies, and the correct one for a client verifying a broker. It implies
`check_hostname=True` and `verify_mode=CERT_REQUIRED`, which the tests assert.
An explicitly passed `purpose` is still forwarded unchanged.
Found by the type checker in #758 and marked `XXX` there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012N2yysiNbzzVcvYhtrpsM7
`VersionInfo` declares `major`, `minor` and `micro` as ints, but the module
splatted the regex groups `(prefix, version, suffix)` into them positionally:
VersionInfo(major=None, minor='0.11.5', micro='')
So `faust.version_info.major` was the `'v'` prefix or None, `.minor` was the
whole version string, `.micro` was the suffix, and `.releaselevel` was always
None -- every field wrong except by accident.
Parse the dotted numbers properly instead, putting any non-numeric tail
(`dev1+g1234`, `rc1`, a local segment) into `releaselevel`. Missing components
pad with zero, and an unparsable version degrades to `VersionInfo(0, 0, 0, ...)`
rather than raising, so `import faust` can never fail on the version string --
the old `RuntimeError('THIS IS A BROKEN RELEASE!')` branch is gone with it.
This changes a public value, which is why #758 left it marked `XXX` rather than
fixing it: code reading `faust.version_info.minor` as the version *string* must
switch to `faust.__version__`, which is unchanged. Nothing in the repo consumes
it and no docs reference it.
`_parse_version` is injected into the lazy module's `__dict__` so it is
reachable for testing; it is private and stays out of `__all__` and `dir()`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012N2yysiNbzzVcvYhtrpsM7
`Collection._del_old_keys` read `_partition_timestamp_keys` with
self._partition_timestamp_keys.get((partition, window_range))
where `window_range` is the whole `(start, end)` tuple. The map is keyed
`(partition, range_end)` -- an `(int, float)` pair -- written that way by
`_maybe_set_key_ttl` and read that way by `_maybe_del_key_ttl`.
So the lookup could never hit. `triggered_windows` was always `[None, ...]`,
`window_data` stayed empty, and `on_window_close` was only ever handed the raw
per-key value instead of the aggregated window data it exists to receive.
Use `(partition, window_range[1])`, matching the writer.
This is user-visible: applications with an `on_window_close` handler will start
receiving the aggregated data the API always promised. That is why #758 marked
it `XXX` instead of fixing it.
Two existing tests relied on `mock_ranges` returning bare floats, which is not
what `_window_ranges` yields; they now pass real `(start, end)` tuples. Their
assertions are unchanged and the ranges still match nothing in
`_partition_timestamp_keys`, so they continue to cover the untriggered path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012N2yysiNbzzVcvYhtrpsM7
Two bugs in `Recovery`, both found by the type checker in #758. `_slurp_changelogs` classifies each event's TP as active or standby and binds `table`, `offsets` and `bufsize` accordingly. The `else:` branch for a TP that is neither only logged `"recovery unknown topic"` and fell through -- so the event was applied using the *previous* iteration's bindings: written into an unrelated table's buffer and offset map, and passed to that table's `on_changelog_event`. On the first event of the loop there is nothing bound yet, so it raised `UnboundLocalError` instead. Skip applying an event for an untracked TP. Note a bare `continue` would be wrong: the statements at the bottom of the loop body -- `_maybe_signal_recovery_end()` and the standby-ready bookkeeping -- must keep running on every iteration, or recovery-end signalling loses a trigger. Only the event-application block is skipped. `detect_aborted_tx` compared `await self.app.consumer.position(tp) >= highwater` unguarded. `ConsumerT.position` is `Optional[int]` and does return None when a partition has no position yet, so that raised TypeError -- swallowed by the caller's `except Exception`, which then silently skipped the aborted-transaction fixup for every remaining partition in the loop. Skip a TP with no position. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012N2yysiNbzzVcvYhtrpsM7
`_app_from_str` returns None for a `require_app = False` command invoked
without `-A` -- that is the documented escape hatch, and `faust completion` is
exactly such a command. `_finalize_app` handed that None straight back, and
`AppCommand.__init__` then did
self.key_serializer = key_serializer or self.app.conf.key_serializer
unconditionally, so the command died with
`AttributeError: 'NoneType' object has no attribute 'conf'`. The escape hatch
was unusable: `faust completion` could not run without the `-A` it is written
not to need.
Make `AppCommand` tolerate having no app. `self.app` becomes a property over
an `Optional[AppT]`, the serializer defaults fall back to None when there is no
app, and `on_stop` and `blocking_timeout` no longer assume one. Behaviour with
an app present is unchanged, and a command with `require_app = True` still gets
the same `UsageError` from `_app_from_str` as before.
Found by the type checker in #758 and marked `XXX` there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012N2yysiNbzzVcvYhtrpsM7
`Producer.key_partition` called `self._producer_thread.producer.list_topics()`. `ProducerThread.producer` is the *Faust* Producer; the confluent_kafka handle is `ProducerThread._producer`. Faust producers have no `list_topics`, so the method raised AttributeError and was dead on arrival. Read the confluent producer instead. `Consumer.verify_event_path` delegated to `self._thread.verify_event_path(...)`, but neither `ConsumerThread` nor `ConfluentConsumerThread` defines it, so the commit-livelock detector (`_commit_livelock_detector` -> `verify_all_partitions_active`) raised AttributeError on every tick. Add the no-op to `ConfluentConsumerThread`, matching the documented no-op stub the base `faust.transport.consumer.Consumer.verify_event_path` already is. This makes livelock detection inert for this driver rather than raising -- a real implementation is separate work. Both were found by the type checker in #758 and marked `XXX` there. Note these tests do not run here or in CI: tests/unit/transport/drivers/test_confluent.py starts with `pytest.importorskip("confluent_kafka")`, and confluent-kafka is the optional `faust[ckafka]` extra, which the CI test environment does not install. The fixes are verified by reading the class definitions, not by an executed test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012N2yysiNbzzVcvYhtrpsM7
wbarnha
force-pushed
the
claude/faust-latent-bug-fixes
branch
from
August 6, 2026 20:08
14601cb to
f3b2097
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Six bugs across five modules, all surfaced by the type checker in #758 and left marked
XXXthere because fixing them changes runtime behaviour. One commit per bug, each with a regression test.What was broken
auth.pySSLCredentials()raisedTypeError— could not build a context at all__init__.pyfaust.version_info.majorwas'v'; every field was wrongtables/base.pyon_window_closenever received the aggregated window datatables/recovery.pycli/base.pyrequire_app = Falsecommands crashed —faust completionunusable without-Atransport/drivers/confluent.pykey_partitiondead on arrival; livelock detector raised every tickSSLCredentialscould not be constructedpurposedefaulted toNoneand went straight intossl.create_default_context(purpose=...), which startsif not isinstance(purpose, _ASN1Object): raise TypeError(purpose). Any call that did not pass an explicitcontextraised — which defeats thecafile/capath/cadataparameters entirely. Now defaults tossl.Purpose.SERVER_AUTH, the same defaultcreate_default_context()itself uses and the right one for a client verifying a broker; the tests assert the resulting context hascheck_hostname=Trueandverify_mode=CERT_REQUIRED.faust.version_infoheld strings in its int fieldsThe regex groups
(prefix, version, suffix)were splatted positionally intoVersionInfo(major, minor, micro, ...), givingVersionInfo(major=None, minor='0.11.5', micro=''). Now parsed properly, with any non-numeric tail (dev1+g1234,rc1) going toreleaselevel. An unparsable version degrades toVersionInfo(0, 0, 0, ...)instead of raising, soimport faustcan no longer fail on it.This changes a public value. Code reading
faust.version_info.minoras the version string must usefaust.__version__, which is unchanged. Nothing in the repo consumes it and no docs reference it.on_window_closenever got its window data_del_old_keysread_partition_timestamp_keyswith the whole(start, end)tuple as the second key element; the map is keyed onrange_endalone, written that way by_maybe_set_key_ttland read that way by_maybe_del_key_ttl. The lookup could never hit, sotriggered_windowswas always[None, ...]andon_window_closeonly ever saw the raw per-key value.User-visible: applications with an
on_window_closehandler will start receiving the aggregated data the API always promised.Recovery applied events to whichever table came last
_slurp_changelogsbindstable/offsets/bufsizeper TP. Theelse:branch for an untracked TP only logged a warning and fell through, so the event was applied using the previous iteration's bindings — or raisedUnboundLocalErrorif it was the first event. Now skipped. Worth noting a barecontinuewould be wrong:_maybe_signal_recovery_end()and the standby bookkeeping at the bottom of the loop must still run every iteration, so only the application block is skipped.Separately,
detect_aborted_txcomparedawait consumer.position(tp) >= highwaterunguarded.positionisOptional[int]and does return None, raisingTypeError— swallowed by the caller'sexcept Exception, which then skipped the aborted-transaction fixup for every remaining partition.require_app = Falsewas unusable_app_from_strreturns None for such a command invoked without-A, thenAppCommand.__init__didkey_serializer or self.app.conf.key_serializerunconditionally.self.appis now a property over anOptional[AppT], withon_stopandblocking_timeoutno longer assuming an app. Behaviour with an app present is unchanged, andrequire_app = Truestill raises the sameUsageError.confluent driver
key_partitionreached forlist_topicsonProducerThread.producer— the Faust producer — rather than._producer, the confluent handle. AndConsumer.verify_event_pathdelegated to a method no thread class defines, so the livelock detector raised every tick; it now has the same documented no-op the baseConsumer.verify_event_pathalready is. A real livelock implementation is separate work.Verification
mypy -p faustclean,scripts/checkclean, suites 2207 → 2234 passed, 4 skipped.Every new test was checked against the pre-fix code by overlaying the test files onto a pristine tree built from
git archive. All fail there except the two noted below.Two honest caveats:
tests/unit/transport/drivers/test_confluent.pyopens withpytest.importorskip("confluent_kafka"), and confluent-kafka is the optionalfaust[ckafka]extra that the CI test environment does not install. The tests are written and will run where the extra is present; here they are skipped. Those two fixes rest on reading the class definitions.tests/unit/cli/test_base.py::test_init__serializers_without_apppasses against pre-fix code too — both serializers it passes are truthy, so the old code short-circuits before dereferencing the None app. It documents the case but is not a guard;test_init__no_app_when_not_requiredis the one that actually fails pre-fix.No existing test was weakened or deleted. The only removed test lines are an unused import, three
mock_rangescalls replaced with explicit(start, end)tuples (the mock returned bare floats, which is not what_window_rangesyields), and two mock-setup lines that pointed at the very attribute the confluent fix corrects.🤖 Generated with Claude Code