opentelemetry-instrumentation-kafka-python: keep the producer span open across the send() call - #4935
Open
Yigtwxx wants to merge 2 commits into
Open
Conversation
The producer span ended before the wrapped KafkaProducer.send() was invoked, so an exception raised synchronously by send() happened after the span had closed. The exported span reported StatusCode.UNSET with no exception event, hiding the failure from traces. Move the call inside the span's context manager, matching what the confluent-kafka instrumentation already does. Header injection, the produce hook and the span attributes are unaffected. Assisted-by: Claude Opus 5
iamroylim
approved these changes
Aug 8, 2026
iamroylim
left a comment
iamroylim
left a comment
There was a problem hiding this comment.
Verified this at head 6735b24 on Python 3.14.5 with uvx tox -e py314-test-instrumentation-kafka-python-latest: 16 tests passed. The change keeps KafkaProducer.send() inside start_as_current_span, so synchronous exceptions exit through the span context manager and produce ERROR status plus an exception event, while the successful return path remains unchanged. The Kafka future-completion path is deliberately out of scope. No blocking issue found.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
_wrap_sendclosed the producer span before calling the wrappedKafkaProducer.send(), so an exception raised synchronously bysend()(connection failure, serialization error) happened after the span had already ended. The exported span reportedStatusCode.UNSETwith no exception event, so a failed produce looked successful in traces.This moves
return func(*args, **kwargs)inside the span's context manager, which is what theconfluent-kafkainstrumentation already does. Header injection, the produce hook and the span attributes are unchanged.Scoped to the producer only. #4871 also reports a consumer-side gap, but that part carries an open question to maintainers about whether receive failures should produce an errored span, so it is left out here and the issue stays open.
Related to #4871
Type of change
How Has This Been Tested?
Two regression tests added to
tests/test_utils.py, using the real SDK (TestBasewith the in-memory exporter) rather than a mock tracer, since a mock tracer cannot observe span status:test_wrap_send_records_exception_raised_by_send— the exception propagates unmodified and the exported span carriesStatusCode.ERRORplus anexceptionevent. This test fails onmain(StatusCode.UNSET is not StatusCode.ERROR) and passes with the fix.test_wrap_send_leaves_successful_send_unchanged— span name, kind, status and the return value are unchanged on the success path.The four existing
_wrap_send/_wrap_nexttests pass unmodified.tox -e py311-test-instrumentation-kafka-python-latest(9 passed)tox -e py311-test-instrumentation-kafka-python-oldest(9 passed)tox -e lint-instrumentation-kafka-python,pre-commit run ruffDoes This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.