Skip to content

[fix] expose BatchSpanProcessor queue, exporter and sampler tuning in register() - #192

Open
JayaSurya-27 wants to merge 1 commit into
devfrom
fix/span-processor-tuning
Open

[fix] expose BatchSpanProcessor queue, exporter and sampler tuning in register()#192
JayaSurya-27 wants to merge 1 commit into
devfrom
fix/span-processor-tuning

Conversation

@JayaSurya-27

Copy link
Copy Markdown
Collaborator

What does this PR do?

Exposes span-processor / exporter / provider tuning through register() and the SDK processor classes: max_queue_size, schedule_delay_millis, max_export_batch_size, export_timeout_millis, timeout, sampler, span_limits, span_exporter. All default to None, falling back to the standard OTEL_* env vars / upstream defaults, so existing callers see zero behavior change.

Why?

Under high-throughput ingestion the OTel BatchSpanProcessor queue (default 2048) overflows and spans are silently dropped (Queue full, dropping Span). The SDK's BatchSpanProcessor documented tuning params but never accepted or forwarded them, and register() exposed none of them — so there was no supported Python-API way to prevent the drop.

Fixed TH-7478

Also fixed along the way:

  • register(batch=False) now warns when batch-only tuning params are passed (previously silently dropped; the warning also works on Python 3.10/3.11, not just 3.12+).
  • register(span_exporter=...) no longer crashes with non-OTLP exporters (config banner read _endpoint/_headers unguarded).
  • Removed docstring claims of an unsupported endpoint kwarg on TracerProvider/SimpleSpanProcessor/BatchSpanProcessor.
  • De-duplicated the default-exporter construction into _build_default_exporter.
  • Re-exported Sampler, ParentBased, TraceIdRatioBased, SpanLimits for usability.
  • Version bump 1.0.0 -> 1.1.0 + changelog entry.

Note: exporter compression was deliberately NOT exposed — the fi-collector does not decode gzip (HTTP or gRPC), so enabling it would break ingestion against self-hosted deployments. To be added together with collector gzip support.

How was it tested?

  • Unit tests added / updated (pytest) — 31 new tests in python/tests/test_batch_span_processor_tuning.py: kwarg/env/default precedence, forwarding to upstream, sampler effectiveness (ratio 0/1), endpoint= TypeError guards, gRPC timeout forwarding, batch=False warning, end-to-end emit, custom-exporter regression.
  • Integration tests pass (or N/A — no gateway behavior changed) — verified live against a self-hosted fi-collector: wire format unchanged (POST /tracer/v1/traces, application/x-protobuf, no Content-Encoding), plus a forced-drop demo (queue=10 drops, queue=100000 does not).
  • ruff check passes on changed files (only pre-existing findings remain)
  • Public types, env vars, or SDK behavior changes are documented in the relevant README — documented in register() docstring and python/CHANGELOG.md

Checklist

  • Commit messages follow the repo convention
  • No TODOs or commented-out code left in
  • No real API keys or secrets in the diff

Notes for reviewers

  • Backward compatibility was the hard constraint: with no new args, construction paths are byte-identical (upstream reads env vars only when a param is None).
  • timeout is seconds (exporter) while *_millis params are milliseconds (batch processor) — matches upstream OTel naming; called out in the register() docstring.
  • Pre-existing, untouched: tests/test_otel.py fails collection (stale SESSION_NAME import) and one env-dependent test_settings.py failure — both reproduce on dev without this change.

… register()

Previously documented tuning params were silently dropped, making span drops
under high throughput unavoidable via the Python API.
@JayaSurya-27 JayaSurya-27 self-assigned this Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant