diff --git a/.github/workflows/sync-docs.yml b/.github/workflows/sync-docs.yml new file mode 100644 index 0000000..51de2fa --- /dev/null +++ b/.github/workflows/sync-docs.yml @@ -0,0 +1,41 @@ +name: Synchronize docs to the Dash0 website + +on: + workflow_dispatch: + inputs: + dry-run: + description: Only verify transformations + coverage; do not open a PR. + type: boolean + default: true + workflow_call: + inputs: + dry-run: + type: boolean + default: false + pull_request: + paths: + - 'docs/**' + - '.github/workflows/sync-docs/**' + - '.github/workflows/sync-docs.yml' + +jobs: + sync-docs: + name: Sync docs to Dash0 website + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@v4 + + - name: Sync docs to the Dash0 website + uses: dash0hq/sync-docs-action@3fbe55fed800da21ee24d6b6dba1a86435f61565 # v0.4.0 + with: + dry-run: ${{ github.event_name == 'pull_request' || inputs.dry-run }} + target-repository: ${{ secrets.SYNC_DOCUMENTATION_TARGET_REPOSITORY }} + target-directory: ${{ secrets.SYNC_DOCUMENTATION_TARGET_DIRECTORY }} + target-github-token: ${{ secrets.DASH0_WEBSITE_SYNC_TOKEN }} + pr-branch: sync/opentelemetry-python-distribution-docs + pr-title: "docs: synchronize Python distribution documentation" + pr-body: | + Synchronizes the OpenTelemetry Python distribution documentation + from [dash0hq/opentelemetry-python-distribution](https://github.com/dash0hq/opentelemetry-python-distribution). + pr-reviewers: mmanciop diff --git a/.github/workflows/sync-docs/transformations.yaml b/.github/workflows/sync-docs/transformations.yaml new file mode 100644 index 0000000..98bd739 --- /dev/null +++ b/.github/workflows/sync-docs/transformations.yaml @@ -0,0 +1,62 @@ +common: + - description: Strip the leading H1 heading (the frontmatter title replaces it on the website). + type: replace-regex + find: '^# [^\n]*\n' + replace: "" + + - description: Rewrite relative cross-references between distribution docs to absolute website paths. + type: replace-regex + find: '\]\(([a-z][a-z0-9-]*)\)' + replace: '](/docs/dash0/opentelemetry/distributions/python/\1)' + required: false + +files: + - source: docs/overview.md + target: dash0/opentelemetry/distributions/python/overview.md + title: Dash0 OpenTelemetry Python Distribution + description: Overview of the Dash0 OpenTelemetry distribution for Python — zero-code instrumentation with pure-Python OTLP export and no native dependencies. + + - source: docs/getting-started.md + target: dash0/opentelemetry/distributions/python/getting-started.md + title: Getting Started + description: Install the Dash0 OpenTelemetry Python distribution and send your first traces, metrics, and logs in minutes. + + - source: docs/configuration.md + target: dash0/opentelemetry/distributions/python/configuration.md + title: Configuration + description: All Dash0-specific and standard OpenTelemetry environment variables for the Python distribution. + + - source: docs/auto-instrumentation.md + target: dash0/opentelemetry/distributions/python/auto-instrumentation.md + title: Auto-instrumentation + description: The full set of libraries and frameworks automatically instrumented by the Dash0 OpenTelemetry Python distribution. + + - source: docs/resource-detection.md + target: dash0/opentelemetry/distributions/python/resource-detection.md + title: Resource Detection + description: Built-in resource detectors for distribution metadata, Kubernetes pod UID, and service name fallback. + + - source: docs/kubernetes-injection.md + target: dash0/opentelemetry/distributions/python/kubernetes-injection.md + title: Kubernetes Injection + description: How the Dash0 Python distribution is injected into Kubernetes workloads via sitecustomize.py and PYTHONPATH, with safety guards and graceful self-deactivation. + + - source: docs/exporters.md + target: dash0/opentelemetry/distributions/python/exporters.md + title: OTLP Exporters + description: Pure-Python OTLP/HTTP and OTLP/gRPC exporters with no native dependencies, intelligent endpoint rewriting, and drop-in compatibility with the upstream exporter names. + +coverage: + include: + - "docs/*.md" + - "docs/**/*.md" + ignore: + - "docs/brainstorms/2026-07-22-distro-publishing-requirements.md" + - "docs/plans/2026-07-22-001-feat-self-hosted-index-publishing-plan.md" + +nav: + target: dash0/opentelemetry/distributions/python/nav.json + order: 66 + id: opentelemetry-python-distribution + parentPath: Dash0 Distributions + title: Python diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..46c4b32 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,95 @@ +# CLAUDE.md + +Coding conventions and agentic guidelines for the Dash0 OpenTelemetry Python distribution. + +## Project overview + +A `uv` workspace. The Dash0-authored code lives entirely in +`packages/dash0-opentelemetry/`. The vendored pyproto exporter packages +(`packages/opentelemetry-*/`) are kept close to upstream; do not apply local +style changes to them (they are excluded from linting). + +## Coding conventions + +- Every function has a name that precisely describes what it does. +- Prefer `from x import y` over `import x`. +- Keep the distribution vendor-neutral except where Dash0 behavior is intentional. + +## Maintaining the website documentation + +The `docs/` directory contains Markdown files that are synced to the Dash0 +website via `.github/workflows/sync-docs.yml`. Keep them accurate whenever +the relevant source changes. + +### Source-of-truth mapping + +Each doc file has a single authoritative source. When those sources change, +update the corresponding doc. + +| Doc file | Authoritative source(s) | What to check | +|---|---|---| +| `docs/overview.md` | `README.rst`, `packages/dash0-opentelemetry/README.rst` | Overall description, key capabilities, architecture | +| `docs/getting-started.md` | `README.rst` (Quick start section), `examples/dash0-distro-flask/` | Quick start commands, example setup | +| `docs/configuration.md` | `packages/dash0-opentelemetry/src/dash0/opentelemetry/_environment_variables.py`, `packages/dash0-opentelemetry/src/dash0/opentelemetry/distro.py`, `packages/dash0-opentelemetry/src/dash0/opentelemetry/settings.py` | All `DASH0_*` variables and the `OTEL_*` defaults the distro sets | +| `docs/auto-instrumentation.md` | `packages/dash0-opentelemetry/pyproject.toml` (the `[project].dependencies` block, approx. lines 50–115) | The exact list of `opentelemetry-instrumentation-*` packages | +| `docs/resource-detection.md` | `packages/dash0-opentelemetry/src/dash0/opentelemetry/resource.py`, `packages/dash0-opentelemetry/README.rst` | Detector names, attributes emitted, detection logic | +| `docs/kubernetes-injection.md` | `packages/dash0-opentelemetry/src/dash0/opentelemetry/injector/sitecustomize.py` | Safety checks, deactivation logic, PYTHONPATH bridge | +| `docs/exporters.md` | `packages/dash0-opentelemetry/src/dash0/opentelemetry/distro.py`, `packages/opentelemetry-exporter-otlp-proto-http/`, `packages/opentelemetry-exporter-otlp-proto-grpc/` | Exporter names, protocol selection, port-rewriting logic | + +### When to update docs + +Update the relevant doc file whenever a PR: + +- Adds, renames, or removes a `DASH0_*` environment variable → `docs/configuration.md` +- Adds or removes an `opentelemetry-instrumentation-*` dependency → `docs/auto-instrumentation.md` +- Changes how a resource detector works or what attributes it emits → `docs/resource-detection.md` +- Changes the `sitecustomize.py` safety checks or deactivation flow → `docs/kubernetes-injection.md` +- Changes the exporter selection logic, port-rewriting, or protocol handling → `docs/exporters.md` +- Changes the overall architecture, entry points, or quick-start workflow → `docs/overview.md`, `docs/getting-started.md` + +### Auto-instrumentation list: always derive from source + +**Never maintain `docs/auto-instrumentation.md` by hand.** Always derive the +library table from the actual dependency block in +`packages/dash0-opentelemetry/pyproject.toml`. + +To regenerate the list: + +```bash +grep 'opentelemetry-instrumentation-' packages/dash0-opentelemetry/pyproject.toml \ + | sed 's/.*"\(opentelemetry-instrumentation-[a-z0-9_-]*\).*/\1/' +``` + +Map each package name to its target library using the +[opentelemetry-python-contrib instrumentation index](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation) +and update the tables in `docs/auto-instrumentation.md` to match. +Keep the same category groupings (Web frameworks, HTTP clients, Databases, etc.). +Add new packages to the appropriate category; remove packages that are no longer in `pyproject.toml`. + +Also update the "What is not included" section whenever a package is deliberately +excluded (the comments in `pyproject.toml` explain the reason for each exclusion). + +### Keeping transformations.yaml in sync + +`.github/workflows/sync-docs/transformations.yaml` declares which `docs/*.md` +files are synced to the website. It has a `coverage` block that fails CI if a +file under `docs/*.md` is not explicitly listed. + +When you add a new `docs/*.md` file, add a corresponding entry to the `files:` +block in `transformations.yaml` with: +- `source:` — path relative to the repo root (e.g., `docs/new-topic.md`) +- `target:` — destination path on the website (follow the `opentelemetry/python/.md` pattern) +- `title:` — short display title for the page +- `description:` — one-sentence description used as the page meta description + +When you remove a `docs/*.md` file, remove the corresponding `files:` entry too. + +### Doc style rules + +- One sentence per line (semantic line breaks) — this is the project's RST convention carried into Markdown. +- Sentence-case headings: `## Getting started`, not `## Getting Started`. +- Active voice: "The distro sets" not "The value is set by". +- Inline code for all env var names, file paths, package names, and CLI commands. +- No em-dashes; use shorter sentences instead. +- Tables for environment variables: three columns (variable name, description, notes/default). +- Do not add comments explaining what the doc is or when it was updated. diff --git a/docs/auto-instrumentation.md b/docs/auto-instrumentation.md new file mode 100644 index 0000000..9ff9105 --- /dev/null +++ b/docs/auto-instrumentation.md @@ -0,0 +1,163 @@ +# Auto-instrumentation + +## How it works + +The distribution depends on the full set of upstream auto-instrumentation packages, all pinned to a single contrib version. +When `opentelemetry-instrument` starts your application, each installed instrumentor is activated automatically. +Instrumentors whose target library is not installed are silently skipped. +**No `opentelemetry-bootstrap` step is required.** + +## Defensive loading + +Each instrumentor is loaded defensively: if one fails (due to a version incompatibility, a missing dependency, or a bug), the failure is logged and the instrumentor is skipped. +Instrumentation of the rest of the process continues. +A single broken instrumentor does not abort the entire auto-instrumentation process. + +## Supported libraries and frameworks + +### Web frameworks + +
+ +| Library | Instrumentation package | +|---|---| +| Flask | [`opentelemetry-instrumentation-flask`](https://pypi.org/project/opentelemetry-instrumentation-flask/) | +| Django | [`opentelemetry-instrumentation-django`](https://pypi.org/project/opentelemetry-instrumentation-django/) | +| FastAPI | [`opentelemetry-instrumentation-fastapi`](https://pypi.org/project/opentelemetry-instrumentation-fastapi/) | +| Falcon | [`opentelemetry-instrumentation-falcon`](https://pypi.org/project/opentelemetry-instrumentation-falcon/) | +| Pyramid | [`opentelemetry-instrumentation-pyramid`](https://pypi.org/project/opentelemetry-instrumentation-pyramid/) | +| Starlette | [`opentelemetry-instrumentation-starlette`](https://pypi.org/project/opentelemetry-instrumentation-starlette/) | +| Tornado | [`opentelemetry-instrumentation-tornado`](https://pypi.org/project/opentelemetry-instrumentation-tornado/) | +| ASGI | [`opentelemetry-instrumentation-asgi`](https://pypi.org/project/opentelemetry-instrumentation-asgi/) | +| WSGI | [`opentelemetry-instrumentation-wsgi`](https://pypi.org/project/opentelemetry-instrumentation-wsgi/) | + +
+ +### HTTP clients + +
+ +| Library | Instrumentation package | +|---|---| +| aiohttp client | [`opentelemetry-instrumentation-aiohttp-client`](https://pypi.org/project/opentelemetry-instrumentation-aiohttp-client/) | +| aiohttp server | [`opentelemetry-instrumentation-aiohttp-server`](https://pypi.org/project/opentelemetry-instrumentation-aiohttp-server/) | +| httpx | [`opentelemetry-instrumentation-httpx`](https://pypi.org/project/opentelemetry-instrumentation-httpx/) | +| requests | [`opentelemetry-instrumentation-requests`](https://pypi.org/project/opentelemetry-instrumentation-requests/) | +| urllib | [`opentelemetry-instrumentation-urllib`](https://pypi.org/project/opentelemetry-instrumentation-urllib/) | +| urllib3 | [`opentelemetry-instrumentation-urllib3`](https://pypi.org/project/opentelemetry-instrumentation-urllib3/) | + +
+ +### Databases and caches + +
+ +| Library | Instrumentation package | +|---|---| +| psycopg2 (PostgreSQL) | [`opentelemetry-instrumentation-psycopg2`](https://pypi.org/project/opentelemetry-instrumentation-psycopg2/) | +| psycopg (PostgreSQL) | [`opentelemetry-instrumentation-psycopg`](https://pypi.org/project/opentelemetry-instrumentation-psycopg/) | +| asyncpg (async PostgreSQL) | [`opentelemetry-instrumentation-asyncpg`](https://pypi.org/project/opentelemetry-instrumentation-asyncpg/) | +| aiopg (async PostgreSQL) | [`opentelemetry-instrumentation-aiopg`](https://pypi.org/project/opentelemetry-instrumentation-aiopg/) | +| mysql-connector-python | [`opentelemetry-instrumentation-mysql`](https://pypi.org/project/opentelemetry-instrumentation-mysql/) | +| mysqlclient | [`opentelemetry-instrumentation-mysqlclient`](https://pypi.org/project/opentelemetry-instrumentation-mysqlclient/) | +| PyMySQL | [`opentelemetry-instrumentation-pymysql`](https://pypi.org/project/opentelemetry-instrumentation-pymysql/) | +| pymssql (SQL Server) | [`opentelemetry-instrumentation-pymssql`](https://pypi.org/project/opentelemetry-instrumentation-pymssql/) | +| SQLite3 | [`opentelemetry-instrumentation-sqlite3`](https://pypi.org/project/opentelemetry-instrumentation-sqlite3/) | +| PyMongo (MongoDB) | [`opentelemetry-instrumentation-pymongo`](https://pypi.org/project/opentelemetry-instrumentation-pymongo/) | +| Cassandra / ScyllaDB | [`opentelemetry-instrumentation-cassandra`](https://pypi.org/project/opentelemetry-instrumentation-cassandra/) | +| Redis | [`opentelemetry-instrumentation-redis`](https://pypi.org/project/opentelemetry-instrumentation-redis/) | +| pymemcache | [`opentelemetry-instrumentation-pymemcache`](https://pypi.org/project/opentelemetry-instrumentation-pymemcache/) | +| DB-API 2.0 | [`opentelemetry-instrumentation-dbapi`](https://pypi.org/project/opentelemetry-instrumentation-dbapi/) | + +
+ +### ORMs and query builders + +
+ +| Library | Instrumentation package | +|---|---| +| SQLAlchemy | [`opentelemetry-instrumentation-sqlalchemy`](https://pypi.org/project/opentelemetry-instrumentation-sqlalchemy/) | +| TortoiseORM | [`opentelemetry-instrumentation-tortoiseorm`](https://pypi.org/project/opentelemetry-instrumentation-tortoiseorm/) | + +
+ +### Message queues and task queues + +
+ +| Library | Instrumentation package | +|---|---| +| confluent-kafka | [`opentelemetry-instrumentation-confluent-kafka`](https://pypi.org/project/opentelemetry-instrumentation-confluent-kafka/) | +| kafka-python | [`opentelemetry-instrumentation-kafka-python`](https://pypi.org/project/opentelemetry-instrumentation-kafka-python/) | +| aiokafka | [`opentelemetry-instrumentation-aiokafka`](https://pypi.org/project/opentelemetry-instrumentation-aiokafka/) | +| aio-pika (RabbitMQ) | [`opentelemetry-instrumentation-aio-pika`](https://pypi.org/project/opentelemetry-instrumentation-aio-pika/) | +| pika (RabbitMQ) | [`opentelemetry-instrumentation-pika`](https://pypi.org/project/opentelemetry-instrumentation-pika/) | +| Celery | [`opentelemetry-instrumentation-celery`](https://pypi.org/project/opentelemetry-instrumentation-celery/) | +| Remoulade | [`opentelemetry-instrumentation-remoulade`](https://pypi.org/project/opentelemetry-instrumentation-remoulade/) | +| AWS SQS (boto3sqs) | [`opentelemetry-instrumentation-boto3sqs`](https://pypi.org/project/opentelemetry-instrumentation-boto3sqs/) | + +
+ +### Async and concurrency + +
+ +| Library | Instrumentation package | +|---|---| +| asyncio | [`opentelemetry-instrumentation-asyncio`](https://pypi.org/project/opentelemetry-instrumentation-asyncio/) | +| threading | [`opentelemetry-instrumentation-threading`](https://pypi.org/project/opentelemetry-instrumentation-threading/) | + +
+ +### CLI frameworks + +
+ +| Library | Instrumentation package | +|---|---| +| Click | [`opentelemetry-instrumentation-click`](https://pypi.org/project/opentelemetry-instrumentation-click/) | +| asyncclick | [`opentelemetry-instrumentation-asyncclick`](https://pypi.org/project/opentelemetry-instrumentation-asyncclick/) | + +
+ +### AI and cloud + +
+ +| Library | Instrumentation package | +|---|---| +| OpenAI | [`opentelemetry-instrumentation-openai-v2`](https://pypi.org/project/opentelemetry-instrumentation-openai-v2/) | +| Google Vertex AI | [`opentelemetry-instrumentation-vertexai`](https://pypi.org/project/opentelemetry-instrumentation-vertexai/) | +| boto3 / botocore (AWS SDK) | [`opentelemetry-instrumentation-botocore`](https://pypi.org/project/opentelemetry-instrumentation-botocore/) | + +
+ +### Other + +
+ +| Library | Instrumentation package | +|---|---| +| gRPC | [`opentelemetry-instrumentation-grpc`](https://pypi.org/project/opentelemetry-instrumentation-grpc/) | +| Jinja2 | [`opentelemetry-instrumentation-jinja2`](https://pypi.org/project/opentelemetry-instrumentation-jinja2/) | +| logging | [`opentelemetry-instrumentation-logging`](https://pypi.org/project/opentelemetry-instrumentation-logging/) | +| structlog | [`opentelemetry-instrumentation-structlog`](https://pypi.org/project/opentelemetry-instrumentation-structlog/) | +| System metrics | [`opentelemetry-instrumentation-system-metrics`](https://pypi.org/project/opentelemetry-instrumentation-system-metrics/) | +| Exception handling | [`opentelemetry-instrumentation-exceptions`](https://pypi.org/project/opentelemetry-instrumentation-exceptions/) | + +
+ +## Propagators + +The AWS X-Ray propagator ([`opentelemetry-propagator-aws-xray`](https://pypi.org/project/opentelemetry-propagator-aws-xray/)) is explicitly included. +The W3C TraceContext and Baggage propagators are built into the OpenTelemetry SDK and always available. +B3 format propagation is not included; it requires the separate [`opentelemetry-propagator-b3`](https://pypi.org/project/opentelemetry-propagator-b3/) package, which is not a dependency of this distribution. + +## What is not included + +The following packages are deliberately excluded: + +- [`opentelemetry-exporter-prometheus`](https://pypi.org/project/opentelemetry-exporter-prometheus/): not needed; OTLP metrics export covers this use case. +- [`opentelemetry-propagator-ot-trace`](https://pypi.org/project/opentelemetry-propagator-ot-trace/): legacy propagator not required for Dash0. +- [`opentelemetry-instrumentation-aws-lambda`](https://pypi.org/project/opentelemetry-instrumentation-aws-lambda/): Lambda deployments use a dedicated instrumentation layer. diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..f82657d --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,60 @@ +# Configuration + +## Dash0-specific environment variables + +| Variable | Effect | +|---|---| +| `DASH0_OTEL_COLLECTOR_BASE_URL` | **Required.** OTLP endpoint to export to. If unset, the distribution disables itself entirely and sets `OTEL_SDK_DISABLED=true`. | +| `DASH0_DISABLE` | Set to `true` to disable the distribution entirely. The SDK becomes a no-op. | +| `DASH0_AUTOMATIC_SERVICE_NAME` | Set to `false` to opt out of the entrypoint-derived `service.name` fallback. | +| `DASH0_BOOTSTRAP_SPAN` | Name of a single span to emit once at startup. Useful for confirming the distribution is active. | +| `DASH0_FLUSH_ON_SIGTERM_SIGINT` | Set to `true` to flush all telemetry (traces, metrics, logs) on `SIGTERM` or `SIGINT` before exiting. Signal handlers can only be installed from the main thread; if the distribution is configured from a non-main thread, this setting is silently ignored. | + +## Standard OpenTelemetry variables + +All standard `OTEL_*` variables are honored. +The distribution sets defaults using `setdefault` and never overrides values you provide. + +The following variables are relevant to the distribution's behavior: + +| Variable | Default set by distro | Notes | +|---|---|---| +| `OTEL_TRACES_EXPORTER` | `otlp_proto_http` | Pure-Python OTLP/HTTP exporter. | +| `OTEL_METRICS_EXPORTER` | `otlp_proto_http` | Pure-Python OTLP/HTTP exporter. | +| `OTEL_LOGS_EXPORTER` | `otlp_proto_http` | Pure-Python OTLP/HTTP exporter. | +| `OTEL_EXPORTER_OTLP_PROTOCOL` | `http/protobuf` | Set to `grpc` for OTLP/gRPC. | +| `OTEL_EXPORTER_OTLP_ENDPOINT` | Value of `DASH0_OTEL_COLLECTOR_BASE_URL` | Derived automatically; override if needed. | +| `OTEL_EXPORTER_OTLP_HEADERS` | — | Headers sent with every OTLP request. Required for Dash0 authentication: `Authorization=Bearer ,Dash0-Dataset=default`. | +| `OTEL_EXPORTER_OTLP__PROTOCOL` | — | Per-signal protocol override (`TRACES`, `METRICS`, `LOGS`). | +| `OTEL_EXPORTER_OTLP__ENDPOINT` | — | Per-signal endpoint override. Custom ports and explicitly set endpoints are never rewritten by the distro. | +| `OTEL_SERVICE_NAME` | — | Standard service name. The distro adds a fallback derived from the entrypoint when this is unset. | +| `OTEL_RESOURCE_ATTRIBUTES` | — | Additional resource attributes. The distro reads this to detect whether `service.name` is already set, and appends its own detected attributes (`telemetry.distro.*`, `k8s.pod.uid`) to it. | +| `OTEL_PYTHON_DISABLED_INSTRUMENTATIONS` | — | Comma-separated list of instrumentor entry-point names to skip (e.g. `flask,redis`). Honored by the auto-instrumentation loader. | +| `OTEL_EXPERIMENTAL_RESOURCE_DETECTORS` | — | Comma-separated resource detector names. Reference the Dash0 detectors here when using declarative SDK configuration (`OTEL_CONFIG_FILE`). | + +## Disabling the distribution + +If you need to disable the distribution entirely without removing it: + +```bash +export DASH0_DISABLE=true +opentelemetry-instrument python app.py +``` + +The SDK becomes a no-op and no instrumentation or export occurs. + +## Setting a service name + +The distribution uses the following priority order for `service.name`: + +1. `OTEL_SERVICE_NAME` (you set it explicitly) +2. `service.name` from `OTEL_RESOURCE_ATTRIBUTES` +3. Name derived from the entrypoint script basename (e.g., `app.py` → `app`) + +Steps 2 and 3 are suppressed when `DASH0_AUTOMATIC_SERVICE_NAME=false`. + +To disable the entrypoint fallback and keep `service.name` unset when you have not provided one: + +```bash +export DASH0_AUTOMATIC_SERVICE_NAME=false +``` diff --git a/docs/exporters.md b/docs/exporters.md new file mode 100644 index 0000000..0afe89b --- /dev/null +++ b/docs/exporters.md @@ -0,0 +1,68 @@ +# OTLP exporters + +## Why pure Python? + +The distribution ships its own OTLP exporters built on pure-Python protobuf message classes ([`dash0-opentelemetry-pyproto`](https://pypi.org/project/dash0-opentelemetry-pyproto/)). +These exporters have no dependency on `google.protobuf` or `grpcio`, which are C extension packages that require binary compatibility with the target application. + +Injecting a package with native dependencies into an arbitrary process risks ABI conflicts (different versions of the shared library) or version conflicts (the application already uses a different version of `google.protobuf`). +The pure-Python exporters eliminate that class of problem. + +## Available exporters + +### OTLP/HTTP (default) + +**Exporter name:** `otlp_proto_http` +**Package:** [`dash0-opentelemetry-exporter-otlp-pyproto-http`](https://pypi.org/project/dash0-opentelemetry-exporter-otlp-pyproto-http/) + +Selected by default for all three signals (traces, metrics, logs). +Sends protobuf-encoded OTLP over HTTP/1.1. +Compatible with any OTLP-capable collector, including the Dash0 ingress. + +### OTLP/gRPC + +**Exporter name:** `otlp_proto_grpc` +**Package:** [`dash0-opentelemetry-exporter-otlp-pyproto-grpc`](https://pypi.org/project/dash0-opentelemetry-exporter-otlp-pyproto-grpc/) + +Available as an alternative to OTLP/HTTP. +Uses the same pure-Python protobuf implementation. +Activate it by setting the protocol: + +```bash +export OTEL_EXPORTER_OTLP_PROTOCOL=grpc +``` + +Or for a specific signal only: + +```bash +export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=grpc +``` + +## Drop-in replacement for upstream exporters + +The pyproto exporters occupy the same import namespaces and register under the same entry-point names (`otlp_proto_http`, `otlp_proto_grpc`) as the upstream [`opentelemetry-exporter-otlp-proto-http`](https://pypi.org/project/opentelemetry-exporter-otlp-proto-http/) and [`opentelemetry-exporter-otlp-proto-grpc`](https://pypi.org/project/opentelemetry-exporter-otlp-proto-grpc/) packages. + +Because the distribution depends on the pyproto packages rather than the upstream ones, the standard exporter names resolve to the pure-Python implementations automatically. +No configuration change is required; existing `OTEL_EXPORTER_OTLP_*` settings continue to work. + +Once the upstream OpenTelemetry project publishes official pure-Python exporter releases, the distribution will depend on those and deprecate the vendored copies. + +## Protocol selection + +The protocol is selected per signal using the following priority order: + +1. `OTEL_EXPORTER_OTLP__PROTOCOL` (e.g., `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL`) +2. `OTEL_EXPORTER_OTLP_PROTOCOL` +3. Default: `http/protobuf` + +## Intelligent endpoint rewriting + +When a signal uses a different protocol than the shared base URL's conventional port implies, the distribution automatically derives the correct per-signal endpoint. + +**Example:** `DASH0_OTEL_COLLECTOR_BASE_URL=http://collector:4318` (the HTTP port) and `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=grpc`. +The distribution rewrites the traces endpoint to `http://collector:4317` (the gRPC port) so traces are sent to the right listener. + +Rules: +- If the base URL port is the gRPC default (4317) and a signal uses HTTP, the signal's endpoint is rewritten to port 4318 with the `/v1/` path appended. +- If the base URL port is the HTTP default (4318) and a signal uses gRPC, the signal's endpoint is rewritten to port 4317. +- Custom ports and explicitly set per-signal endpoints are never rewritten. diff --git a/docs/getting-started.md b/docs/getting-started.md new file mode 100644 index 0000000..b38b040 --- /dev/null +++ b/docs/getting-started.md @@ -0,0 +1,69 @@ +# Getting started + +## Installation + +Install the distribution from PyPI: + +```bash +pip install dash0-opentelemetry +``` + +This single package pulls in the full upstream auto-instrumentation set, the pure-Python OTLP exporters, and all resource detectors. +No additional `opentelemetry-bootstrap` step is required. + +## Minimal quick start + +Point the distribution at a collector and run your application under `opentelemetry-instrument`: + +```bash +export DASH0_OTEL_COLLECTOR_BASE_URL=http://localhost:4318 +opentelemetry-instrument python app.py +``` + +That is enough to auto-instrument all installed libraries and export traces, metrics, and logs over OTLP/HTTP. +Your application needs zero OpenTelemetry imports. + +## Sending directly to Dash0 + +To export directly to the Dash0 ingress instead of a local collector, use the Dash0 ingress URL and add an authorization header: + +```bash +export DASH0_OTEL_COLLECTOR_BASE_URL=https://ingress...dash0.com +export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $DASH0_AUTH_TOKEN,Dash0-Dataset=default" +opentelemetry-instrument python app.py +``` + +## Running the Flask example + +The repository ships a self-contained, Docker-based Flask demo in `examples/dash0-distro-flask`. +It demonstrates zero-code instrumentation, pure-Python OTLP/HTTP export, Kubernetes pod-UID detection, and `telemetry.distro.name` in resource attributes. + +```bash +cd examples/dash0-distro-flask +export SCRIPT_UID=$(id -u) SCRIPT_GID=$(id -g) +docker compose up +``` + +Exported telemetry is written to JSONL files in `output/` (traces, metrics, logs). + +To overlay Dash0 ingress instead of the local collector: + +```bash +export DASH0_ENDPOINT=https://ingress...dash0.com +export DASH0_AUTH_TOKEN=auth_xxxxxxxxxxxxxxxx +export DASH0_DATASET=default +docker compose -f docker-compose.yml -f docker-compose.dash0.yml up +``` + +## What happens at startup + +When `opentelemetry-instrument` runs: + +1. `Dash0Distro` is discovered via the `opentelemetry_distro` entry point. + It checks `DASH0_DISABLE` and `DASH0_OTEL_COLLECTOR_BASE_URL`. + If either gate fails, it sets `OTEL_SDK_DISABLED=true` and skips all instrumentation. +2. The distro selects the pure-Python OTLP/HTTP exporter, sets `OTEL_EXPORTER_OTLP_ENDPOINT`, and injects detected resource attributes into the environment. +3. Each installed instrumentor is loaded defensively: failures are logged and skipped. +4. `Dash0Configurator` is discovered via the `opentelemetry_configurator` entry point. + It delegates to the standard SDK configurator, then optionally emits a bootstrap span and installs signal handlers. +5. Your application starts. diff --git a/docs/kubernetes-injection.md b/docs/kubernetes-injection.md new file mode 100644 index 0000000..49b028d --- /dev/null +++ b/docs/kubernetes-injection.md @@ -0,0 +1,72 @@ +# Kubernetes injection + +The distribution is designed to be injected into running Python processes by the [Dash0 Operator for Kubernetes](https://www.dash0.com/docs/dash0/monitoring/kubernetes/dash0-operator/overview). +No application code changes are required. + +## How injection works + +The distribution wheel ships a `sitecustomize.py` at `dash0/opentelemetry/injector/sitecustomize.py`. +Python automatically executes any `sitecustomize.py` it finds on `PYTHONPATH` during interpreter startup, before the application's own code runs. + +The Dash0 Operator for Kubernetes injects the distribution directory into `PYTHONPATH`. +When the Python interpreter starts, it finds and executes `sitecustomize.py`, which activates the distribution. + +## Safety checks before instrumentation + +Before activating instrumentation, `sitecustomize.py` performs several guards. +If any guard fails, the distribution deactivates itself gracefully, including from child processes, rather than breaking the application. + +### Python version check + +The distribution requires Python 3.10 or later. +If the interpreter is older (including Python 2.x), the script deactivates itself and exits cleanly. + +The script itself is written in valid Python 2.7+ syntax (including a PEP-263 encoding declaration) so it can safely parse and self-deactivate even on very old interpreters without a syntax error. + +### Double-instrumentation check + +If another OpenTelemetry distribution or the upstream SDK is already active in the process (detected by checking for overlapping OpenTelemetry packages), the script deactivates itself to avoid double-instrumenting the application. + +### Dependency conflict check + +The script validates the application's installed packages against the distribution's pinned dependency manifest (`all-dependencies.txt`). +If a version conflict is detected that would prevent the distribution from loading correctly, it deactivates itself and logs the conflict. + +## Graceful self-deactivation + +When any safety check fails, the distribution: + +1. Sets `DASH0_DISABLE=true` in the current process environment. +2. Sets the same variable for child processes (so they also skip instrumentation). +3. Logs the reason for deactivation at `WARNING` level. +4. Exits `sitecustomize.py` without activating instrumentation. + +The application continues running normally, unmodified. + +## PYTHONPATH bridge + +When running via injection, the `sitecustomize.py` also bridges the environment: + +- If `OTEL_EXPORTER_OTLP_ENDPOINT` is not set but `DASH0_OTEL_COLLECTOR_BASE_URL` is set, the script copies the value to `OTEL_EXPORTER_OTLP_ENDPOINT` before the OpenTelemetry SDK initializes. + +This allows the operator to pass a single `DASH0_OTEL_COLLECTOR_BASE_URL` environment variable without needing to know which specific `OTEL_*` variables to set. + +## Kubernetes pod UID detection + +When running inside a Kubernetes pod, the `dash0_kubernetes` resource detector automatically extracts the pod UID from the cgroup filesystem and adds it as `k8s.pod.uid` to every span, metric, and log record. +See [Resource detection](resource-detection) for details. + +## Example: operator-style injection + +An operator injects the distribution by setting environment variables on the pod and mutating `PYTHONPATH`: + +```yaml +env: + - name: PYTHONPATH + value: /dash0/opentelemetry/injector:$(PYTHONPATH) + - name: DASH0_OTEL_COLLECTOR_BASE_URL + value: http://dash0-collector.dash0-system.svc.cluster.local:4318 +``` + +The application pod requires no changes. +When the Python interpreter starts, it picks up `sitecustomize.py` from the injected `PYTHONPATH` and activates the distribution. diff --git a/docs/overview.md b/docs/overview.md new file mode 100644 index 0000000..c479a45 --- /dev/null +++ b/docs/overview.md @@ -0,0 +1,40 @@ +# Dash0 OpenTelemetry Python Distribution + +The Dash0 distribution of OpenTelemetry for Python is a lightweight, opinionated layer on top of the upstream OpenTelemetry SDK that instruments Python applications with **no code changes** and exports telemetry using a **pure-Python protobuf implementation**. +No `google.protobuf` or `grpcio` native dependencies are required. + +## Why pure Python? + +Removing native dependencies is the core design goal. +It makes the distribution safe to *inject* into an arbitrary process via `PYTHONPATH`, for example from a Kubernetes operator or injector, without risking ABI or version conflicts with the target application. +If the distribution carried a native protobuf or gRPC dependency, injecting it into an application that already uses a different version of those libraries would break the application. +The pure-Python implementation avoids that class of problem entirely. + +This is the Python counterpart of the [Dash0 OpenTelemetry distribution for Node.js](https://github.com/dash0hq/opentelemetry-js-distribution), bringing the same injection-safe model to Python. + +## What you get + +**Zero-code instrumentation.** `Dash0Distro` and `Dash0Configurator` are discovered through standard OpenTelemetry entry points and wire up the SDK before your application starts. +No `import opentelemetry` anywhere in your application code. + +**The full contrib instrumentation set, built in.** Every upstream auto-instrumentation package ships as a pinned dependency. +No `opentelemetry-bootstrap` step is needed. +Instrumentations activate only when their target library is installed, so carrying all 40+ of them is safe. + +**Pure-Python OTLP export.** OTLP/HTTP is selected by default for all three signals (traces, metrics, logs). +OTLP/gRPC is available via `OTEL_EXPORTER_OTLP_PROTOCOL=grpc`. +Neither protocol requires native dependencies. + +**Sensible, injection-friendly defaults.** The distribution sets an enable/disable gate, requires a collector endpoint, detects the Kubernetes pod UID, derives a service name from the entrypoint, marks telemetry with `telemetry.distro.name`, and can flush telemetry gracefully on `SIGTERM`/`SIGINT`. + +**An injector-ready bootstrap script.** The wheel ships a `sitecustomize.py` that the OpenTelemetry injector activates via `PYTHONPATH`. +It includes interpreter-version, double-instrumentation, and dependency-conflict guards with graceful self-deactivation. + +**Defensive instrumentation loading.** A single failing instrumentor is logged and skipped rather than aborting instrumentation of the whole process. + +## Relationship to the upstream OpenTelemetry SDK + +The distribution is a thin layer on top of the upstream SDK. +It sets defaults, registers resource detectors, and selects the pure-Python exporters. +All standard `OTEL_*` environment variables work as documented by the upstream project. +The distribution only sets values with `setdefault` and never overrides values you provide. diff --git a/docs/resource-detection.md b/docs/resource-detection.md new file mode 100644 index 0000000..ecc9df8 --- /dev/null +++ b/docs/resource-detection.md @@ -0,0 +1,82 @@ +# Resource detection + +The distribution ships three resource detectors registered as standard OpenTelemetry entry points. +They are activated automatically when the distribution runs and can also be referenced explicitly via `OTEL_EXPERIMENTAL_RESOURCE_DETECTORS`. + +## Distribution detector (`dash0_distribution`) + +Adds metadata identifying the Dash0 distribution: + +| Attribute | Value | +|---|---| +| `telemetry.distro.name` | `dash0-python` | +| `telemetry.distro.version` | Current distribution version | + +This attribute appears on every span, metric, and log record exported by the distribution. +It lets you filter telemetry by the distribution version in Dash0. + +## Kubernetes detector (`dash0_kubernetes`) + +Detects the Kubernetes pod UID when running inside a Kubernetes pod: + +| Attribute | Value | +|---|---| +| `k8s.pod.uid` | UUID of the current pod | + +**Detection method.** The detector first confirms it is running inside a Kubernetes pod by inspecting `/etc/hosts`. +It then extracts the pod UID from the cgroup filesystem: + +- cgroup v1: reads `/proc/self/mountinfo` and extracts the pod UID from the kubepods hierarchy. +- cgroup v2: reads `/proc/self/cgroup` and parses the pod UID from the cgroup path. + +cgroup v1 is tried first; cgroup v2 is the fallback. + +Both standard cgroup path formats and Kubernetes slice naming conventions are handled. + +If the detector cannot confirm it is inside a Kubernetes pod, or cannot extract the pod UID, it returns an empty resource without failing. + +**Other `k8s.*` attributes** — such as `k8s.namespace.name`, `k8s.node.name`, `k8s.deployment.name`, and similar — cannot be read from the cgroup filesystem and are not set by this detector. +They require access to the Kubernetes API and should be added via the [OpenTelemetry Collector's `k8sattributesprocessor`](https://opentelemetry.io/docs/kubernetes/collector/components/#kubernetes-attributes-processor). +The [Dash0 Operator](https://www.dash0.com/changelog/automatic-kubernetes-resource-detection) handles this automatically for workloads it manages. +See the [OpenTelemetry Kubernetes attributes best practices](https://www.dash0.com/guides/opentelemetry-kubernetes-attributes-best-practices) guide for a full picture of how to enrich telemetry with Kubernetes metadata. + +## Service name detector (`dash0_service_name`) + +Sets `service.name` when it has not been provided explicitly: + +| Attribute | Value | +|---|---| +| `service.name` | Derived from `OTEL_SERVICE_NAME`, `OTEL_RESOURCE_ATTRIBUTES`, or entrypoint basename | +| `service.instance.id` | Stable per-process UUID | + +**Priority order for `service.name`:** + +1. `OTEL_SERVICE_NAME` environment variable. +2. `service.name` key in `OTEL_RESOURCE_ATTRIBUTES`. +3. Basename of the entrypoint script (e.g., `app.py` → `app`). + +The entrypoint fallback ensures that every process has a human-readable `service.name` even in injection scenarios where no explicit name has been configured. +To disable this fallback, set `DASH0_AUTOMATIC_SERVICE_NAME=false`. + +This detector also wraps the upstream `service` detector, so there is no need to list both `service` and `dash0_service_name` in `OTEL_EXPERIMENTAL_RESOURCE_DETECTORS`. + +## Using detectors explicitly + +**You normally do not need this.** +In the default execution path (`opentelemetry-instrument`), the distro injects `telemetry.distro.*`, `k8s.pod.uid`, and the service name fallback into `OTEL_RESOURCE_ATTRIBUTES` and `OTEL_SERVICE_NAME` before the SDK initializes. +The SDK's built-in `OTELResourceDetector` picks those up automatically. +No explicit detector configuration is required. + +**When you do need it: `OTEL_CONFIG_FILE`.** +The experimental declarative SDK configuration (`OTEL_CONFIG_FILE`) ignores `OTEL_RESOURCE_ATTRIBUTES` and `OTEL_SERVICE_NAME` by design. +In that case, reference the detectors explicitly in the config file, or via the environment variable: + +```bash +export OTEL_EXPERIMENTAL_RESOURCE_DETECTORS=dash0_distribution,dash0_kubernetes,dash0_service_name +``` + +You must list all three if you want all three — none of the Dash0 detectors are included by default in the declarative config path. + +**What `OTEL_EXPERIMENTAL_RESOURCE_DETECTORS` does to the default detectors.** +The SDK always appends `service_instance` and `otel` (the `OTELResourceDetector`) to whatever you list. +You do not need to add them yourself, and you cannot remove them by omitting them.