Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ jobs:
with:
enable-cache: true
- name: Ruff lint
run: uvx ruff@latest check packages/dash0-opentelemetry-distro examples scripts
run: uvx ruff@latest check packages/dash0-opentelemetry examples scripts
- name: Ruff format check
run: uvx ruff@latest format --check packages/dash0-opentelemetry-distro examples scripts
run: uvx ruff@latest format --check packages/dash0-opentelemetry examples scripts
- name: Check uv.lock is up to date
run: uv lock --check
- name: Check exact dependency pins
Expand All @@ -44,9 +44,9 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Run distribution unit tests
run: >
uv run --package dash0-opentelemetry-distro
uv run --package dash0-opentelemetry
--python ${{ matrix.python-version }}
python -m pytest packages/dash0-opentelemetry-distro/tests -v
python -m pytest packages/dash0-opentelemetry/tests -v

sitecustomize-python27:
runs-on: ubuntu-latest
Expand Down
60 changes: 59 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,11 @@ jobs:
github.event_name == 'push' ||
(github.event_name == 'workflow_dispatch' && inputs.mode == 'rebuild-assets')
runs-on: ubuntu-latest
outputs:
# JSON array of the PyPI project names built this run; drives the
# per-package publish-pypi matrix below. Empty ("[]") when every
# filename is already indexed and nothing was built.
packages: ${{ steps.select.outputs.packages }}
permissions:
contents: write
id-token: write
Expand Down Expand Up @@ -117,7 +122,7 @@ jobs:
f"releases must carry release-form versions"
)

distro_version = versions["dash0-opentelemetry-distro"]
distro_version = versions["dash0-opentelemetry"]
assert tag == f"v{distro_version}", (
f"tag {tag} does not match the distro version: "
f"expected v{distro_version}"
Expand All @@ -136,6 +141,9 @@ jobs:
prerelease = bool(re.search(r"(?:a|b|rc)[0-9]+", distro_version))
with open(os.environ["GITHUB_OUTPUT"], "a") as out:
out.write(f"prerelease={str(prerelease).lower()}\n")
# The publish-pypi matrix iterates exactly the packages built
# this run, so already-published names are never re-published.
out.write(f"packages={json.dumps(to_build)}\n")
print(f"building: {to_build or 'nothing (all filenames already indexed)'}")
EOF
- name: Build packages
Expand Down Expand Up @@ -275,6 +283,56 @@ jobs:
- uses: actions/deploy-pages@v5
id: deploy

publish-pypi:
needs: [build]
# Publish to PyPI only on tag-triggered runs (not manual dispatches or
# UI-published events) and only when the build job produced artifacts.
if: >-
github.event_name == 'push' && needs.build.result == 'success' &&
needs.build.outputs.packages != '[]'
# One job per package built this run. Each package is published from its
# OWN GitHub environment whose name equals the package's PyPI project
# name. This is load-bearing, not cosmetic: a PyPI *pending* trusted
# publisher is unique on (owner, repo, workflow, environment), so a single
# shared environment could authorize only one of these names. A distinct
# environment per package gives each its own (…, release.yml, <name>)
# tuple, which is what lets this one workflow publish all of them. Each
# environment must exist with a matching pending/trusted publisher on PyPI
# (see RELEASING.rst).
strategy:
fail-fast: false
matrix:
package: ${{ fromJSON(needs.build.outputs.packages) }}
runs-on: ubuntu-latest
environment:
name: ${{ matrix.package }}
url: https://pypi.org/project/${{ matrix.package }}/
permissions:
id-token: write
env:
TAG: ${{ github.ref_name }}
GH_TOKEN: ${{ github.token }}
PACKAGE: ${{ matrix.package }}
steps:
- name: Download this package's release assets
run: |
set -euo pipefail
mkdir dist
# Wheel/sdist filenames normalize runs of -_. to a single _, so the
# trailing hyphen before the version makes each prefix unambiguous:
# "dash0_opentelemetry-*" matches the distro but not
# "dash0_opentelemetry_pyproto-*".
dist_name="${PACKAGE//-/_}"
gh release download "$TAG" \
--repo "$GITHUB_REPOSITORY" \
--pattern "${dist_name}-*.whl" \
--pattern "${dist_name}-*.tar.gz" \
--dir dist/
ls -l dist/
- uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
with:
packages-dir: dist/

index-dry-run:
if: github.event_name == 'workflow_dispatch' && inputs.mode == 'dry-run'
runs-on: ubuntu-latest
Expand Down
12 changes: 6 additions & 6 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Prerequisites
Project layout
==============

- ``packages/dash0-opentelemetry-distro`` — the distribution: ``Dash0Distro``
- ``packages/dash0-opentelemetry`` — the distribution: ``Dash0Distro``
and ``Dash0Configurator`` plus resource detection and settings. **This is the
Dash0-authored code.**
- ``packages/opentelemetry-*`` — the pure-Python (pyproto) OTLP exporter
Expand All @@ -34,7 +34,7 @@ Setup

.. code-block:: bash

uv sync --package dash0-opentelemetry-distro
uv sync --package dash0-opentelemetry

This creates ``.venv`` with the distribution, the in-repo pyproto exporters,
and the dev dependencies.
Expand All @@ -47,8 +47,8 @@ Distribution unit tests (fast, no Docker):

.. code-block:: bash

uv run --package dash0-opentelemetry-distro \
python -m pytest packages/dash0-opentelemetry-distro/tests -v
uv run --package dash0-opentelemetry \
python -m pytest packages/dash0-opentelemetry/tests -v

End-to-end example (requires Docker; builds the distro into a container, runs a
Flask app under ``opentelemetry-instrument``, and asserts on the exported
Expand All @@ -70,8 +70,8 @@ pyproto packages are excluded via ``ruff.toml``):

.. code-block:: bash

uvx ruff@latest check packages/dash0-opentelemetry-distro examples
uvx ruff@latest format --check packages/dash0-opentelemetry-distro examples
uvx ruff@latest check packages/dash0-opentelemetry examples
uvx ruff@latest format --check packages/dash0-opentelemetry examples

Apply fixes with ``ruff check --fix`` and ``ruff format`` (drop ``--check``).

Expand Down
18 changes: 8 additions & 10 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Architecture
::

packages/
dash0-opentelemetry-distro/ the distribution (distro + configurator)
dash0-opentelemetry/ the distribution (distro + configurator)
opentelemetry-pyproto/ pure-Python protobuf message classes
opentelemetry-exporter-otlp-pyproto-common/ shared OTLP encoding
opentelemetry-exporter-otlp-pyproto-http/ pure-Python OTLP/HTTP exporter (default)
Expand Down Expand Up @@ -130,25 +130,23 @@ defaults (with ``setdefault``) and never overrides values you provide.
Installation
============

Releases are published on the Dash0 package index — a static PEP 503 simple
index on GitHub Pages whose artifacts are GitHub release assets of this
repository — not on public PyPI:
Releases are published on **public PyPI** and on the Dash0 package index (a
static PEP 503 simple index on GitHub Pages):

.. code-block:: bash

pip install \
--extra-index-url https://dash0hq.github.io/opentelemetry-python-distribution/simple/ \
dash0-opentelemetry-distro
pip install dash0-opentelemetry

Production consumers (such as the dash0-operator's instrumentation image
build) should install with a fully hashed lockfile, ``--require-hashes``, and
The Dash0 index hosts the full set of workspace packages (including the
vendored pyproto exporters) and remains the authoritative source for
production consumers that install with ``--require-hashes`` and
``--only-binary :all:``; see the consumer contract in `RELEASING.rst`_.

For development, consume the workspace directly:

.. code-block:: bash

uv sync --package dash0-opentelemetry-distro
uv sync --package dash0-opentelemetry


Development
Expand Down
59 changes: 29 additions & 30 deletions RELEASING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,20 @@ on.
Publishing channel
==================

Packages are **not** published to public PyPI. Each release attaches wheels
and sdists as GitHub release assets, and a static `PEP 503
<https://peps.python.org/pep-0503/>`_ "simple" index served from GitHub Pages
points at them with ``#sha256=`` fragments:
Each release publishes packages to **two channels**:

https://dash0hq.github.io/opentelemetry-python-distribution/simple/
1. **Public PyPI** — via OIDC trusted publishing in the ``release.yml``
``publish-pypi`` job. Consumers can install with a plain
``pip install dash0-opentelemetry``.
2. **Dash0 package index** — a static `PEP 503
<https://peps.python.org/pep-0503/>`_ "simple" index served from GitHub
Pages, backed by GitHub release assets with ``#sha256=`` fragments:

https://dash0hq.github.io/opentelemetry-python-distribution/simple/

This index hosts all five workspace packages (including the vendored pyproto
exporters) and is the authoritative source for production consumers that
install with ``--require-hashes`` and ``--only-binary :all:``.

``.github/workflows/release.yml`` implements the pipeline; the index is
regenerated statelessly from all published releases by
Expand All @@ -33,11 +41,10 @@ The three decisions that used to gate publishing are resolved:
entry-point names stay upstream-shaped: when upstream publishes the
official packages, the distribution switches its dependencies over and the
renamed copies are deprecated. The ``dash0-opentelemetry-*`` names are
defensively registered on public PyPI (see `Name reservation`_).
claimed on public PyPI via the first real release.
2. **Scope.** All five workspace packages are published to the self-hosted
index. Public PyPI publication of the real packages is deferred until
there is a non-operator audience; the artifacts and the already-claimed
names make that step additive.
index. The main ``dash0-opentelemetry`` package and the four supporting
pyproto packages are also published to public PyPI.
3. **Versioning.** The distribution versions independently (semver). The
vendored packages carry their upstream base version with ``.postN``
increments for Dash0-side changes (e.g. ``1.44.0``, ``1.44.0.post1``).
Expand Down Expand Up @@ -146,29 +153,21 @@ Repository (an admin, before the first release):
manifest commits; do not widen branch protection for the generic Actions
token instead.
6. Create the ``pypi`` environment with **required reviewers** and a
deployment policy restricted to the default branch (used only by the name
reservation below).

.. _Name reservation:
deployment policy restricted to tag refs matching ``v*`` (the
``publish-pypi`` job runs on every tag push).

Name reservation on public PyPI (an org admin, **before the rename lands on
the default branch** — the names are guessable from the public history the
moment it merges):
PyPI trusted publishing (an org admin, once, before the first release):

1. Create the Dash0 **organization account** on PyPI with enforced 2FA.
2. Check each of the five names is still unclaimed, then add a **pending
trusted publisher** per name, bound to ``reserve-pypi-names.yml`` and the
``pypi`` environment of this repository. Pending publishers do not reserve
names — only the first upload does — so proceed immediately.
3. Run the ``Reserve PyPI names`` workflow. It publishes ``0.0.0.devN`` stubs
whose README points at the real index. Re-runs are safe
(``skip-existing``).
4. **Mandatory:** remove the five trusted publishers on PyPI in the same
session and verify in each project's Publishing settings that none remain.
A standing publisher binding is a publish capability to the public names
for anyone who can run the workflow. (Re-adding one takes two minutes if a
stub ever needs refreshing; increment ``N`` — PyPI permanently forbids
filename reuse.)
2. For each of the five package names, add a **pending trusted publisher**
bound to ``release.yml``, the ``publish-pypi`` job, and the ``pypi``
environment of this repository. Pending publishers claim names on first
upload without pre-registering them.
3. SHA-pin ``pypa/gh-action-pypi-publish`` in ``release.yml`` (the current
placeholder uses ``@release/v1``; replace with the SHA of the latest
release and a version comment before merging).
4. Verify by pushing a pre-release tag (e.g. ``v0.3.0rc1``) and confirming
all five packages appear on PyPI under the Dash0 organization.
5. When the PEP 755 namespace-grant process goes live on PyPI, apply for a
restricted grant on the ``dash0`` prefix through the organization account.

Expand Down Expand Up @@ -247,7 +246,7 @@ Local build (dry run)
dash0-opentelemetry-exporter-otlp-pyproto-common \
dash0-opentelemetry-exporter-otlp-pyproto-http \
dash0-opentelemetry-exporter-otlp-pyproto-grpc \
dash0-opentelemetry-distro; do
dash0-opentelemetry; do
uv build --package "$name" --out-dir dist \
--build-constraints scripts/build-constraints.txt
done
2 changes: 1 addition & 1 deletion examples/dash0-distro-flask/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ services:
/workspace/packages/opentelemetry-exporter-otlp-pyproto-common \
/workspace/packages/opentelemetry-exporter-otlp-pyproto-http \
/workspace/packages/opentelemetry-exporter-otlp-pyproto-grpc \
/workspace/packages/dash0-opentelemetry-distro \
/workspace/packages/dash0-opentelemetry \
flask \
requests \
opentelemetry-instrumentation-flask
Expand Down
4 changes: 2 additions & 2 deletions examples/dash0-distro-flask/verify_pyproto.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"dash0-opentelemetry-exporter-otlp-pyproto-common",
"dash0-opentelemetry-exporter-otlp-pyproto-http",
"dash0-opentelemetry-exporter-otlp-pyproto-grpc",
"dash0-opentelemetry-distro",
"dash0-opentelemetry",
)
# The non-opentelemetry-* packages the agent is allowed to pull in transitively,
# as PEP 503-normalized names. Most are pure-Python support libraries of
Expand All @@ -33,7 +33,7 @@
# a compiled extension but is deliberately accepted so the curated set can ship
# system metrics. Everything else - including grpcio and protobuf - is rejected.
# Keep this list in sync when bumping the pinned opentelemetry versions in
# dash0-opentelemetry-distro's pyproject.toml.
# dash0-opentelemetry's pyproject.toml.
ALLOWED_NON_OTEL_REQUIREMENTS = (
"asgiref",
"packaging",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
==========================
dash0-opentelemetry-distro
==========================
==================
dash0-opentelemetry
==================

The Dash0 OpenTelemetry distribution for Python. It is the Python counterpart of
the Dash0 Node.js distribution (``@dash0/opentelemetry``), adapted to Python's
Expand Down Expand Up @@ -127,7 +127,7 @@ building an injectable tree — such as the dash0-operator's instrumentation
image —

1. installs the distribution into a self-contained directory:
``pip install --target <dir> dash0-opentelemetry-distro``;
``pip install --target <dir> dash0-opentelemetry``;
2. copies the script to ``<dir>/sitecustomize.py``;
3. generates ``<dir>/all-dependencies.txt``, a flattened list of every
requirement of the installed tree (one PEP 508 requirement per line).
Expand Down Expand Up @@ -164,6 +164,6 @@ Status

Prototype. The ``otlp_proto_http`` exporter is resolved from the in-repo pyproto
workspace member, published as ``dash0-opentelemetry-exporter-otlp-pyproto-http``
on the Dash0 package index (not on public PyPI). Not yet done: the injector's
on the Dash0 package index and on public PyPI. Not yet done: the injector's
per-``libc`` packaging and the integration/injection test matrix. Unit tests do
not require a running collector.
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "dash0-opentelemetry-distro"
name = "dash0-opentelemetry"
dynamic = ["version"]
description = "Dash0 OpenTelemetry distribution for Python"
readme = "README.rst"
Expand Down Expand Up @@ -33,8 +33,7 @@ dependencies = [
# Pure-Python OTLP exporters (no native dependencies), vendored in this
# workspace from open-telemetry/opentelemetry-packaging under Dash0-owned
# distribution names (import paths and entry points stay upstream-shaped).
# Published on the Dash0 package index, not on public PyPI; resolvable there
# or from this workspace.
# Published on both the Dash0 package index and public PyPI.
"dash0-opentelemetry-exporter-otlp-pyproto-http == 1.44.0",
"dash0-opentelemetry-exporter-otlp-pyproto-grpc == 1.44.0",

Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# -*- coding: utf-8 -*-
# Bootstrap script for injector-based activation of the Dash0 OpenTelemetry distribution for Python.
#
# This file ships inside the dash0-opentelemetry-distro wheel, but it is not meant to be imported from its packaged
# This file ships inside the dash0-opentelemetry wheel, but it is not meant to be imported from its packaged
# location (importing it executes it). Consumers that build an injectable tree — such as the dash0-operator's
# instrumentation image build — install the distribution into a self-contained directory
# (`pip install --target <dir> dash0-opentelemetry-distro`) and copy this file from its packaged location to
# (`pip install --target <dir> dash0-opentelemetry`) and copy this file from its packaged location to
# `<dir>/sitecustomize.py`. The OpenTelemetry injector (https://github.com/open-telemetry/opentelemetry-injector) then
# prepends `<dir>` to the PYTHONPATH environment variable of the processes to instrument, so that Python's `site`
# machinery imports this script on interpreter startup. The script initializes the OpenTelemetry
Expand Down Expand Up @@ -110,7 +110,7 @@ def _self_deactivate(current_site):

def _shipped_opentelemetry_package_names(current_site):
# The OpenTelemetry packages this distribution ships: the injected tree at current_site is exactly the pinned
# dependency closure of dash0-opentelemetry-distro (see its pyproject.toml), so enumerate the packages installed
# dependency closure of dash0-opentelemetry (see its pyproject.toml), so enumerate the packages installed
# there instead of maintaining a hardcoded list. Only the OpenTelemetry-related packages count for the double
# instrumentation check; overlap on the general-purpose support packages we also ship (wrapt, psutil, ...) is
# common and is handled by the dependency version conflict check instead.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from opentelemetry.instrumentation import bootstrap_gen
from packaging.requirements import Requirement

_DISTRO = "dash0-opentelemetry-distro"
_DISTRO = "dash0-opentelemetry"


def _declared():
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ members = ["packages/*"]
exclude = ["examples/*"]

[tool.uv.sources]
dash0-opentelemetry-distro = { workspace = true }
dash0-opentelemetry = { workspace = true }
dash0-opentelemetry-pyproto = { workspace = true }
dash0-opentelemetry-exporter-otlp-pyproto-common = { workspace = true }
dash0-opentelemetry-exporter-otlp-pyproto-http = { workspace = true }
Expand Down
Loading
Loading