Frontier has logs and Prometheus metrics, but no request tracing. Metrics show counts and averages; they cannot show where one slow request spent its time.
What exists today
- The otelconnect interceptor already collects per-RPC metrics through the OTel SDK and exports them to Prometheus (
pkg/server/server.go). Tracing is explicitly disabled there with WithoutTracing().
- Every Postgres query goes through
Client.WithTimeout in pkg/db/db.go, which records Prometheus latency histograms.
- Every SpiceDB call goes through
internal/store/spicedb/relation_repository.go.
otelgrpc and otelhttp are already indirect dependencies.
Proposal
- Add an OTel
TracerProvider with an OTLP exporter, configured via a new config block: enabled flag, endpoint, sampling ratio.
- Remove
WithoutTracing() from the otelconnect interceptor so each RPC gets a span.
- Add child spans for Postgres queries in
WithTimeout, or adopt otelsqlx (an existing TODO in pkg/db/db.go asks for this).
- Add spans for SpiceDB calls in the relation repository, or add the otelgrpc client interceptor to the SpiceDB connection.
With this, one slow request shows its timing breakdown — interceptors, each SQL query, each SpiceDB check — and traces can go to any OTLP backend.
Follow-up to #1783, where moving from New Relic to OpenTelemetry was suggested.
Frontier has logs and Prometheus metrics, but no request tracing. Metrics show counts and averages; they cannot show where one slow request spent its time.
What exists today
pkg/server/server.go). Tracing is explicitly disabled there withWithoutTracing().Client.WithTimeoutinpkg/db/db.go, which records Prometheus latency histograms.internal/store/spicedb/relation_repository.go.otelgrpcandotelhttpare already indirect dependencies.Proposal
TracerProviderwith an OTLP exporter, configured via a new config block: enabled flag, endpoint, sampling ratio.WithoutTracing()from the otelconnect interceptor so each RPC gets a span.WithTimeout, or adopt otelsqlx (an existing TODO inpkg/db/db.goasks for this).With this, one slow request shows its timing breakdown — interceptors, each SQL query, each SpiceDB check — and traces can go to any OTLP backend.
Follow-up to #1783, where moving from New Relic to OpenTelemetry was suggested.