Skip to content

Add OpenTelemetry tracing #1875

Description

@AmanGIT07

Frontier has logs and Prometheus metrics, but no request tracing. Metrics show counts and averages; they cannot show where one slow request spent its time.

What exists today

  • The otelconnect interceptor already collects per-RPC metrics through the OTel SDK and exports them to Prometheus (pkg/server/server.go). Tracing is explicitly disabled there with WithoutTracing().
  • Every Postgres query goes through Client.WithTimeout in pkg/db/db.go, which records Prometheus latency histograms.
  • Every SpiceDB call goes through internal/store/spicedb/relation_repository.go.
  • otelgrpc and otelhttp are already indirect dependencies.

Proposal

  • Add an OTel TracerProvider with an OTLP exporter, configured via a new config block: enabled flag, endpoint, sampling ratio.
  • Remove WithoutTracing() from the otelconnect interceptor so each RPC gets a span.
  • Add child spans for Postgres queries in WithTimeout, or adopt otelsqlx (an existing TODO in pkg/db/db.go asks for this).
  • Add spans for SpiceDB calls in the relation repository, or add the otelgrpc client interceptor to the SpiceDB connection.

With this, one slow request shows its timing breakdown — interceptors, each SQL query, each SpiceDB check — and traces can go to any OTLP backend.

Follow-up to #1783, where moving from New Relic to OpenTelemetry was suggested.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions