Skip to content

[Bug]: Intermittent empty-body HTTP 500s -- "NoMethodError: undefined method 'tracestate' for an instance of OpenTelemetry::Context" #513

Description

@reevejd

Problem Description

A Rails app using Puma and the instana gem intermittently returns empty-body 500s. The failing requests never reach application controllers and are not recorded in Instana, and the 500 response carries none of the usual headers (x-request-id, x-runtime, x-instana-*). I think this is a sibling issue of #507.

I've opened a PR for what I think is the fix here: #514.

In the meantime I've applied this monkey patch to mitigate the issue:

if defined?(Instana::Trace::Samplers)
  module Instana
    module Trace
      module Samplers
        def self.should_sample?(trace_id:, parent_context:, links:, name:, kind:, attributes:)
          tracestate =
            begin
              span_context = OpenTelemetry::Trace.current_span(parent_context).context
              if span_context.respond_to?(:tracestate)
                span_context.tracestate
              else
                OpenTelemetry::Trace::Tracestate::DEFAULT
              end
            rescue StandardError
              OpenTelemetry::Trace::Tracestate::DEFAULT
            end

          Result.new(decision: :__record_only__, tracestate: tracestate)
        end
      end
    end
  end
end

Minimal, Complete, Verifiable, Example

require "instana"

# A non-recording span built from an OpenTelemetry::Context (as start_span
# produces while the agent is not ready) exposes that Context as its #context.
poisoned_parent = OpenTelemetry::Trace.context_with_span(
  Instana::Trace.non_recording_span(OpenTelemetry::Context.empty),
  parent_context: OpenTelemetry::Context.empty
)

Instana::Trace::Samplers.should_sample?(
  trace_id: "t", parent_context: poisoned_parent,
  links: nil, name: "rack", kind: :server, attributes: nil
)
# => NoMethodError: undefined method 'tracestate' for an instance of OpenTelemetry::Context
#      .../instana/samplers/samplers.rb:71:in 'should_sample?'

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    base64 (0.3.0)
    bigdecimal (4.1.2)
    cgi (0.5.2)
    concurrent-ruby (1.3.7)
    csv (3.3.5)
    ffi (1.17.4)
    instana (2.7.1)
      base64 (>= 0.1)
      cgi
      concurrent-ruby (>= 1.1)
      csv (>= 0.1)
      logger
      oj (>= 3.0.11)
      opentelemetry-api (~> 1.4)
      opentelemetry-common
      sys-proctable (>= 1.2.2)
    logger (1.7.0)
    oj (3.17.3)
      bigdecimal (>= 3.0)
      ostruct (>= 0.2)
    opentelemetry-api (1.9.0)
      logger
    opentelemetry-common (0.24.0)
      opentelemetry-api (~> 1.0)
    ostruct (0.6.3)
    rack (3.2.6)
    sys-proctable (1.3.0)
      ffi (~> 1.1)

PLATFORMS
  ruby
  x86_64-linux-gnu

DEPENDENCIES
  instana (= 2.7.1)
  opentelemetry-api (= 1.9.0)
  opentelemetry-common (= 0.24.0)
  rack (~> 3.0)

BUNDLED WITH
   2.6.9

Ruby Version

3.4.7

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions