[FEATURE] Expose `config.faraday_middleware` so observability gems can stop monkey-patching

[sergey-homenko]

Scope check

• This is core LLM communication (not application logic)
• This benefits most users (not just my use case)
• This can't be solved in application code with current RubyLLM
• I read the Contributing Guide

Due diligence

• I searched existing issues
• I checked the documentation

What problem does this solve?

Three observability gems hook into RubyLLM today, and all three monkey-patch:

sinaptia/ruby_llm-instrumentation — includes modules into Chat, Embedding, Image, Transcription, Moderation and aliases the originals.
thoughtbot/opentelemetry-instrumentation-ruby_llm — prepends RubyLLM::Chat and RubyLLM::Embedding.
sergey-homenko/llm_cost_tracker (mine) — prepends RubyLLM::Provider#complete, #embed, #transcribe.
Three pinch points, three different strategies. Load order between them is unstable and aliases can collide. As a developer writing one of these gems, the experience is: pick a class, hope no other gem patches the same method, ship, and break on the next RubyLLM release that reshapes internals.

#414 has been open for a year asking for an observability seam. #525 (AS::Notifications) and #556 (OTel) both proposed shapes you closed in favor of "instrumentation lives outside RubyLLM" — which I agree with. The thing that's still missing is a way for those external gems to attach without monkey-patching.

Proposed solution

Expose the Faraday middleware stack that Connection#setup_middleware already builds. About nine lines of production code:

lib/ruby_llm/configuration.rb:

option :faraday_middleware, -> { [] }

lib/ruby_llm/connection.rb:

def setup_middleware(faraday)
  faraday.request :multipart
  faraday.request :json
  faraday.response :json
  faraday.adapter :net_http
  faraday.use :llm_errors, provider: @provider
  apply_user_middleware(faraday)
end

def apply_user_middleware(faraday)
  Array(@config.faraday_middleware).each do |entry|
    entry.respond_to?(:call) ? entry.call(faraday) : faraday.use(*entry)
  end
end

Usage from a third-party gem:

RubyLLM.configure do |config|
  config.faraday_middleware = [
    [MyTracingMiddleware, tags: { app: "myapp" }],
    ->(faraday) { faraday.use MyOtherMiddleware }
  ]
end

User middleware runs last in the stack, so on the response path it sees raw bodies first and can observe non-2xx responses before :llm_errors raises. On the request path it runs after JSON encoding, just before the adapter.

Default is [], so existing users see no behavior change. Happy to send a PR with the diff above plus specs in spec/ruby_llm/connection_middleware_spec.rb (array-form, callable-form, empty-default) and a short README paragraph.

Why this belongs in RubyLLM

The seam has to live where the Faraday connection is built. Connection#setup_middleware is private, @connection is created inside Provider#initialize and not reassignable from outside, and there's no public method to register middleware against an existing connection. There's no way for application code or a third-party gem to inject middleware without reaching into RubyLLM's internals — which is what all three observability gems above are doing.

This isn't adding an instrumentation API to RubyLLM. It's exposing one line of Faraday's existing contract — the same Faraday whose request_timeout, max_retries, http_proxy you already surface as configuration options. Tracing-format opinions, payload shapes, and notification names stay entirely in the external gems.

[sergey-homenko]

Hey @crmne 👋 bumping for sign-off per CONTRIBUTING.md. Concrete shape:

# configuration.rb
option :faraday_middleware, -> { [] }

# connection.rb (private)
def apply_user_middleware(faraday)
  Array(@config.faraday_middleware).each do |entry|
    if entry.respond_to?(:call) && !entry.is_a?(Array)
      entry.call(faraday)
    else
      faraday.use(*Array(entry))
    end
  end
end

Called from initialize after setup_middleware, so user middleware sits outside :llm_errors and observes raw responses before they're turned into exceptions.

Tests would land in spec/ruby_llm/connection_middleware_spec.rb (default empty, both registration forms, + one integration check that user middleware runs outside :llm_errors).

One question before I cut a branch: faraday_middleware is honest, but commits the public option name to Faraday forever. connection_middleware is more neutral. Preference?

Happy to ship on your OK.