Skip to content

Performance: reduce per-call allocations in channel recreation outcome tracking #777

Description

@berndverst

1. What is the issue?

When channel recreation is enabled, every unary gRPC call registers a continuation and boxes a tuple state object solely to update consecutive-failure tracking.

2. Likely priority

Low. This is a small cost per call, but it can become observable for high-QPS instance-management or signaling clients.

3. Impact on performance

Each unary call allocates continuation-related state and a boxed (ChannelRecreatingCallInvoker, methodName) tuple. The resulting continuous allocation rate can increase GC pressure even when all RPCs succeed.

4. Details and code reference

Unary-call interception:

public override AsyncUnaryCall<TResponse> AsyncUnaryCall<TRequest, TResponse>(
Method<TRequest, TResponse> method, string? host, CallOptions options, TRequest request)
{
TransportState current = Volatile.Read(ref this.state);
AsyncUnaryCall<TResponse> call = current.Invoker.AsyncUnaryCall(method, host, options, request);
this.ObserveOutcome(call.ResponseAsync, method.FullName);
return call;
}

Outcome observation:

void ObserveOutcome<TResponse>(Task<TResponse> responseAsync, string methodFullName)
{
// Use ContinueWith with TaskScheduler.Default so we don't capture sync context.
responseAsync.ContinueWith(
(t, state) =>
{
var (self, name) = ((ChannelRecreatingCallInvoker, string))state!;
if (t.Status == TaskStatus.RanToCompletion)
{
self.RecordSuccess();
}
else if (t.Exception?.InnerException is RpcException rpcEx)
{
self.RecordFailure(rpcEx.StatusCode, name);
}
},
(this, methodFullName),
CancellationToken.None,
TaskContinuationOptions.ExecuteSynchronously,
TaskScheduler.Default);
}

ObserveOutcome registers ContinueWith and passes (this, methodFullName) through the object-typed continuation state parameter.

5. Guidance for how to fix

Profile allocation rates first. If material, redesign the observation helper to avoid boxed state while preserving current success and transport-failure classification, execution-synchronous continuation behavior, and method-name diagnostics. Do not compromise the existing lock-free channel lifecycle or failure accounting for a micro-optimization.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions