Performance: avoid repeated protobuf sizing during orchestration response chunking

[berndverst]

1. What is the issue?

The orchestration response chunking path can calculate protobuf action sizes multiple times: once for validation, once as part of whole-response sizing, and again while filling chunks.

2. Likely priority

Medium. The largest redundant portion is for workers without the LargePayloads capability; the chunking-specific repeat work occurs only for oversized responses. It is most visible with large payloads or many actions per orchestration turn.

3. Impact on performance

CalculateSize() walks the protobuf message graph, including serialized string payloads. Repeating it adds additional O(payload size x action count) CPU work to an orchestration completion, increasing replay-turn latency for fan-out responses.

4. Details and code reference

Relevant chunking implementation:

durabletask-dotnet/src/Worker/Grpc/GrpcDurableTaskWorker.Processor.cs

Lines 1096 to 1210 in 883211a

	async Task CompleteOrchestratorTaskWithChunkingAsync(
	P.OrchestratorResponse response,
	int maxChunkBytes,
	CancellationToken cancellationToken)
	{
	// Validate that no single action exceeds the maximum chunk size
	static P.TaskFailureDetails? ValidateActionsSize(IEnumerable<P.OrchestratorAction> actions, int maxChunkBytes)
	{
	foreach (P.OrchestratorAction action in actions)
	{
	int actionSize = action.CalculateSize();
	if (actionSize > maxChunkBytes)
	{
	// TODO: large payload doc is not available yet on aka.ms, add doc link to below error message
	string errorMessage = $"A single orchestrator action of type {action.OrchestratorActionTypeCase} with id {action.Id} " +
	$"exceeds the {maxChunkBytes / 1024.0 / 1024.0:F2}MB limit: {actionSize / 1024.0 / 1024.0:F2}MB. " +
	"Enable large-payload externalization to Azure Blob Storage to support oversized actions.";
	return new P.TaskFailureDetails
	{
	ErrorType = typeof(InvalidOperationException).FullName,
	ErrorMessage = errorMessage,
	IsNonRetriable = true,
	};
	}
	}
	return null;
	}
	P.TaskFailureDetails? validationFailure = this.worker.grpcOptions.Capabilities.Contains(P.WorkerCapability.LargePayloads)
	? null
	: ValidateActionsSize(response.Actions, maxChunkBytes);
	if (validationFailure != null)
	{
	// Complete the orchestration with a failed status and failure details
	P.OrchestratorResponse failureResponse = new()
	{
	InstanceId = response.InstanceId,
	CompletionToken = response.CompletionToken,
	OrchestrationTraceContext = response.OrchestrationTraceContext,
	Actions =
	{
	new P.OrchestratorAction
	{
	CompleteOrchestration = new P.CompleteOrchestrationAction
	{
	OrchestrationStatus = P.OrchestrationStatus.Failed,
	FailureDetails = validationFailure,
	},
	},
	},
	};
	await this.ExecuteWithRetryAsync(
	async () => await this.client.CompleteOrchestratorTaskAsync(failureResponse, cancellationToken: cancellationToken),
	nameof(this.client.CompleteOrchestratorTaskAsync),
	cancellationToken);
	return;
	}
	// Helper to add an action to the current chunk if it fits
	static bool TryAddAction(
	Google.Protobuf.Collections.RepeatedField<P.OrchestratorAction> dest,
	P.OrchestratorAction action,
	ref int currentSize,
	int maxChunkBytes)
	{
	int actionSize = action.CalculateSize();
	if (currentSize + actionSize > maxChunkBytes && currentSize > 0)
	{
	return false;
	}
	dest.Add(action);
	currentSize += actionSize;
	return true;
	}
	// Check if the entire response fits in one chunk
	int totalSize = response.CalculateSize();
	if (totalSize <= maxChunkBytes)
	{
	// Response fits in one chunk, send it directly (isPartial defaults to false)
	await this.ExecuteWithRetryAsync(
	async () => await this.client.CompleteOrchestratorTaskAsync(response, cancellationToken: cancellationToken),
	nameof(this.client.CompleteOrchestratorTaskAsync),
	cancellationToken);
	return;
	}
	// Response is too large, split into multiple chunks
	int actionsCompletedSoFar = 0, chunkIndex = 0;
	List<P.OrchestratorAction> allActions = response.Actions.ToList();
	bool isPartial = true;
	bool isChunkedMode = false;
	while (isPartial)
	{
	P.OrchestratorResponse chunkedResponse = new()
	{
	InstanceId = response.InstanceId,
	CustomStatus = response.CustomStatus,
	CompletionToken = response.CompletionToken,
	RequiresHistory = response.RequiresHistory,
	NumEventsProcessed = 0,
	};
	int chunkPayloadSize = 0;
	// Fill the chunk with actions until we reach the size limit
	while (actionsCompletedSoFar < allActions.Count &&
	TryAddAction(chunkedResponse.Actions, allActions[actionsCompletedSoFar], ref chunkPayloadSize, maxChunkBytes))
	{
	actionsCompletedSoFar++;
	}

The method validates every action with action.CalculateSize(), then invokes response.CalculateSize(), and TryAddAction calculates each action size again while constructing chunks.

5. Guidance for how to fix

Calculate the whole response size first. If it fits in one chunk, no individual action can exceed that same limit, so send it without an additional validation pass. When chunking is required, compute action sizes once and reuse the cached values for validation and chunk packing. Preserve the current oversized-single-action failure behavior when LargePayloads is unavailable, and add boundary tests around exact chunk limits and capability combinations.