fix: async transform recursion + type introspection cache perf#1296
Open
giulio-leone wants to merge 4 commits intoanthropics:mainfrom
Open
fix: async transform recursion + type introspection cache perf#1296giulio-leone wants to merge 4 commits intoanthropics:mainfrom
giulio-leone wants to merge 4 commits intoanthropics:mainfrom
Conversation
…edundant dispatch The _transform_recursive function and its async variant performed type introspection (strip_annotated_type, get_origin, is_typeddict, is_list_type, is_union_type, etc.) on every recursive call, even though the type annotation is the same for all values of a given field. On large payloads (~90K messages), this consumed ~6.6% of total CPU time with zero transformation output since Messages API types have no PropertyInfo annotations. Changes: - Add _cached_transform_dispatch(): LRU-cached function that precomputes the dispatch path (typeddict/dict/sequence/union/other) and extracts type args once per annotation type. Subsequent calls are O(1) dict lookups instead of re-running type introspection. - Add _get_field_key_map(): LRU-cached function that precomputes the key alias mapping for each TypedDict type, replacing per-field _maybe_transform_key calls with a single dict.get() lookup. - Expand _no_transform_needed() to include str and bool, allowing lists of strings/bools to skip per-element recursion. - Apply same optimizations to _async_transform_recursive and _async_transform_typeddict. Fixes anthropics#1195
Address review feedback: 1. The dict branch in _async_transform_recursive called the synchronous _transform_recursive, defeating async benefits. Changed to await _async_transform_recursive. 2. Relaxed wall-clock assertion in performance test from 2s to 10s to avoid flakiness in CI environments with variable load.
Use sync-vs-async relative timing instead of a fixed threshold so the test is resilient to CI environments with variable load while still catching performance regressions. Refs: anthropics#1195
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_transform_recursiveto eliminate redundant dispatch overhead_async_transform_recursivefor correct async semanticsThese patches fix async transform correctness and improve performance in hot paths.