perf(strawberryshake): index data-type components by runtime name#18
Open
cliedeman wants to merge 1 commit into
Open
perf(strawberryshake): index data-type components by runtime name#18cliedeman wants to merge 1 commit into
cliedeman wants to merge 1 commit into
Conversation
DataTypeDescriptorMapper scaled super-quadratically with operation count because, for every component of every data type, it resolved the matching descriptor with `context.Types.Single(t => t.RuntimeType.Name.Equals(name))`. `context.Types` grows with the number of operations (tens of thousands of entries for large clients), and `Single` performs a full linear scan that never short-circuits. Total cost was O(sum-of-components x |Types|), and since component count, data-type count, and the type list all grow together, the wall time exploded as ops increased. Fix: build a `Dictionary<string, RuntimeTypeMatch>` keyed on `RuntimeType.Name` once, then resolve each component via an O(1) lookup. `RuntimeTypeMatch` keeps the first match plus a count so the original `Single(...)` semantics are preserved exactly: an absent name and a duplicated name both still throw. The overall pass is now near-linear in the number of types. Generated output is byte-for-byte unchanged: across the CSharp generator test suite (114 tests), the produced sources are identical with and without this change (verified by diffing snapshot output captured with the fix stashed vs. applied; the suite's pre-existing failures in this environment are formatter drift unrelated to this mapper and have identical counts either way). Measured on the real 672-op ILOPS client (schema reused, warm), isolating DataTypeDescriptorMapper.Map: N=200 (types=33,215): 2,804 ms -> 40 ms (~70x) N=275 (types=48,189): 11,376 ms -> 34 ms (~335x) The mapper is now flat across N instead of super-quadratic. Relates to #17 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
DataTypeDescriptorMapperscaled super-quadratically with operation count and was the worst non-formatting cost inCSharpGenerator.Generate.For every component of every data type it resolved the matching descriptor with a full linear scan over the entire type list:
context.Typesgrows with the number of operations (33k entries at 200 ops, 48k at 275 ops for the real client used here), andEnumerable.Singlescans the whole list and never short-circuits (it must verify uniqueness). The total cost wasO(sum-of-components × |Types|). Because component count, data-type count, and the type list all grow together as ops increase, the wall time exploded.The fix
src/StrawberryShake/CodeGeneration/src/CodeGeneration/Mappers/DataTypeDescriptorMapper.csBuild a
Dictionary<string, RuntimeTypeMatch>keyed onRuntimeType.Nameonce (DataTypeDescriptorMapper.cs:34-48), then resolve each component via an O(1) lookup (:97, helper at:105-125). The whole pass is now near-linear in the number of types.RuntimeTypeMatch(:127-146) stores the first match plus a count, so the originalSingle(...)semantics are preserved exactly: an absent runtime name and a duplicated runtime name both still throwInvalidOperationException. Names that appear in the type list but are never used as a data-type component are indexed but never trigger a throw, matching the original lazy behaviour. The struct holds only first + count (no per-duplicate list), keeping the index allocation-light.Correctness: generated output is byte-identical
This is a pure performance change. Verified across the full CSharp generator test suite (114 tests) by capturing snapshot output with the fix stashed vs. applied and diffing:
The suite has pre-existing failures in this environment (Roslyn formatter / SDK drift, unrelated to this mapper); the failure counts are identical with and without the change (114 total, 111 fail, 2 pass, 1 skip both ways), and the generated sources themselves are byte-for-byte equal.
Before / after
Measured on the real 672-op ILOPS client (schema reused, warm) via the file-based
CSharpGenerator.Generatepipeline. TheDataTypeDescriptorMapper.Map(...)call was temporarily wrapped in aStopwatch(instrumentation removed before commit; the commit is the clean fix only).DataTypeDescriptorMapper.MapbeforeFull
generatestep (single-file, includes Roslyn formatting which dominates and is out of scope here):The mapper itself went from a super-quadratic curve (2,804 → 11,376 ms, ~4× for 1.375× more ops) to flat (~34-40 ms regardless of N). The remaining
generatecost is Roslyn whitespace normalization /Formatter.Format, a separate hotspot.Relates to #17
— Claude
🤖 Generated with Claude Code