Skip to content

avoid quadratic group value recomputation in TokenList#848

Merged
andialbrecht merged 1 commit into
andialbrecht:masterfrom
alhudz:tokenlist-value-quadratic
Jun 6, 2026
Merged

avoid quadratic group value recomputation in TokenList#848
andialbrecht merged 1 commit into
andialbrecht:masterfrom
alhudz:tokenlist-value-quadratic

Conversation

@alhudz

@alhudz alhudz commented Jun 1, 2026

Copy link
Copy Markdown

group_tokens and TokenList.init recompute a group's value by deep-flattening its whole subtree, so grouping an N-element identifier list is O(N^2): a 4000-column SELECT (~23 KB, under the 10000-token cap) takes ~1.8s to parse. Build the value from the direct children's already-current values and append only the newly added tokens on extend. Linear time, byte-identical parse output, full test suite passes.

@codecov

codecov Bot commented Jun 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.11%. Comparing base (7334ac9) to head (a116cbe).
⚠️ Report is 100 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #848      +/-   ##
==========================================
+ Coverage   97.04%   97.11%   +0.06%     
==========================================
  Files          20       31      +11     
  Lines        1558     3642    +2084     
  Branches        0      327     +327     
==========================================
+ Hits         1512     3537    +2025     
- Misses         46       63      +17     
- Partials        0       42      +42     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@andialbrecht andialbrecht merged commit 939b129 into andialbrecht:master Jun 6, 2026
11 checks passed
@andialbrecht

Copy link
Copy Markdown
Owner

@alhudz thanks for the patch!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants