⚡️ Speed up function _bounded_levenshtein by 995% in PR #1691 (codeflash/optimize-pr1689-2026-02-27T17.52.22)

[codeflash-ai]

⚡️ This pull request contains optimizations for PR #1691

If you approve this dependent PR, these changes will be merged into the original PR branch codeflash/optimize-pr1689-2026-02-27T17.52.22.

This PR will be automatically closed if the original PR is merged.

---

📄 995% (9.95x) speedup for _bounded_levenshtein in codeflash/discovery/functions_to_optimize.py

⏱️ Runtime : 97.4 milliseconds → 8.90 milliseconds (best of 157 runs)

📝 Explanation and details

Primary benefit (runtime): The optimized implementation cuts total runtime from ~97.4 ms to ~8.90 ms (≈10× faster, reported as a 994% speedup). This is the reason the change was accepted.

What changed (concrete edits)

• Replaced the two Python loops that filled "outside the band" (current[k] = max_distance + 1 for k before start and after end) with a single C-level list multiplication per row: current = [max_dist] * (n + 1).
• Eliminated the slice+min pass row_min = min(previous[start:end+1]) by tracking min_in_band while computing values inside the inner loop.
• Precomputed max_dist = max_distance + 1 and used a local val variable for clarity; returned max_dist instead of recomputing the expression repeatedly.
• Kept the same DP recurrence (min of deletion/insertion/substitution) but moved a small min-of-three into a local val for readability.

Why this speeds up the code (Python-level reasoning)

• The original implementation performed many per-element Python assignments to set outside-band sentinel values (two for-loops per row). Those Python-level loops ran millions of times and dominated the profile. Replacing them with list multiplication moves that work into C (one fast allocation + repeated pointer writes inside CPython), which is orders of magnitude cheaper than executing the equivalent Python loop iterations. The original profiler shows the outside-band assignments as the largest hot spots; removing them removes the dominant Python overhead.
• The original used slicing and min(previous[start:end+1]) to compute the row minimum for early exit. Slicing allocates a new list and then min scans it; tracking the minimum while we compute the band avoids that extra allocation and pass, saving another expensive Python-level operation.
• Fewer Python-level loops/assignments and fewer temporary allocations mean much less interpreter overhead (function/bytecode dispatch, range iteration, index operations). The inner DP loop remains but is now the main work — everything else is minimized.

Evidence in the line profiles and tests

• The original profile shows the outside-band filling loops and the right-side loop consumed large amounts of CPU time; the optimized profile replaces those with a single list multiplication and an inline min tracker. The optimized profile shows far less time outside the inner DP loop and no separate slice+min pass.
• Tests that stress long strings with few edits (the intended use-case: bounded Levenshtein with a small band) see the biggest wins. For example, large strings with 3–5 substitutions drop from tens of milliseconds to a few milliseconds (see annotated tests: 32.3ms -> 3.44ms and 32.2ms -> 2.55ms). Cases where many rows previously did heavy outside-band work benefit the most.
• Cases that are already trivial (very short strings or exact-equality fast path) see little change or a tiny regression in a handful of microbenchmarks. This is expected: we now allocate a new current list each row (list multiplication cost) instead of reusing and writing into an existing list; for very tiny inputs that allocation can be a small extra cost. This is an acceptable trade-off given the large runtime wins in hot/higher-cost scenarios.

Impact on callers / hot-path considerations

• closest_matching_file_function_name calls _bounded_levenshtein repeatedly across candidate names and is a natural hot path where many short-to-moderate comparisons occur. Because the optimized function reduces per-call interpreter overhead dramatically (especially for moderate-length strings where the band is narrow), callers that run many comparisons will see an aggregate speedup roughly proportional to the per-call improvement. That makes this optimization effective where candidate lists are large (file/function name fuzzy-matching).
• Workloads that perform many long-string comparisons but expect only a few edits (narrow band) benefit most. Workloads composed mostly of trivially short comparisons will still be correct and may see neutral or slightly lower performance, but overall throughput in realistic search/matching scenarios improves substantially.

Correctness and trade-offs

• The algorithmic result is unchanged (same DP recurrence, same early-exit behavior and return convention). The small regressions on trivial micro-benchmarks stem from the change to allocate a fresh current list each row (fast C-level allocation) instead of reusing and writing into an existing list; this is a modest memory/alloc overhead traded for removing expensive per-element Python assignments and slice+min passes.
• Given the intended hot-path use (many comparisons, long-ish strings, narrow bands), this trade-off is overwhelmingly positive for runtime and throughput.

In short: the optimized code reduces Python-level loop/assignment and temporary-allocation overhead by using C-level list multiplication for sentinel initialization and by keeping the band minimum inline. Those two changes remove the dominant interpreter work observed in the original profile and produce the measured ~10× runtime improvement for real workloads while preserving correctness.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	🔘 None Found
🌀 Generated Regression Tests	✅ 39 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	🔘 None Found
📊 Tests Coverage	100.0%

🌀 Click to see Generated Regression Tests

import pytest  # used for our unit tests
from codeflash.discovery.functions_to_optimize import _bounded_levenshtein

def test_equal_strings_returns_zero():
    # Identical strings must return 0 quickly via the fast-path equality check.
    codeflash_output = _bounded_levenshtein("kitten", "kitten", 2) # 470ns -> 501ns (6.19% slower)

def test_single_substitution_within_bound():
    # One substitution between the strings -> distance 1; with max_distance >= 1
    # the function should compute and return the exact distance 1.
    codeflash_output = _bounded_levenshtein("kitten", "sitten", 2) # 18.5μs -> 14.6μs (26.6% faster)

def test_single_insertion_within_bound_and_order_irrelevant():
    # One insertion: "kitten" -> "kittens" has distance 1.
    # Also check that swapping arguments yields the same numeric result
    # (the implementation swaps to ensure s1 is shorter).
    codeflash_output = _bounded_levenshtein("kitten", "kittens", 1) # 16.9μs -> 12.1μs (39.0% faster)
    codeflash_output = _bounded_levenshtein("kittens", "kitten", 1) # 13.1μs -> 9.30μs (41.4% faster)

def test_both_empty_strings():
    # Both inputs empty -> zero edits.
    codeflash_output = _bounded_levenshtein("", "", 0) # 461ns -> 451ns (2.22% faster)

def test_empty_vs_nonempty_with_length_exceeds_max_triggers_early_return():
    # If length difference already exceeds max_distance, the function returns max_distance+1.
    s1 = "a" * 3
    s2 = "a" * 10
    # m - n = 7 > 5 so we expect 5 + 1 = 6
    codeflash_output = _bounded_levenshtein(s1, s2, 5) # 682ns -> 701ns (2.71% slower)

def test_max_distance_zero_non_equal_strings_returns_one():
    # When max_distance is 0, any non-equal strings must cause an immediate "too large"
    # result which the implementation represents as max_distance + 1 == 1.
    codeflash_output = _bounded_levenshtein("a", "b", 0) # 5.65μs -> 4.56μs (23.9% faster)

def test_empty_s1_and_short_s2_observed_behavior():
    # Important: the current implementation contains a case where s1 == "" and s2 is short
    # (len(s2) <= max_distance) leads to an early "band empty" return of max_distance+1.
    # This test captures that observed behavior (ensures tests align with implementation).
    # For s1 = "" and s2 = "x" with max_distance = 1, the function returns 2 (1 + 1).
    codeflash_output = _bounded_levenshtein("", "x", 1) # 3.27μs -> 2.90μs (12.4% faster)

def test_unicode_and_special_characters_substitution():
    # Ensure non-ASCII characters are handled; a single emoji substitution counts as 1.
    s1 = "hello🙂world"
    s2 = "hello😃world"  # replace one emoji
    codeflash_output = _bounded_levenshtein(s1, s2, 2) # 33.2μs -> 25.2μs (31.7% faster)

def test_long_identical_strings_returns_zero():
    # Very long identical strings should still hit the fast equality path and return 0 quickly.
    long_s = "a" * 1000
    codeflash_output = _bounded_levenshtein(long_s, long_s, 10) # 401ns -> 401ns (0.000% faster)

def test_long_strings_with_few_substitutions_within_band():
    # Create two long strings that differ at a small number of positions.
    # Since the number of substitutions is <= max_distance, the function should return the exact count.
    n = 1000
    s1 = list("a" * n)
    s2 = list("a" * n)
    # Introduce 5 deterministic substitutions at known positions (centered).
    indices = [490, 495, 500, 505, 510]
    for idx in indices:
        s2[idx] = "b"
    s1 = "".join(s1)
    s2 = "".join(s2)
    # max_distance larger than the number of substitutions ensures the exact distance is returned.
    codeflash_output = _bounded_levenshtein(s1, s2, 10) # 32.3ms -> 3.44ms (837% faster)

def test_large_length_difference_exceeds_max_fast_path():
    # Very long strings where the length difference exceeds max_distance should trigger the length-check fast-path.
    s_short = "a" * 1000
    s_long = "a" * 1100  # difference = 100
    # If max_distance is 50, expect return of 51 (max_distance + 1).
    codeflash_output = _bounded_levenshtein(s_short, s_long, 50) # 962ns -> 942ns (2.12% faster)

def test_returns_max_plus_one_when_distance_exceeds_bound():
    # Example where true Levenshtein distance is 3 but max_distance is 2.
    s1 = "abc"
    s2 = "xyz"
    # true distance = 3, so implementation should return max_distance + 1 = 3
    codeflash_output = _bounded_levenshtein(s1, s2, 2) # 11.4μs -> 9.43μs (20.7% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

import pytest  # used for our unit tests
from codeflash.discovery.functions_to_optimize import _bounded_levenshtein

# Helper: a plain (unbounded) Levenshtein distance implementation for verification.
# This is intentionally simple, deterministic, and used only for small inputs in tests.
def _levenshtein_unbounded(a: str, b: str) -> int:
    # Standard dynamic programming implementation (O(len(a)*len(b)) time).
    na, nb = len(a), len(b)
    # If one is empty, distance is length of the other.
    if na == 0:
        return nb
    if nb == 0:
        return na
    prev = list(range(nb + 1))
    cur = [0] * (nb + 1)
    for i in range(1, na + 1):
        cur[0] = i
        ai = a[i - 1]
        for j in range(1, nb + 1):
            if ai == b[j - 1]:
                cur[j] = prev[j - 1]
            else:
                cur[j] = 1 + min(prev[j], cur[j - 1], prev[j - 1])
        prev, cur = cur, prev
    return prev[nb]

def test_equal_strings_return_zero():
    # identical strings must return 0 regardless of max_distance
    codeflash_output = _bounded_levenshtein("abc", "abc", 0) # 451ns -> 441ns (2.27% faster)
    codeflash_output = _bounded_levenshtein("abc", "abc", 5) # 241ns -> 230ns (4.78% faster)

def test_simple_known_distances_with_sufficient_max():
    # "kitten" -> "sitting" is known Levenshtein distance 3
    s1 = "kitten"
    s2 = "sitting"
    # When max_distance is at least the true distance, the function should return the exact distance.
    codeflash_output = _bounded_levenshtein(s1, s2, 3) # 22.2μs -> 18.1μs (22.5% faster)
    # If max_distance is larger, still exact distance.
    codeflash_output = _bounded_levenshtein(s1, s2, 10) # 20.5μs -> 17.9μs (14.7% faster)
    # Verify against our unbounded helper for an additional small example.
    a = "flaw"
    b = "lawn"
    expected = _levenshtein_unbounded(a, b)
    codeflash_output = _bounded_levenshtein(a, b, 5) # 9.36μs -> 8.38μs (11.7% faster)

def test_symmetry_of_arguments():
    # Levenshtein distance is symmetric; the bounded implementation should be symmetric too.
    a = "abcdef"
    b = "azced"
    # Use a max_distance large enough not to trigger early bounding.
    codeflash_output = _bounded_levenshtein(a, b, 10); r1 = codeflash_output # 19.8μs -> 17.0μs (16.9% faster)
    codeflash_output = _bounded_levenshtein(b, a, 10); r2 = codeflash_output # 16.1μs -> 13.7μs (17.7% faster)
    # Also test symmetry when max_distance is small but still allows computation:
    codeflash_output = _bounded_levenshtein(a, b, 2); r3 = codeflash_output # 13.8μs -> 10.4μs (32.2% faster)
    codeflash_output = _bounded_levenshtein(b, a, 2); r4 = codeflash_output # 12.7μs -> 9.86μs (29.2% faster)

def test_empty_string_cases():
    # Both empty -> zero distance
    codeflash_output = _bounded_levenshtein("", "", 0) # 401ns -> 420ns (4.52% slower)
    # One empty, other non-empty: if max_distance >= length, exact length should be returned
    codeflash_output = _bounded_levenshtein("", "hello", 5) # 3.11μs -> 2.65μs (17.0% faster)
    # If max_distance smaller than needed deletions/insertions, it should return max_distance + 1
    codeflash_output = _bounded_levenshtein("", "hello", 3) # 351ns -> 360ns (2.50% slower)

def test_max_distance_zero_behavior():
    # When max_distance is zero, only identical strings return 0; any difference yields 1 (0+1)
    codeflash_output = _bounded_levenshtein("a", "a", 0) # 431ns -> 420ns (2.62% faster)
    codeflash_output = _bounded_levenshtein("a", "b", 0) # 5.24μs -> 4.23μs (23.9% faster)
    # Different lengths with max_distance 0 should early-exit with 1
    codeflash_output = _bounded_levenshtein("ab", "abc", 0) # 400ns -> 420ns (4.76% slower)

def test_length_difference_immediate_exit():
    # If length difference already exceeds max_distance, the function returns max_distance + 1 quickly.
    s_short = "abc"
    s_long = "x" * 10
    # length difference = 7 > 5, so expect 6
    codeflash_output = _bounded_levenshtein(s_short, s_long, 5) # 641ns -> 712ns (9.97% slower)
    # Check order doesn't matter (function ensures shorter is used internally)
    codeflash_output = _bounded_levenshtein(s_long, s_short, 5) # 441ns -> 451ns (2.22% slower)

def test_band_empty_early_exit_scenario():
    # Create scenario where in some row start > end and function must return max_distance+1.
    # Example: short s1 length 3, s2 length 5, with max_distance 0 -> band becomes empty when i > 3
    s1 = "abc"
    s2 = "abcXY"  # differ and longer
    codeflash_output = _bounded_levenshtein(s1, s2, 0) # 702ns -> 751ns (6.52% slower)

def test_negative_max_distance_handling():
    # The implementation does not explicitly forbid negative max_distance.
    # Behavior: if strings are equal, 0 is returned; otherwise if m-n > max_distance it's max_distance+1.
    # Example: non-equal strings with max_distance = -1 should normally return 0 (i.e., -1 + 1)
    # but only when m - n > -1 (which is true for any positive length difference).
    codeflash_output = _bounded_levenshtein("x", "y", -1) # 701ns -> 732ns (4.23% slower)
    # If the strings are equal, equality is checked first and returns 0 (consistent)
    codeflash_output = _bounded_levenshtein("same", "same", -5) # 251ns -> 260ns (3.46% slower)

def test_unicode_and_special_characters():
    # Ensure multi-byte characters and emojis are handled character-by-character.
    s1 = "café"  # includes an accented char
    s2 = "cafe"  # accent removed -> substitution
    codeflash_output = _bounded_levenshtein(s1, s2, 2) # 13.6μs -> 11.2μs (22.1% faster)
    # Emojis should be considered single characters in Python strings
    e1 = "🙂🙂🙂"
    e2 = "🙂🙃🙂"
    codeflash_output = _bounded_levenshtein(e1, e2, 2) # 6.88μs -> 5.91μs (16.4% faster)

def test_large_strings_small_number_of_edits_within_max_distance():
    # Construct two long strings (length 1000) which are identical except for 3 substitutions.
    base = ["a"] * 1000
    s1 = "".join(base)
    base2 = base.copy()
    # Make 3 substitutions at spaced positions
    base2[100] = "b"
    base2[500] = "c"
    base2[900] = "d"
    s2 = "".join(base2)
    # With max_distance 5 the bounded implementation should compute exact distance 3.
    codeflash_output = _bounded_levenshtein(s1, s2, 5) # 32.2ms -> 2.55ms (1162% faster)
    # Symmetry check on large strings
    codeflash_output = _bounded_levenshtein(s2, s1, 5) # 32.1ms -> 2.53ms (1172% faster)

def test_large_strings_entirely_different_exceeds_small_max_distance():
    # Two long strings of length 1000 but totally different characters -> true distance 1000 (all substitutions).
    # With a small max_distance the implementation should quickly return max_distance + 1 (here 11).
    s1 = "a" * 1000
    s2 = "b" * 1000
    max_d = 10
    codeflash_output = _bounded_levenshtein(s1, s2, max_d); result = codeflash_output # 419μs -> 87.2μs (381% faster)

def test_bounded_returns_exact_when_within_band_width_edge():
    # Test a case where the band is exactly as wide as needed: two strings with distance equal to max_distance.
    a = "x" * 50
    # b differs by substitutions at exactly max_distance positions
    b_list = list(a)
    indices = [0, 10, 20, 30, 40]  # 5 substitutions
    for idx in indices:
        b_list[idx] = "y"
    b = "".join(b_list)
    max_distance = len(indices)
    # The function should compute exact distance equal to number of substitutions.
    codeflash_output = _bounded_levenshtein(a, b, max_distance) # 158μs -> 86.9μs (82.0% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

To edit these changes git checkout codeflash/optimize-pr1691-2026-02-27T18.02.53 and push.

[claude]

PR Review Summary

Prek Checks

✅ Passed (after auto-fix)

• Fixed 1 issue: PLR1730 (if-stmt-min-max) in codeflash/discovery/functions_to_optimize.py — replaced manual if val < min_in_band with min(min_in_band, val)
• Committed as b1db366c

Mypy

✅ No new issues — all mypy errors on changed files are pre-existing (e.g., Language export, jedi_definition attribute, generic type params in support.py)

Code Review

No critical issues found. This PR is broader than the title suggests — beyond the _bounded_levenshtein optimization, it includes:

1. CST-only discovery: Removes legacy AST-based Python discovery (find_functions_with_return_statement, _find_all_functions_in_python_file, function_has_return_statement, function_is_a_property) and routes Python through _find_all_functions_via_language_support like other languages
2. Nested function skipping: FunctionVisitor.visit_FunctionDef now returns early if parent is a FunctionDef, only discovering top-level and class-level functions
3. Property filtering: New is_property() static method also handles cached_property
4. Type consistency: dict[str, ...] → dict[Path, ...] in multiple function signatures
5. Error propagation: PythonSupport.discover_functions no longer silently catches parse errors (callers in _find_all_functions_via_language_support still catch at the outer level)

All behavioral changes have corresponding test updates. No stale documentation references found.

Minor note: _bounded_levenshtein uses a leading underscore, which the project code style discourages — not blocking.

Test Coverage

File	PR	main	Δ
codeflash/discovery/functions_to_optimize.py	64%	70%	⚠️ -6%
codeflash/languages/python/support.py	47%	49%	-2%
codeflash/lsp/beta.py	0%	0%	—
Total (changed files)	58%	62%	-4%

⚠️ functions_to_optimize.py coverage decreased by 6%. The new _bounded_levenshtein function (45 lines) and the refactored closest_matching_file_function_name are exercised via existing tests, but removed functions had higher coverage. The lsp/beta.py change is a one-line type annotation fix with no coverage impact.

Test results: 2468 passed, 57 skipped, 8 failed (all failures in test_tracer.py — pre-existing, unrelated to this PR)

---

Last updated: 2026-02-27T18:30:00Z

[codeflash-ai]

This PR has been automatically closed because the original PR #1691 by codeflash-ai[bot] was closed.