Coalesce hash repartition batches using BatchCoalescer by Dandandan · Pull Request #20741 · apache/datafusion

Dandandan · 2026-03-06T10:17:40Z

Summary

Use coalesce batches during repartition, not when pulling repartitions. Now we can avoid skipping yielding every partition batches, as the upstream plan will have some batch to work with (rather than really small batches).

Dandandan · 2026-03-06T10:19:25Z

run benchmarks

alamb-ghbot · 2026-03-06T10:19:35Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (15f10c8) to d025869 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

In hash repartitioning, input batches are split into sub-batches per output partition. With many partitions, these sub-batches can be very small (e.g. 8192/16 = ~512 rows). Previously each small sub-batch was sent immediately through the channel. This change uses Arrow's BatchCoalescer per output partition to accumulate rows via push_batch_with_indices until target_batch_size is reached before sending. This reduces channel traffic and produces properly-sized output batches. For hash repartitioning, receiver-side coalescing is removed since the sender now produces properly-sized batches. Round-robin and preserve-order modes retain their existing coalescing behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

alamb-ghbot · 2026-03-06T11:00:29Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query    ┃        HEAD ┃ repartition-coalesce-hash ┃    Change ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 0 │  2366.03 ms │                2417.31 ms │ no change │
│ QQuery 1 │   910.67 ms │                 902.27 ms │ no change │
│ QQuery 2 │  1811.94 ms │                1775.43 ms │ no change │
│ QQuery 3 │  1141.10 ms │                1111.98 ms │ no change │
│ QQuery 4 │  2392.50 ms │                2348.34 ms │ no change │
│ QQuery 5 │ 28440.60 ms │               28995.04 ms │ no change │
│ QQuery 6 │  3912.91 ms │                3941.51 ms │ no change │
│ QQuery 7 │  2790.70 ms │                2723.19 ms │ no change │
└──────────┴─────────────┴───────────────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 43766.47ms │
│ Total Time (repartition-coalesce-hash)   │ 44215.07ms │
│ Average Time (HEAD)                      │  5470.81ms │
│ Average Time (repartition-coalesce-hash) │  5526.88ms │
│ Queries Faster                           │          0 │
│ Queries Slower                           │          0 │
│ Queries with No Change                   │          8 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │     2.56 ms │                   2.65 ms │     no change │
│ QQuery 1  │    50.43 ms │                  48.49 ms │     no change │
│ QQuery 2  │   164.25 ms │                 164.80 ms │     no change │
│ QQuery 3  │   171.60 ms │                 169.43 ms │     no change │
│ QQuery 4  │  1059.49 ms │                1122.80 ms │  1.06x slower │
│ QQuery 5  │  1284.74 ms │                1321.70 ms │     no change │
│ QQuery 6  │     7.11 ms │                   8.11 ms │  1.14x slower │
│ QQuery 7  │    55.44 ms │                  56.57 ms │     no change │
│ QQuery 8  │  1523.19 ms │                1517.45 ms │     no change │
│ QQuery 9  │  1940.19 ms │                1850.98 ms │     no change │
│ QQuery 10 │   341.37 ms │                 337.78 ms │     no change │
│ QQuery 11 │   384.97 ms │                 394.86 ms │     no change │
│ QQuery 12 │  1214.28 ms │                1251.03 ms │     no change │
│ QQuery 13 │  2043.85 ms │                2027.73 ms │     no change │
│ QQuery 14 │  1233.25 ms │                1254.63 ms │     no change │
│ QQuery 15 │  1267.99 ms │                1246.07 ms │     no change │
│ QQuery 16 │  2600.94 ms │                2624.79 ms │     no change │
│ QQuery 17 │  2590.22 ms │                2624.63 ms │     no change │
│ QQuery 18 │  5187.66 ms │                5296.85 ms │     no change │
│ QQuery 19 │   126.23 ms │                 124.83 ms │     no change │
│ QQuery 20 │  1859.84 ms │                1898.62 ms │     no change │
│ QQuery 21 │  2135.01 ms │                2107.54 ms │     no change │
│ QQuery 22 │  3711.83 ms │                3703.33 ms │     no change │
│ QQuery 23 │ 15882.64 ms │               11966.06 ms │ +1.33x faster │
│ QQuery 24 │   199.51 ms │                 186.49 ms │ +1.07x faster │
│ QQuery 25 │   431.94 ms │                 425.99 ms │     no change │
│ QQuery 26 │   204.18 ms │                 191.71 ms │ +1.07x faster │
│ QQuery 27 │  2750.94 ms │                2719.16 ms │     no change │
│ QQuery 28 │ 23245.74 ms │               23201.77 ms │     no change │
│ QQuery 29 │  1060.14 ms │                1015.37 ms │     no change │
│ QQuery 30 │  1265.02 ms │                1249.28 ms │     no change │
│ QQuery 31 │  1370.85 ms │                1301.52 ms │ +1.05x faster │
│ QQuery 32 │  4823.23 ms │                4839.70 ms │     no change │
│ QQuery 33 │  6007.25 ms │                5924.39 ms │     no change │
│ QQuery 34 │  6096.55 ms │                6175.37 ms │     no change │
│ QQuery 35 │  2002.70 ms │                2034.92 ms │     no change │
│ QQuery 36 │   184.85 ms │                 197.79 ms │  1.07x slower │
│ QQuery 37 │    73.20 ms │                  70.83 ms │     no change │
│ QQuery 38 │   111.64 ms │                 109.34 ms │     no change │
│ QQuery 39 │   327.61 ms │                 352.25 ms │  1.08x slower │
│ QQuery 40 │    40.64 ms │                  40.54 ms │     no change │
│ QQuery 41 │    36.24 ms │                  34.77 ms │     no change │
│ QQuery 42 │    30.07 ms │                  31.43 ms │     no change │
└───────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 97101.36ms │
│ Total Time (repartition-coalesce-hash)   │ 93224.37ms │
│ Average Time (HEAD)                      │  2258.17ms │
│ Average Time (repartition-coalesce-hash) │  2168.01ms │
│ Queries Faster                           │          4 │
│ Queries Slower                           │          4 │
│ Queries with No Change                   │         35 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 124.31 ms │                 129.05 ms │     no change │
│ QQuery 2  │  32.65 ms │                  33.46 ms │     no change │
│ QQuery 3  │  35.11 ms │                  39.52 ms │  1.13x slower │
│ QQuery 4  │  35.80 ms │                  33.09 ms │ +1.08x faster │
│ QQuery 5  │  92.97 ms │                  92.26 ms │     no change │
│ QQuery 6  │  24.17 ms │                  24.90 ms │     no change │
│ QQuery 7  │ 157.35 ms │                 162.09 ms │     no change │
│ QQuery 8  │  40.41 ms │                  42.44 ms │  1.05x slower │
│ QQuery 9  │ 113.31 ms │                 112.15 ms │     no change │
│ QQuery 10 │  71.26 ms │                  74.69 ms │     no change │
│ QQuery 11 │  18.68 ms │                  18.97 ms │     no change │
│ QQuery 12 │  64.39 ms │                  64.36 ms │     no change │
│ QQuery 13 │  53.26 ms │                  53.11 ms │     no change │
│ QQuery 14 │  16.27 ms │                  16.34 ms │     no change │
│ QQuery 15 │  33.32 ms │                  33.90 ms │     no change │
│ QQuery 16 │  29.87 ms │                  30.15 ms │     no change │
│ QQuery 17 │ 167.46 ms │                 167.69 ms │     no change │
│ QQuery 18 │ 302.76 ms │                 290.01 ms │     no change │
│ QQuery 19 │  51.96 ms │                  53.77 ms │     no change │
│ QQuery 20 │  63.12 ms │                  61.68 ms │     no change │
│ QQuery 21 │ 199.02 ms │                 195.09 ms │     no change │
│ QQuery 22 │  24.67 ms │                  24.74 ms │     no change │
└───────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1752.11ms │
│ Total Time (repartition-coalesce-hash)   │ 1753.50ms │
│ Average Time (HEAD)                      │   79.64ms │
│ Average Time (repartition-coalesce-hash) │   79.70ms │
│ Queries Faster                           │         1 │
│ Queries Slower                           │         2 │
│ Queries with No Change                   │        19 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

Dandandan · 2026-03-06T11:05:01Z

run benchmark tpch

alamb-ghbot · 2026-03-06T11:05:11Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (ef571e2) to d025869 diff using: tpch
Results will be posted here when complete

alamb-ghbot · 2026-03-06T11:18:36Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 173.15 ms │                 180.52 ms │     no change │
│ QQuery 2  │  88.29 ms │                  86.30 ms │     no change │
│ QQuery 3  │ 119.15 ms │                 117.86 ms │     no change │
│ QQuery 4  │  80.68 ms │                  79.55 ms │     no change │
│ QQuery 5  │ 176.65 ms │                 177.78 ms │     no change │
│ QQuery 6  │  63.90 ms │                  65.08 ms │     no change │
│ QQuery 7  │ 207.68 ms │                 205.78 ms │     no change │
│ QQuery 8  │ 166.20 ms │                 163.09 ms │     no change │
│ QQuery 9  │ 234.11 ms │                 236.96 ms │     no change │
│ QQuery 10 │ 188.18 ms │                 203.49 ms │  1.08x slower │
│ QQuery 11 │  62.04 ms │                  62.84 ms │     no change │
│ QQuery 12 │ 117.54 ms │                 115.17 ms │     no change │
│ QQuery 13 │ 238.58 ms │                 210.83 ms │ +1.13x faster │
│ QQuery 14 │  89.43 ms │                  94.31 ms │  1.05x slower │
│ QQuery 15 │ 119.67 ms │                 119.75 ms │     no change │
│ QQuery 16 │  57.43 ms │                  62.12 ms │  1.08x slower │
│ QQuery 17 │ 270.54 ms │                 263.86 ms │     no change │
│ QQuery 18 │ 337.93 ms │                 328.85 ms │     no change │
│ QQuery 19 │ 124.69 ms │                 132.55 ms │  1.06x slower │
│ QQuery 20 │ 129.86 ms │                 130.22 ms │     no change │
│ QQuery 21 │ 262.88 ms │                 255.64 ms │     no change │
│ QQuery 22 │  41.14 ms │                  38.59 ms │ +1.07x faster │
└───────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 3349.71ms │
│ Total Time (repartition-coalesce-hash)   │ 3331.15ms │
│ Average Time (HEAD)                      │  152.26ms │
│ Average Time (repartition-coalesce-hash) │  151.42ms │
│ Queries Faster                           │         2 │
│ Queries Slower                           │         4 │
│ Queries with No Change                   │        16 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

Dandandan · 2026-03-06T11:36:15Z

run benchmarks

Dandandan · 2026-03-06T11:36:19Z

run benchmark tpch

alamb-ghbot · 2026-03-06T11:36:21Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (c47210b) to d025869 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb-ghbot · 2026-03-06T12:14:35Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query    ┃        HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │  2427.61 ms │                2339.72 ms │     no change │
│ QQuery 1 │   905.91 ms │                 921.17 ms │     no change │
│ QQuery 2 │  1748.55 ms │                1845.56 ms │  1.06x slower │
│ QQuery 3 │  1142.07 ms │                1151.06 ms │     no change │
│ QQuery 4 │  2391.49 ms │                2230.34 ms │ +1.07x faster │
│ QQuery 5 │ 28009.62 ms │               27823.60 ms │     no change │
│ QQuery 6 │  4020.80 ms │                4214.34 ms │     no change │
│ QQuery 7 │  2908.08 ms │                2607.34 ms │ +1.12x faster │
└──────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 43554.12ms │
│ Total Time (repartition-coalesce-hash)   │ 43133.13ms │
│ Average Time (HEAD)                      │  5444.26ms │
│ Average Time (repartition-coalesce-hash) │  5391.64ms │
│ Queries Faster                           │          2 │
│ Queries Slower                           │          1 │
│ Queries with No Change                   │          5 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │     2.63 ms │                   2.62 ms │     no change │
│ QQuery 1  │    50.60 ms │                  49.78 ms │     no change │
│ QQuery 2  │   168.00 ms │                 164.22 ms │     no change │
│ QQuery 3  │   171.80 ms │                 171.66 ms │     no change │
│ QQuery 4  │  1046.19 ms │                1089.51 ms │     no change │
│ QQuery 5  │  1310.29 ms │                1297.10 ms │     no change │
│ QQuery 6  │     6.18 ms │                   6.63 ms │  1.07x slower │
│ QQuery 7  │    54.74 ms │                  54.94 ms │     no change │
│ QQuery 8  │  1483.11 ms │                1492.08 ms │     no change │
│ QQuery 9  │  1833.06 ms │                1805.51 ms │     no change │
│ QQuery 10 │   347.94 ms │                 352.27 ms │     no change │
│ QQuery 11 │   397.66 ms │                 394.01 ms │     no change │
│ QQuery 12 │  1210.75 ms │                1256.93 ms │     no change │
│ QQuery 13 │  1952.65 ms │                1919.80 ms │     no change │
│ QQuery 14 │  1247.23 ms │                1249.10 ms │     no change │
│ QQuery 15 │  1241.17 ms │                1254.65 ms │     no change │
│ QQuery 16 │  2579.47 ms │                2625.02 ms │     no change │
│ QQuery 17 │  2561.73 ms │                2638.35 ms │     no change │
│ QQuery 18 │  6009.55 ms │                5198.03 ms │ +1.16x faster │
│ QQuery 19 │   126.26 ms │                 123.55 ms │     no change │
│ QQuery 20 │  1921.82 ms │                1945.47 ms │     no change │
│ QQuery 21 │  2131.26 ms │                2325.40 ms │  1.09x slower │
│ QQuery 22 │  3905.86 ms │                3941.19 ms │     no change │
│ QQuery 23 │ 11905.79 ms │               12260.35 ms │     no change │
│ QQuery 24 │   199.92 ms │                 197.09 ms │     no change │
│ QQuery 25 │   436.16 ms │                 451.66 ms │     no change │
│ QQuery 26 │   197.09 ms │                 205.40 ms │     no change │
│ QQuery 27 │  2727.61 ms │                2861.92 ms │     no change │
│ QQuery 28 │ 22981.98 ms │               25933.05 ms │  1.13x slower │
│ QQuery 29 │  1080.29 ms │                1034.56 ms │     no change │
│ QQuery 30 │  1243.67 ms │                1249.28 ms │     no change │
│ QQuery 31 │  1375.08 ms │                1263.34 ms │ +1.09x faster │
│ QQuery 32 │  4837.11 ms │                4245.75 ms │ +1.14x faster │
│ QQuery 33 │  6123.27 ms │                6011.58 ms │     no change │
│ QQuery 34 │  6264.43 ms │                6515.29 ms │     no change │
│ QQuery 35 │  2004.88 ms │                2047.18 ms │     no change │
│ QQuery 36 │   186.47 ms │                 200.46 ms │  1.07x slower │
│ QQuery 37 │    70.79 ms │                  75.44 ms │  1.07x slower │
│ QQuery 38 │   108.14 ms │                 115.63 ms │  1.07x slower │
│ QQuery 39 │   336.75 ms │                 365.10 ms │  1.08x slower │
│ QQuery 40 │    40.38 ms │                  42.50 ms │  1.05x slower │
│ QQuery 41 │    33.84 ms │                  36.62 ms │  1.08x slower │
│ QQuery 42 │    32.35 ms │                  30.82 ms │     no change │
└───────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 93945.97ms │
│ Total Time (repartition-coalesce-hash)   │ 96500.82ms │
│ Average Time (HEAD)                      │  2184.79ms │
│ Average Time (repartition-coalesce-hash) │  2244.21ms │
│ Queries Faster                           │          3 │
│ Queries Slower                           │          9 │
│ Queries with No Change                   │         31 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 126.46 ms │                 130.76 ms │     no change │
│ QQuery 2  │  32.85 ms │                  32.82 ms │     no change │
│ QQuery 3  │  40.48 ms │                  39.49 ms │     no change │
│ QQuery 4  │  35.81 ms │                  32.35 ms │ +1.11x faster │
│ QQuery 5  │  93.78 ms │                  88.24 ms │ +1.06x faster │
│ QQuery 6  │  25.10 ms │                  25.22 ms │     no change │
│ QQuery 7  │ 155.56 ms │                 147.09 ms │ +1.06x faster │
│ QQuery 8  │  41.74 ms │                  40.67 ms │     no change │
│ QQuery 9  │ 112.84 ms │                 111.62 ms │     no change │
│ QQuery 10 │  73.22 ms │                  71.69 ms │     no change │
│ QQuery 11 │  18.81 ms │                  18.63 ms │     no change │
│ QQuery 12 │  65.84 ms │                  65.37 ms │     no change │
│ QQuery 13 │  53.94 ms │                  51.66 ms │     no change │
│ QQuery 14 │  16.03 ms │                  15.81 ms │     no change │
│ QQuery 15 │  33.36 ms │                  33.70 ms │     no change │
│ QQuery 16 │  29.75 ms │                  29.54 ms │     no change │
│ QQuery 17 │ 168.51 ms │                 163.66 ms │     no change │
│ QQuery 18 │ 295.81 ms │                 309.91 ms │     no change │
│ QQuery 19 │  51.80 ms │                  54.09 ms │     no change │
│ QQuery 20 │  61.80 ms │                  60.95 ms │     no change │
│ QQuery 21 │ 203.76 ms │                 192.86 ms │ +1.06x faster │
│ QQuery 22 │  25.27 ms │                  24.53 ms │     no change │
└───────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1762.53ms │
│ Total Time (repartition-coalesce-hash)   │ 1740.64ms │
│ Average Time (HEAD)                      │   80.12ms │
│ Average Time (repartition-coalesce-hash) │   79.12ms │
│ Queries Faster                           │         4 │
│ Queries Slower                           │         0 │
│ Queries with No Change                   │        18 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

alamb-ghbot · 2026-03-06T12:15:04Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (20d15e2) to d025869 diff using: tpch
Results will be posted here when complete

alamb-ghbot · 2026-03-06T12:28:29Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 195.04 ms │                 197.46 ms │     no change │
│ QQuery 2  │  86.19 ms │                  82.78 ms │     no change │
│ QQuery 3  │ 123.06 ms │                 119.87 ms │     no change │
│ QQuery 4  │  80.42 ms │                  74.94 ms │ +1.07x faster │
│ QQuery 5  │ 186.20 ms │                 172.17 ms │ +1.08x faster │
│ QQuery 6  │  67.78 ms │                  66.94 ms │     no change │
│ QQuery 7  │ 212.96 ms │                 204.55 ms │     no change │
│ QQuery 8  │ 167.28 ms │                 166.39 ms │     no change │
│ QQuery 9  │ 231.08 ms │                 225.96 ms │     no change │
│ QQuery 10 │ 187.57 ms │                 201.75 ms │  1.08x slower │
│ QQuery 11 │  60.55 ms │                  60.06 ms │     no change │
│ QQuery 12 │ 117.34 ms │                 119.80 ms │     no change │
│ QQuery 13 │ 245.12 ms │                 205.68 ms │ +1.19x faster │
│ QQuery 14 │  83.33 ms │                  92.69 ms │  1.11x slower │
│ QQuery 15 │ 119.30 ms │                 121.93 ms │     no change │
│ QQuery 16 │  62.78 ms │                  60.44 ms │     no change │
│ QQuery 17 │ 271.59 ms │                 268.99 ms │     no change │
│ QQuery 18 │ 328.20 ms │                 323.66 ms │     no change │
│ QQuery 19 │ 133.36 ms │                 126.42 ms │ +1.05x faster │
│ QQuery 20 │ 127.43 ms │                 123.30 ms │     no change │
│ QQuery 21 │ 263.25 ms │                 249.44 ms │ +1.06x faster │
│ QQuery 22 │  39.49 ms │                  42.10 ms │  1.07x slower │
└───────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 3389.32ms │
│ Total Time (repartition-coalesce-hash)   │ 3307.31ms │
│ Average Time (HEAD)                      │  154.06ms │
│ Average Time (repartition-coalesce-hash) │  150.33ms │
│ Queries Faster                           │         5 │
│ Queries Slower                           │         3 │
│ Queries with No Change                   │        14 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

Dandandan · 2026-03-06T12:51:31Z

run benchmark tpch_mem tpch tpcds

alamb-ghbot · 2026-03-06T12:51:39Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (20d15e2) to d025869 diff using: tpch_mem
Results will be posted here when complete

alamb-ghbot · 2026-03-06T12:52:03Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 130.72 ms │                 128.93 ms │     no change │
│ QQuery 2  │  32.67 ms │                  32.65 ms │     no change │
│ QQuery 3  │  39.65 ms │                  39.07 ms │     no change │
│ QQuery 4  │  35.39 ms │                  31.98 ms │ +1.11x faster │
│ QQuery 5  │  89.84 ms │                  90.56 ms │     no change │
│ QQuery 6  │  24.93 ms │                  25.48 ms │     no change │
│ QQuery 7  │ 153.59 ms │                 158.93 ms │     no change │
│ QQuery 8  │  41.00 ms │                  39.52 ms │     no change │
│ QQuery 9  │ 109.94 ms │                 111.36 ms │     no change │
│ QQuery 10 │  71.22 ms │                  69.14 ms │     no change │
│ QQuery 11 │  19.02 ms │                  19.40 ms │     no change │
│ QQuery 12 │  68.34 ms │                  67.69 ms │     no change │
│ QQuery 13 │  53.96 ms │                  48.04 ms │ +1.12x faster │
│ QQuery 14 │  15.38 ms │                  16.13 ms │     no change │
│ QQuery 15 │  32.86 ms │                  33.04 ms │     no change │
│ QQuery 16 │  30.52 ms │                  30.53 ms │     no change │
│ QQuery 17 │ 170.33 ms │                 159.85 ms │ +1.07x faster │
│ QQuery 18 │ 312.08 ms │                 321.54 ms │     no change │
│ QQuery 19 │  52.55 ms │                  54.96 ms │     no change │
│ QQuery 20 │  62.48 ms │                  61.28 ms │     no change │
│ QQuery 21 │ 199.88 ms │                 196.41 ms │     no change │
│ QQuery 22 │  25.29 ms │                  24.53 ms │     no change │
└───────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1771.63ms │
│ Total Time (repartition-coalesce-hash)   │ 1761.04ms │
│ Average Time (HEAD)                      │   80.53ms │
│ Average Time (repartition-coalesce-hash) │   80.05ms │
│ Queries Faster                           │         3 │
│ Queries Slower                           │         0 │
│ Queries with No Change                   │        19 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

alamb-ghbot · 2026-03-06T12:52:05Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (20d15e2) to d025869 diff using: tpch
Results will be posted here when complete

alamb-ghbot · 2026-03-06T12:52:42Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 191.46 ms │                 190.09 ms │     no change │
│ QQuery 2  │  84.05 ms │                  83.89 ms │     no change │
│ QQuery 3  │ 119.79 ms │                 118.36 ms │     no change │
│ QQuery 4  │  79.84 ms │                  71.98 ms │ +1.11x faster │
│ QQuery 5  │ 181.83 ms │                 168.16 ms │ +1.08x faster │
│ QQuery 6  │  64.87 ms │                  65.97 ms │     no change │
│ QQuery 7  │ 208.16 ms │                 198.81 ms │     no change │
│ QQuery 8  │ 170.45 ms │                 157.21 ms │ +1.08x faster │
│ QQuery 9  │ 233.85 ms │                 220.63 ms │ +1.06x faster │
│ QQuery 10 │ 185.11 ms │                 200.27 ms │  1.08x slower │
│ QQuery 11 │  58.99 ms │                  62.58 ms │  1.06x slower │
│ QQuery 12 │ 119.41 ms │                 116.31 ms │     no change │
│ QQuery 13 │ 227.30 ms │                 200.67 ms │ +1.13x faster │
│ QQuery 14 │  88.41 ms │                  92.89 ms │  1.05x slower │
│ QQuery 15 │ 119.14 ms │                 120.77 ms │     no change │
│ QQuery 16 │  56.93 ms │                  60.50 ms │  1.06x slower │
│ QQuery 17 │ 263.97 ms │                 267.54 ms │     no change │
│ QQuery 18 │ 330.58 ms │                 310.18 ms │ +1.07x faster │
│ QQuery 19 │ 133.63 ms │                 126.02 ms │ +1.06x faster │
│ QQuery 20 │ 126.09 ms │                 120.99 ms │     no change │
│ QQuery 21 │ 257.82 ms │                 250.09 ms │     no change │
│ QQuery 22 │  39.67 ms │                  39.34 ms │     no change │
└───────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 3341.36ms │
│ Total Time (repartition-coalesce-hash)   │ 3243.25ms │
│ Average Time (HEAD)                      │  151.88ms │
│ Average Time (repartition-coalesce-hash) │  147.42ms │
│ Queries Faster                           │         7 │
│ Queries Slower                           │         4 │
│ Queries with No Change                   │        11 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

alamb-ghbot · 2026-03-06T12:52:44Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (20d15e2) to d025869 diff using: tpcds
Results will be posted here when complete

Dandandan · 2026-03-06T12:57:58Z

Looks slightly faster with the (now I think unnecessary) yielding removed.

alamb-ghbot · 2026-03-06T13:01:32Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │    71.74 ms │                  72.69 ms │     no change │
│ QQuery 2  │   230.29 ms │                 230.42 ms │     no change │
│ QQuery 3  │   155.93 ms │                 153.43 ms │     no change │
│ QQuery 4  │  1873.93 ms │                1878.78 ms │     no change │
│ QQuery 5  │   262.28 ms │                 264.49 ms │     no change │
│ QQuery 6  │  1451.24 ms │                1425.78 ms │     no change │
│ QQuery 7  │   504.33 ms │                 469.70 ms │ +1.07x faster │
│ QQuery 8  │   170.59 ms │                 171.94 ms │     no change │
│ QQuery 9  │   305.48 ms │                 309.05 ms │     no change │
│ QQuery 10 │   162.53 ms │                 164.45 ms │     no change │
│ QQuery 11 │  1255.09 ms │                1245.39 ms │     no change │
│ QQuery 12 │    64.20 ms │                  71.98 ms │  1.12x slower │
│ QQuery 13 │   548.00 ms │                 549.51 ms │     no change │
│ QQuery 14 │  1928.46 ms │                1907.55 ms │     no change │
│ QQuery 15 │    29.44 ms │                  28.32 ms │     no change │
│ QQuery 16 │    65.64 ms │                  63.88 ms │     no change │
│ QQuery 17 │   363.65 ms │                 327.65 ms │ +1.11x faster │
│ QQuery 18 │   192.26 ms │                 181.38 ms │ +1.06x faster │
│ QQuery 19 │   211.98 ms │                 212.90 ms │     no change │
│ QQuery 20 │    22.71 ms │                  25.82 ms │  1.14x slower │
│ QQuery 21 │    36.11 ms │                  35.12 ms │     no change │
│ QQuery 22 │   714.59 ms │                 715.67 ms │     no change │
│ QQuery 23 │  1907.69 ms │                1812.67 ms │     no change │
│ QQuery 24 │   679.18 ms │                 625.65 ms │ +1.09x faster │
│ QQuery 25 │   527.64 ms │                 493.54 ms │ +1.07x faster │
│ QQuery 26 │   122.99 ms │                 118.68 ms │     no change │
│ QQuery 27 │   498.61 ms │                 462.64 ms │ +1.08x faster │
│ QQuery 28 │   339.62 ms │                 336.09 ms │     no change │
│ QQuery 29 │   454.65 ms │                 412.91 ms │ +1.10x faster │
│ QQuery 30 │    74.45 ms │                  69.16 ms │ +1.08x faster │
│ QQuery 31 │   313.25 ms │                 311.99 ms │     no change │
│ QQuery 32 │    80.66 ms │                  80.12 ms │     no change │
│ QQuery 33 │   203.15 ms │                 207.70 ms │     no change │
│ QQuery 34 │   154.53 ms │                 156.72 ms │     no change │
│ QQuery 35 │   171.84 ms │                 161.90 ms │ +1.06x faster │
│ QQuery 36 │   285.68 ms │                 284.85 ms │     no change │
│ QQuery 37 │   254.27 ms │                 252.31 ms │     no change │
│ QQuery 38 │   148.90 ms │                 152.14 ms │     no change │
│ QQuery 39 │   195.87 ms │                 202.03 ms │     no change │
│ QQuery 40 │   181.54 ms │                 171.87 ms │ +1.06x faster │
│ QQuery 41 │    26.34 ms │                  27.07 ms │     no change │
│ QQuery 42 │   142.61 ms │                 141.68 ms │     no change │
│ QQuery 43 │   122.99 ms │                 124.46 ms │     no change │
│ QQuery 44 │    28.09 ms │                  27.78 ms │     no change │
│ QQuery 45 │    85.99 ms │                  79.50 ms │ +1.08x faster │
│ QQuery 46 │   313.59 ms │                 322.20 ms │     no change │
│ QQuery 47 │  1028.14 ms │                1027.75 ms │     no change │
│ QQuery 48 │   406.36 ms │                 410.19 ms │     no change │
│ QQuery 49 │   370.45 ms │                 367.25 ms │     no change │
│ QQuery 50 │   337.97 ms │                 309.49 ms │ +1.09x faster │
│ QQuery 51 │   317.87 ms │                 314.19 ms │     no change │
│ QQuery 52 │   145.91 ms │                 144.24 ms │     no change │
│ QQuery 53 │   144.83 ms │                 145.25 ms │     no change │
│ QQuery 54 │   201.69 ms │                 203.77 ms │     no change │
│ QQuery 55 │   144.22 ms │                 144.18 ms │     no change │
│ QQuery 56 │   204.21 ms │                 202.94 ms │     no change │
│ QQuery 57 │   288.76 ms │                 294.67 ms │     no change │
│ QQuery 58 │   495.22 ms │                 536.56 ms │  1.08x slower │
│ QQuery 59 │   307.63 ms │                 314.71 ms │     no change │
│ QQuery 60 │   208.73 ms │                 209.97 ms │     no change │
│ QQuery 61 │   245.38 ms │                 247.66 ms │     no change │
│ QQuery 62 │  1364.78 ms │                1339.96 ms │     no change │
│ QQuery 63 │   145.77 ms │                 147.19 ms │     no change │
│ QQuery 64 │  1153.77 ms │                1065.94 ms │ +1.08x faster │
│ QQuery 65 │   356.45 ms │                 350.81 ms │     no change │
│ QQuery 66 │   388.70 ms │                 374.55 ms │     no change │
│ QQuery 67 │   540.58 ms │                 536.69 ms │     no change │
│ QQuery 68 │   375.54 ms │                 370.93 ms │     no change │
│ QQuery 69 │   151.81 ms │                 158.00 ms │     no change │
│ QQuery 70 │   483.94 ms │                 496.88 ms │     no change │
│ QQuery 71 │   185.47 ms │                 182.83 ms │     no change │
│ QQuery 72 │  2183.17 ms │                1864.74 ms │ +1.17x faster │
│ QQuery 73 │   150.67 ms │                 152.83 ms │     no change │
│ QQuery 74 │   788.27 ms │                 786.54 ms │     no change │
│ QQuery 75 │   413.72 ms │                 420.79 ms │     no change │
│ QQuery 76 │   181.39 ms │                 186.03 ms │     no change │
│ QQuery 77 │   276.64 ms │                 281.61 ms │     no change │
│ QQuery 78 │   605.48 ms │                 563.46 ms │ +1.07x faster │
│ QQuery 79 │   316.28 ms │                 322.57 ms │     no change │
│ QQuery 80 │   518.51 ms │                 483.63 ms │ +1.07x faster │
│ QQuery 81 │    54.84 ms │                  54.77 ms │     no change │
│ QQuery 82 │   279.39 ms │                 272.14 ms │     no change │
│ QQuery 83 │    76.68 ms │                  78.73 ms │     no change │
│ QQuery 84 │    66.64 ms │                  64.08 ms │     no change │
│ QQuery 85 │   209.65 ms │                 212.88 ms │     no change │
│ QQuery 86 │    58.04 ms │                  57.59 ms │     no change │
│ QQuery 87 │   155.49 ms │                 151.21 ms │     no change │
│ QQuery 88 │   293.11 ms │                 293.69 ms │     no change │
│ QQuery 89 │   160.41 ms │                 165.73 ms │     no change │
│ QQuery 90 │    46.83 ms │                  46.74 ms │     no change │
│ QQuery 91 │    87.83 ms │                  92.96 ms │  1.06x slower │
│ QQuery 92 │    80.72 ms │                  82.75 ms │     no change │
│ QQuery 93 │   290.88 ms │                 268.27 ms │ +1.08x faster │
│ QQuery 94 │    91.67 ms │                  91.53 ms │     no change │
│ QQuery 95 │   254.92 ms │                 229.09 ms │ +1.11x faster │
│ QQuery 96 │   110.37 ms │                 114.21 ms │     no change │
│ QQuery 97 │   194.85 ms │                 185.30 ms │     no change │
│ QQuery 98 │   206.86 ms │                 209.12 ms │     no change │
│ QQuery 99 │ 15880.46 ms │               15916.30 ms │     no change │
└───────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 51962.58ms │
│ Total Time (repartition-coalesce-hash)   │ 51087.46ms │
│ Average Time (HEAD)                      │   524.87ms │
│ Average Time (repartition-coalesce-hash) │   516.03ms │
│ Queries Faster                           │         18 │
│ Queries Slower                           │          4 │
│ Queries with No Change                   │         77 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘

Dandandan · 2026-03-06T13:08:03Z

run benchmarks

alamb-ghbot · 2026-03-06T13:08:10Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (20d15e2) to d025869 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb-ghbot · 2026-03-06T13:33:47Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query    ┃        HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │  2427.83 ms │                2485.70 ms │     no change │
│ QQuery 1 │   896.09 ms │                 906.86 ms │     no change │
│ QQuery 2 │  1806.20 ms │                1909.54 ms │  1.06x slower │
│ QQuery 3 │  1118.86 ms │                1118.30 ms │     no change │
│ QQuery 4 │  2371.48 ms │                2194.60 ms │ +1.08x faster │
│ QQuery 5 │ 27705.89 ms │               27640.55 ms │     no change │
│ QQuery 6 │  3973.61 ms │                4183.86 ms │  1.05x slower │
│ QQuery 7 │  2739.66 ms │                3022.96 ms │  1.10x slower │
└──────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 43039.62ms │
│ Total Time (repartition-coalesce-hash)   │ 43462.37ms │
│ Average Time (HEAD)                      │  5379.95ms │
│ Average Time (repartition-coalesce-hash) │  5432.80ms │
│ Queries Faster                           │          1 │
│ Queries Slower                           │          3 │
│ Queries with No Change                   │          4 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │     2.58 ms │                   2.67 ms │     no change │
│ QQuery 1  │    50.95 ms │                  52.25 ms │     no change │
│ QQuery 2  │   167.36 ms │                 167.33 ms │     no change │
│ QQuery 3  │   170.24 ms │                 169.49 ms │     no change │
│ QQuery 4  │  1081.62 ms │                1069.96 ms │     no change │
│ QQuery 5  │  1269.74 ms │                1296.79 ms │     no change │
│ QQuery 6  │     6.80 ms │                   7.00 ms │     no change │
│ QQuery 7  │    53.93 ms │                  57.15 ms │  1.06x slower │
│ QQuery 8  │  1514.94 ms │                1456.61 ms │     no change │
│ QQuery 9  │  1853.85 ms │                1894.14 ms │     no change │
│ QQuery 10 │   337.29 ms │                 345.56 ms │     no change │
│ QQuery 11 │   393.48 ms │                 398.58 ms │     no change │
│ QQuery 12 │  1199.98 ms │                1216.77 ms │     no change │
│ QQuery 13 │  1994.35 ms │                1959.10 ms │     no change │
│ QQuery 14 │  1237.24 ms │                1230.56 ms │     no change │
│ QQuery 15 │  1237.68 ms │                1262.69 ms │     no change │
│ QQuery 16 │  2563.73 ms │                2604.74 ms │     no change │
│ QQuery 17 │  2529.50 ms │                2611.42 ms │     no change │
│ QQuery 18 │  5711.70 ms │                5310.94 ms │ +1.08x faster │
│ QQuery 19 │   126.01 ms │                 130.08 ms │     no change │
│ QQuery 20 │  1833.81 ms │                1991.89 ms │  1.09x slower │
│ QQuery 21 │  2097.95 ms │                2283.87 ms │  1.09x slower │
│ QQuery 22 │  3741.77 ms │                3948.08 ms │  1.06x slower │
│ QQuery 23 │ 18459.54 ms │               12208.23 ms │ +1.51x faster │
│ QQuery 24 │   202.81 ms │                 199.60 ms │     no change │
│ QQuery 25 │   438.07 ms │                 451.58 ms │     no change │
│ QQuery 26 │   199.12 ms │                 210.67 ms │  1.06x slower │
│ QQuery 27 │  2736.21 ms │                2828.07 ms │     no change │
│ QQuery 28 │ 22678.58 ms │               25602.33 ms │  1.13x slower │
│ QQuery 29 │  1034.75 ms │                1067.64 ms │     no change │
│ QQuery 30 │  1249.97 ms │                1249.33 ms │     no change │
│ QQuery 31 │  1371.34 ms │                1256.14 ms │ +1.09x faster │
│ QQuery 32 │  4941.87 ms │                4603.32 ms │ +1.07x faster │
│ QQuery 33 │  5928.98 ms │                6168.08 ms │     no change │
│ QQuery 34 │  6475.96 ms │                6402.30 ms │     no change │
│ QQuery 35 │  1981.35 ms │                2012.44 ms │     no change │
│ QQuery 36 │   188.67 ms │                 213.22 ms │  1.13x slower │
│ QQuery 37 │    71.15 ms │                  74.96 ms │  1.05x slower │
│ QQuery 38 │   109.71 ms │                 119.05 ms │  1.09x slower │
│ QQuery 39 │   345.08 ms │                 383.31 ms │  1.11x slower │
│ QQuery 40 │    40.34 ms │                  48.12 ms │  1.19x slower │
│ QQuery 41 │    35.72 ms │                  36.13 ms │     no change │
│ QQuery 42 │    31.79 ms │                  32.64 ms │     no change │
└───────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 99697.52ms │
│ Total Time (repartition-coalesce-hash)   │ 96634.85ms │
│ Average Time (HEAD)                      │  2318.55ms │
│ Average Time (repartition-coalesce-hash) │  2247.32ms │
│ Queries Faster                           │          4 │
│ Queries Slower                           │         11 │
│ Queries with No Change                   │         28 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 128.48 ms │                 129.29 ms │     no change │
│ QQuery 2  │  32.38 ms │                  32.07 ms │     no change │
│ QQuery 3  │  39.40 ms │                  40.21 ms │     no change │
│ QQuery 4  │  36.76 ms │                  32.18 ms │ +1.14x faster │
│ QQuery 5  │  92.40 ms │                  88.94 ms │     no change │
│ QQuery 6  │  24.92 ms │                  24.69 ms │     no change │
│ QQuery 7  │ 155.62 ms │                 154.26 ms │     no change │
│ QQuery 8  │  41.98 ms │                  40.06 ms │     no change │
│ QQuery 9  │ 109.44 ms │                 114.10 ms │     no change │
│ QQuery 10 │  71.14 ms │                  70.12 ms │     no change │
│ QQuery 11 │  18.95 ms │                  18.77 ms │     no change │
│ QQuery 12 │  66.54 ms │                  64.53 ms │     no change │
│ QQuery 13 │  57.53 ms │                  51.91 ms │ +1.11x faster │
│ QQuery 14 │  15.39 ms │                  14.67 ms │     no change │
│ QQuery 15 │  33.59 ms │                  34.05 ms │     no change │
│ QQuery 16 │  30.15 ms │                  30.10 ms │     no change │
│ QQuery 17 │ 177.22 ms │                 175.91 ms │     no change │
│ QQuery 18 │ 306.79 ms │                 301.75 ms │     no change │
│ QQuery 19 │  48.12 ms │                  53.85 ms │  1.12x slower │
│ QQuery 20 │  63.23 ms │                  60.34 ms │     no change │
│ QQuery 21 │ 202.52 ms │                 191.27 ms │ +1.06x faster │
│ QQuery 22 │  25.91 ms │                  24.62 ms │     no change │
└───────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1778.48ms │
│ Total Time (repartition-coalesce-hash)   │ 1747.68ms │
│ Average Time (HEAD)                      │   80.84ms │
│ Average Time (repartition-coalesce-hash) │   79.44ms │
│ Queries Faster                           │         3 │
│ Queries Slower                           │         1 │
│ Queries with No Change                   │        18 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

…rtition Use LimitedBatchCoalescer on the sender side for all partitioning schemes, removing the separate receiver-side coalescing and the hash_partition_indices optimization. This simplifies the code significantly (-155 lines net). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Integrate LimitedBatchCoalescer into BatchPartitioner so hash repartitioning uses push_batch_with_indices directly, avoiding materializing intermediate sub-batches. Skip sender-side coalescing for preserve_order mode since the subsequent merge sort already does batching. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Dandandan · 2026-03-06T17:23:28Z

run benchmarks

Dandandan · 2026-03-06T17:23:38Z

run benchmark tpch tpch_mem

alamb-ghbot · 2026-03-06T18:21:53Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (d659edb) to d025869 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb-ghbot · 2026-03-06T19:02:40Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query    ┃        HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │  2414.11 ms │                2445.83 ms │     no change │
│ QQuery 1 │   900.28 ms │                 972.44 ms │  1.08x slower │
│ QQuery 2 │  1800.07 ms │                1869.45 ms │     no change │
│ QQuery 3 │  1148.22 ms │                1143.83 ms │     no change │
│ QQuery 4 │  2396.55 ms │                2269.31 ms │ +1.06x faster │
│ QQuery 5 │ 27750.02 ms │               27831.15 ms │     no change │
│ QQuery 6 │  3975.19 ms │                4192.60 ms │  1.05x slower │
│ QQuery 7 │  2902.33 ms │                2769.59 ms │     no change │
└──────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 43286.77ms │
│ Total Time (repartition-coalesce-hash)   │ 43494.21ms │
│ Average Time (HEAD)                      │  5410.85ms │
│ Average Time (repartition-coalesce-hash) │  5436.78ms │
│ Queries Faster                           │          1 │
│ Queries Slower                           │          2 │
│ Queries with No Change                   │          5 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃        HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │     2.58 ms │                   2.70 ms │     no change │
│ QQuery 1  │    50.66 ms │                  52.45 ms │     no change │
│ QQuery 2  │   160.20 ms │                 162.51 ms │     no change │
│ QQuery 3  │   172.85 ms │                 170.28 ms │     no change │
│ QQuery 4  │  1114.20 ms │                1140.65 ms │     no change │
│ QQuery 5  │  1286.69 ms │                1320.89 ms │     no change │
│ QQuery 6  │     6.29 ms │                   6.73 ms │  1.07x slower │
│ QQuery 7  │    59.74 ms │                  56.67 ms │ +1.05x faster │
│ QQuery 8  │  1520.05 ms │                1604.38 ms │  1.06x slower │
│ QQuery 9  │  1828.52 ms │                1835.94 ms │     no change │
│ QQuery 10 │   343.07 ms │                 368.23 ms │  1.07x slower │
│ QQuery 11 │   390.85 ms │                 412.30 ms │  1.05x slower │
│ QQuery 12 │  1227.92 ms │                1293.00 ms │  1.05x slower │
│ QQuery 13 │  1980.31 ms │                1969.83 ms │     no change │
│ QQuery 14 │  1276.14 ms │                1289.50 ms │     no change │
│ QQuery 15 │  1261.48 ms │                1252.13 ms │     no change │
│ QQuery 16 │  2611.15 ms │                2668.78 ms │     no change │
│ QQuery 17 │  2616.66 ms │                2633.25 ms │     no change │
│ QQuery 18 │  5225.29 ms │                5261.93 ms │     no change │
│ QQuery 19 │   125.47 ms │                 127.82 ms │     no change │
│ QQuery 20 │  1830.27 ms │                1936.76 ms │  1.06x slower │
│ QQuery 21 │  2125.08 ms │                2268.21 ms │  1.07x slower │
│ QQuery 22 │  3697.08 ms │                3860.54 ms │     no change │
│ QQuery 23 │ 11765.54 ms │               12546.38 ms │  1.07x slower │
│ QQuery 24 │   191.46 ms │                 197.90 ms │     no change │
│ QQuery 25 │   438.79 ms │                 447.71 ms │     no change │
│ QQuery 26 │   198.85 ms │                 207.02 ms │     no change │
│ QQuery 27 │  2683.77 ms │                2762.10 ms │     no change │
│ QQuery 28 │ 22499.68 ms │               24575.20 ms │  1.09x slower │
│ QQuery 29 │  1031.56 ms │                1045.86 ms │     no change │
│ QQuery 30 │  1285.71 ms │                1284.88 ms │     no change │
│ QQuery 31 │  1365.70 ms │                1277.34 ms │ +1.07x faster │
│ QQuery 32 │  4753.09 ms │                4413.17 ms │ +1.08x faster │
│ QQuery 33 │  5744.21 ms │                5963.09 ms │     no change │
│ QQuery 34 │  6495.73 ms │                6120.84 ms │ +1.06x faster │
│ QQuery 35 │  2009.76 ms │                2056.09 ms │     no change │
│ QQuery 36 │   189.60 ms │                 194.16 ms │     no change │
│ QQuery 37 │    73.46 ms │                  77.29 ms │  1.05x slower │
│ QQuery 38 │   110.99 ms │                 114.92 ms │     no change │
│ QQuery 39 │   343.21 ms │                 374.55 ms │  1.09x slower │
│ QQuery 40 │    41.60 ms │                  39.72 ms │     no change │
│ QQuery 41 │    34.25 ms │                  34.22 ms │     no change │
│ QQuery 42 │    32.04 ms │                  31.32 ms │     no change │
└───────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 92201.54ms │
│ Total Time (repartition-coalesce-hash)   │ 95459.27ms │
│ Average Time (HEAD)                      │  2144.22ms │
│ Average Time (repartition-coalesce-hash) │  2219.98ms │
│ Queries Faster                           │          4 │
│ Queries Slower                           │         11 │
│ Queries with No Change                   │         28 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 127.23 ms │                 130.82 ms │     no change │
│ QQuery 2  │  32.25 ms │                  33.98 ms │  1.05x slower │
│ QQuery 3  │  36.10 ms │                  41.26 ms │  1.14x slower │
│ QQuery 4  │  35.63 ms │                  32.10 ms │ +1.11x faster │
│ QQuery 5  │  93.25 ms │                  90.62 ms │     no change │
│ QQuery 6  │  25.23 ms │                  24.76 ms │     no change │
│ QQuery 7  │ 156.62 ms │                 152.86 ms │     no change │
│ QQuery 8  │  41.26 ms │                  40.20 ms │     no change │
│ QQuery 9  │ 116.07 ms │                 113.97 ms │     no change │
│ QQuery 10 │  74.67 ms │                  68.79 ms │ +1.09x faster │
│ QQuery 11 │  19.33 ms │                  20.20 ms │     no change │
│ QQuery 12 │  66.61 ms │                  68.22 ms │     no change │
│ QQuery 13 │  56.89 ms │                  55.03 ms │     no change │
│ QQuery 14 │  16.51 ms │                  17.28 ms │     no change │
│ QQuery 15 │  33.51 ms │                  34.07 ms │     no change │
│ QQuery 16 │  30.76 ms │                  30.60 ms │     no change │
│ QQuery 17 │ 173.64 ms │                 165.19 ms │     no change │
│ QQuery 18 │ 304.02 ms │                 302.38 ms │     no change │
│ QQuery 19 │  52.82 ms │                  54.14 ms │     no change │
│ QQuery 20 │  63.54 ms │                  62.42 ms │     no change │
│ QQuery 21 │ 200.71 ms │                 194.33 ms │     no change │
│ QQuery 22 │  25.65 ms │                  23.88 ms │ +1.07x faster │
└───────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1782.29ms │
│ Total Time (repartition-coalesce-hash)   │ 1757.11ms │
│ Average Time (HEAD)                      │   81.01ms │
│ Average Time (repartition-coalesce-hash) │   79.87ms │
│ Queries Faster                           │         3 │
│ Queries Slower                           │         2 │
│ Queries with No Change                   │        17 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

alamb-ghbot · 2026-03-06T19:03:19Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (d659edb) to d025869 diff using: tpch
Results will be posted here when complete

alamb-ghbot · 2026-03-06T19:03:56Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 185.52 ms │                 177.44 ms │     no change │
│ QQuery 2  │  89.22 ms │                  87.84 ms │     no change │
│ QQuery 3  │ 123.00 ms │                 121.09 ms │     no change │
│ QQuery 4  │  81.59 ms │                  73.75 ms │ +1.11x faster │
│ QQuery 5  │ 180.95 ms │                 169.46 ms │ +1.07x faster │
│ QQuery 6  │  67.51 ms │                  67.04 ms │     no change │
│ QQuery 7  │ 213.17 ms │                 206.82 ms │     no change │
│ QQuery 8  │ 170.91 ms │                 164.89 ms │     no change │
│ QQuery 9  │ 235.45 ms │                 231.03 ms │     no change │
│ QQuery 10 │ 189.66 ms │                 206.01 ms │  1.09x slower │
│ QQuery 11 │  59.60 ms │                  62.30 ms │     no change │
│ QQuery 12 │ 117.62 ms │                 118.24 ms │     no change │
│ QQuery 13 │ 242.52 ms │                 209.53 ms │ +1.16x faster │
│ QQuery 14 │  89.10 ms │                  93.63 ms │  1.05x slower │
│ QQuery 15 │ 124.27 ms │                 122.53 ms │     no change │
│ QQuery 16 │  59.56 ms │                  63.33 ms │  1.06x slower │
│ QQuery 17 │ 266.95 ms │                 278.50 ms │     no change │
│ QQuery 18 │ 331.46 ms │                 317.57 ms │     no change │
│ QQuery 19 │ 126.76 ms │                 129.40 ms │     no change │
│ QQuery 20 │ 135.18 ms │                 127.72 ms │ +1.06x faster │
│ QQuery 21 │ 268.15 ms │                 257.53 ms │     no change │
│ QQuery 22 │  41.97 ms │                  40.89 ms │     no change │
└───────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 3400.11ms │
│ Total Time (repartition-coalesce-hash)   │ 3326.53ms │
│ Average Time (HEAD)                      │  154.55ms │
│ Average Time (repartition-coalesce-hash) │  151.21ms │
│ Queries Faster                           │         4 │
│ Queries Slower                           │         3 │
│ Queries with No Change                   │        15 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

alamb-ghbot · 2026-03-06T19:03:59Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing repartition-coalesce-hash (d659edb) to d025869 diff using: tpch_mem
Results will be posted here when complete

alamb-ghbot · 2026-03-06T19:04:23Z

🤖: Benchmark completed

Details

Comparing HEAD and repartition-coalesce-hash
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃      HEAD ┃ repartition-coalesce-hash ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │ 130.54 ms │                 130.55 ms │     no change │
│ QQuery 2  │  32.46 ms │                  32.85 ms │     no change │
│ QQuery 3  │  40.61 ms │                  39.92 ms │     no change │
│ QQuery 4  │  35.99 ms │                  32.06 ms │ +1.12x faster │
│ QQuery 5  │  95.06 ms │                  90.35 ms │     no change │
│ QQuery 6  │  26.09 ms │                  24.81 ms │     no change │
│ QQuery 7  │ 163.40 ms │                 156.41 ms │     no change │
│ QQuery 8  │  45.19 ms │                  40.38 ms │ +1.12x faster │
│ QQuery 9  │ 116.37 ms │                 113.31 ms │     no change │
│ QQuery 10 │  71.76 ms │                  71.25 ms │     no change │
│ QQuery 11 │  18.81 ms │                  19.67 ms │     no change │
│ QQuery 12 │  67.15 ms │                  66.60 ms │     no change │
│ QQuery 13 │  55.85 ms │                  50.04 ms │ +1.12x faster │
│ QQuery 14 │  15.92 ms │                  16.00 ms │     no change │
│ QQuery 15 │  33.68 ms │                  35.58 ms │  1.06x slower │
│ QQuery 16 │  30.32 ms │                  30.60 ms │     no change │
│ QQuery 17 │ 176.24 ms │                 170.71 ms │     no change │
│ QQuery 18 │ 304.05 ms │                 299.49 ms │     no change │
│ QQuery 19 │  54.24 ms │                  53.26 ms │     no change │
│ QQuery 20 │  63.97 ms │                  60.00 ms │ +1.07x faster │
│ QQuery 21 │ 197.92 ms │                 196.29 ms │     no change │
│ QQuery 22 │  25.71 ms │                  24.21 ms │ +1.06x faster │
└───────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1801.33ms │
│ Total Time (repartition-coalesce-hash)   │ 1754.37ms │
│ Average Time (HEAD)                      │   81.88ms │
│ Average Time (repartition-coalesce-hash) │   79.74ms │
│ Queries Faster                           │         5 │
│ Queries Slower                           │         1 │
│ Queries with No Change                   │        16 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

github-actions bot added the physical-plan Changes to the physical-plan crate label Mar 6, 2026

Dandandan marked this pull request as draft March 6, 2026 10:19

Dandandan force-pushed the repartition-coalesce-hash branch from 15f10c8 to ef571e2 Compare March 6, 2026 10:46

Disable yielding every n

c47210b

Fmt

20d15e2

Dandandan and others added 2 commits March 6, 2026 15:24

Fixes

07d3ff6

Conversation

Dandandan commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

Dandandan commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

Dandandan commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

Dandandan commented Mar 6, 2026

Uh oh!

Dandandan commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

Dandandan commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

Dandandan commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

Dandandan commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

Dandandan commented Mar 6, 2026

Uh oh!

Dandandan commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

alamb-ghbot commented Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Dandandan commented Mar 6, 2026 •

edited

Loading

Dandandan commented Mar 6, 2026 •

edited

Loading