perf: Optimise map_rows performance for Object dtype

[alexander-beedie]

Ref: #25688.

Added a dedicated Object path in map_rows that gets us a 50-60% speedup (on the given test-case). I think I can see a few opportunities to get a smaller speed-up for scalar/primitive types too, but will leave that for a separate PR.

Benchmark

import polars as pl
df = pl.DataFrame(
  data={
    "a": [1,2,3] * 1_000_000, 
    "b": [1,2,3] * 1_000_000,
  },
  schema={"a": pl.Int64, "b": pl.Int64},
)
%timeit df.map_rows(lambda d: (d[0] + d[1]), return_dtype=pl.Object)

Timings¹ 🚀

Before: 306 ms ± 2.87 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
 After: 194 ms ± 1.44 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Footnotes

1. Tested using make build-dist-release on an Apple Silicon M4 Max ↩

[codecov]

Codecov Report

❌ Patch coverage is 52.08333% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.54%. Comparing base (405b194) to head (18e87bf).
⚠️ Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/polars-python/src/dataframe/map.rs	52.08%	23 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #25702      +/-   ##
==========================================
- Coverage   80.55%   80.54%   -0.01%     
==========================================
  Files        1756     1757       +1     
  Lines      241908   242143     +235     
  Branches     3040     3040              
==========================================
+ Hits       194869   195036     +167     
- Misses      46256    46325      +69     
+ Partials      783      782       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[orlp]

I literally just removed all the code that special-cases all this stuff because it all wasn't being updated and was buggy :(

Can we not make the generic path faster by profiling/optimizing AnyValue instead of immediately adding special-cases back?

[alexander-beedie]

I literally just removed all the code that special-cases all this stuff because it all wasn't being updated and was buggy :(

Not sure how I'm supposed to know that code I can't see used to be there 🤷

Can we not make the generic path faster by profiling/optimizing AnyValue instead of immediately adding special-cases back?

I'll take a look tomorrow.

[orlp]

Not sure how I'm supposed to know that code I can't see used to be there 🤷

You're not, I'm not blaming you :)