feat(encoding): add generic block compression

[Xuanwo]

Lance 2.3 needs a reusable, fallible block-sequence compression contract so variable-width mini-block offsets can use Range, Delta, RLE, dictionary, bitpacking, and general compression through the same concrete codec-tree model used by the existing encoding architecture.

This draft contains the complete end-to-end implementation to make the format boundary and dependency graph reviewable before it is split into smaller stacked PRs. It keeps Lance 2.0-2.2 persisted bytes, selection, and reader behavior unchanged, preserves chunk-local random-read granularity, and does not migrate the SparseLayout wire format.

The branch also includes the 2.3 generic-offset format documentation, compatibility and malformed-input coverage, and controlled encoder/decoder benchmarks. The draft will remain non-mergeable while the implementation is decomposed into reviewable units.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: QUIET

Plan: Pro Plus

Run ID: 6ec34aed-c28f-405b-808c-a6982db1f265

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch xuanwo/generic-block-compression

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Important

This PR touches the Lance format specification.

Substantive changes to the format specification — the .proto definitions
and the spec docs under docs/src/format/ — require a PMC vote before merge.
Minor edits such as typo fixes, wording, or formatting are excluded; use your
judgment.

If this is a meaningful format change:

• Start a vote following the Lance community voting process.
  Format specification modifications need 3 binding +1 votes (excluding the
  proposer), held on GitHub Discussions, with a minimum voting period of 1 week.
• Once the vote passes, link the completed vote in this PR. It should not be
  merged until the vote is linked.

[codecov]

Codecov Report

❌ Patch coverage is 73.47811% with 1490 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...-encoding/src/encodings/physical/binary/offsets.rs	74.84%	213 Missing and 69 partials ⚠️
...st/lance-encoding/src/encodings/physical/binary.rs	76.63%	188 Missing and 91 partials ⚠️
...st/lance-encoding/src/compression/block/factory.rs	62.31%	166 Missing and 33 partials ⚠️
rust/lance-encoding/src/encodings/physical/rle.rs	72.36%	114 Missing and 33 partials ⚠️
...ust/lance-encoding/src/encodings/physical/block.rs	62.39%	74 Missing and 14 partials ⚠️
rust/lance-encoding/src/compression.rs	70.25%	64 Missing and 19 partials ⚠️
...ance-encoding/src/encodings/physical/dictionary.rs	63.87%	63 Missing and 19 partials ⚠️
...ust/lance-encoding/src/encodings/physical/delta.rs	65.17%	57 Missing and 13 partials ⚠️
.../src/encodings/physical/binary/offsets/selector.rs	61.22%	54 Missing and 3 partials ⚠️
...ust/lance-encoding/src/encodings/physical/range.rs	71.08%	33 Missing and 15 partials ⚠️
... and 13 more

📢 Thoughts on this report? Let us know!