PoC(CI): Introduce xtask to keep local test and CI commands in sync

[2010YOUY01]

Which issue does this PR close?

Part of #21048

Rationale for this change

The end goal is easy local CI reproducer like

# run full ci tests
cargo xtask ci full
# run fast tests
cargo xtask ci fast

This PR only try to encapsulate single CI step with xtask, and hope later build towards that goal. Running test commands locally mirroring CI become easier after this PR.

For example, the default test run command is:

datafusion/.github/workflows/rust.yml

Lines 324 to 335 in 26fbf4d

	cargo llvm-cov \
	--profile ci \
	--exclude datafusion-examples \
	--exclude ffi_example_table_provider \
	--exclude datafusion-cli \
	--workspace \
	--lib \
	--tests \
	--bins \
	--features serde,avro,json,backtrace,integration-tests,parquet_encryption,substrait \
	--codecov \
	--output-path target/codecov.json

Now both ci and local run can be executed with

# Use `cargo xtask help` for the full usage
# syntax: cargo xtask ci step [step-name] [extra-args]
cargo xtask ci step test workspace
(...run tests)

# Use 'explain' to show the underlying command
cargo xtask ci step explain test workspace
(...print the full command)

The command is a bit verbose, since it's a common pattern (see top comments at xtask/src/main.rs) for all internal dev tools, so we might need additional domains like cargo xtask benchmark [bench-name]

xtask implementation mechanism

See xtask/src/main.rs top comments for details.

TLDR: internal binary, no external dependencies.

xtask v.s. shell script

The existing CI jobs consist of lot of similar commands, that is parameterized with features, settings, so it would be easier to implement the runner that can handle args, to make the maintenance easier.

We also want some extra functionalities, like the above explain feature, or more complex orchestration needed for the 'one-click local CI run' end goal.

At this complexity, I find implementing it in rust is easier than shell scripts.

What changes are included in this PR?

In rust CI workflow (rust.yml), encapsulate cargo test ... and cargo check ... family into xtask runner

To review the core changes, start from the top comments in xtask/src/main.rs

Are these changes tested?

Are there any user-facing changes?

[codecov-commenter]

Codecov Report

❌ Patch coverage is 63.36996% with 200 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.86%. Comparing base (9051efd) to head (6a25550).
⚠️ Report is 12 commits behind head on main.

Files with missing lines	Patch %	Lines
xtask/src/ci_steps.rs	67.18%	162 Missing and 7 partials ⚠️
xtask/src/main.rs	0.00%	31 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24083      +/-   ##
==========================================
- Coverage   80.87%   80.86%   -0.02%     
==========================================
  Files        1101     1104       +3     
  Lines      375716   376657     +941     
  Branches   375716   376657     +941     
==========================================
+ Hits       303874   304587     +713     
- Misses      53739    53953     +214     
- Partials    18103    18117      +14     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.