Skip to content

Add benchmark_runner info and query commands#22032

Draft
Omega359 wants to merge 1 commit intoapache:mainfrom
Omega359:benchmark-runner-info-query
Draft

Add benchmark_runner info and query commands#22032
Omega359 wants to merge 1 commit intoapache:mainfrom
Omega359:benchmark-runner-info-query

Conversation

@Omega359
Copy link
Copy Markdown
Contributor

@Omega359 Omega359 commented May 6, 2026

Which issue does this PR close?

Rationale for this change

This continues the benchmark_runner CLI work by adding non-execution inspection commands for SQL benchmark suites and native benchmarks.

The goal is to let users discover benchmark metadata and inspect parsed SQL without running benchmarks. This also gives SQL suite metadata a single .suite-driven source of truth for CLI options, examples, and query file resolution.

See #22001 (comment) for future planned steps.

What changes are included in this PR?

This PR adds support for:

  • benchmark_runner info <SUITE>
  • benchmark_runner info <SUITE> <QUERY_ID>
  • benchmark_runner info <NATIVE_BENCHMARK>
  • benchmark_runner query <SUITE> <QUERY_ID>

It also includes:

  • dynamic suite option registration from .suite metadata
  • suite option validation for info and query
  • multi-character suite option alias normalization, such as -sf
  • configurable suite query_pattern support
  • SQL benchmark selector resolution by suite and query id
  • native benchmark metadata support for list and info
  • styled info and query output
  • full SQL info output sections showing the relevant run usage/options surface

Benchmark execution is intentionally not included in this PR. The run command is left for the next stacked PR.

Are these changes tested?

Yes. This PR adds and updates tests for:

  • SQL suite discovery
  • dynamic suite options
  • suite option validation
  • query id resolution
  • configurable query_pattern handling
  • info output for suites and individual queries
  • query output grouped by benchmark directive
  • native benchmark list/info behavior
  • native benchmark name collision handling
  • CLI parsing and suite option alias normalization

Are there any user-facing changes?

Yes. The benchmark_runner binary gains new inspection commands:
• info
• query

These commands are additive and do not change public Rust APIs.

Use of AI

This code was partially written by Codex but edited, reviewed and tested by @Omega359

@Omega359 Omega359 force-pushed the benchmark-runner-info-query branch 3 times, most recently from 920ad3c to fa4a61e Compare May 6, 2026 02:31
@Omega359 Omega359 force-pushed the benchmark-runner-info-query branch from fa4a61e to 8216ecf Compare May 6, 2026 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant