Skip to content

refactor: make index file versions exact - #8028

Draft
Xuanwo wants to merge 2 commits into
xuanwo/exact-version-stack-10-dataset-operationsfrom
xuanwo/exact-version-stack-11-index-consumers
Draft

refactor: make index file versions exact#8028
Xuanwo wants to merge 2 commits into
xuanwo/exact-version-stack-10-dataset-operationsfrom
xuanwo/exact-version-stack-11-index-consumers

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Part 11/12 of #7877. Depends on #8027.

This is an independently reviewable step toward the final layout demonstrated in #7979.

This PR converts scalar and vector index readers, writers, shufflers, and distributed mergers to exact ConcreteFileVersion identities. Dataset-backed indexes inherit the dataset format explicitly; legacy dataset physical indexes map explicitly to V2.0.

With the last transitional consumers migrated, this PR removes file-version ordering and implicit conversions between selectors and exact formats. Remaining version decisions are exhaustive matches at declared boundaries rather than >=, max, or selector round-trips.

Validation:

  • index format inheritance tests
  • legacy dataset physical-index format test
  • V3 shuffler tests
  • cargo clippy --all --tests --benches -- -D warnings
  • Python and Java binding checks

@github-actions github-actions Bot added A-python Python bindings A-index Vector index, linalg, tokenizer A-java Java bindings + JNI A-encoding Encoding, IO, file reader/writer labels Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer A-index Vector index, linalg, tokenizer A-java Java bindings + JNI A-python Python bindings

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant