Skip to content

Add optional fieldId and aliases to FieldSpec for column identity#18833

Open
navina wants to merge 1 commit into
apache:masterfrom
navina:column-mapping
Open

Add optional fieldId and aliases to FieldSpec for column identity#18833
navina wants to merge 1 commit into
apache:masterfrom
navina:column-mapping

Conversation

@navina

@navina navina commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

What

Adds two optional, nullable metadata fields to FieldSpec (pinot-spi):

Field Type Meaning
fieldId Integer A stable, name-independent identifier for the column.
aliases List<String> Alternate / historical names for the column.

Both follow the existing description / tags convention on FieldSpec:

  • fieldId is serialized with @JsonInclude(NON_NULL), aliases with @JsonInclude(NON_EMPTY).
  • Both are mirrored in the manual toJsonObject() serialization and included in equals() / hashCode().
  • They are not part of schema backward-compatibility validation, so adding or changing them doesn't fail a schema update today.

Why

fieldId provides a stable column identity that is decoupled from the column name. Together with aliases, it lays the groundwork for advanced schema evolutions like column rename in Pinot — a rename can preserve the column's identity and record the prior name as an alias, instead of looking like a drop-plus-add.

Compatibility

  • No impact on existing tables. Both fields default to null/empty and are omitted from JSON, so the serialized schema and wire format are unchanged for any column that doesn't set them.
  • Mixed-version safe. Older components ignore the unknown JSON properties on read; newer components emit nothing for columns without these fields. They are excluded from backward-compatibility checks, so a schema carrying them validates against one that doesn't.
  • No updates to query/ingestion paths or segment metadata today — this change only adds the fields to the schema model.

Tests

FieldSpecTest adds coverage for the new getters/setters, JSON round-trip (including omission when unset), and equals/hashCode participation.

Extend FieldSpec with stable fieldId and alias metadata for logical-to-physical column mapping, following the description/tags serde pattern and excluding these fields from backward compatibility checks.

Co-authored-by: Cursor <cursoragent@cursor.com>
@navina navina marked this pull request as ready for review June 22, 2026 21:18
@codecov-commenter

codecov-commenter commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.81%. Comparing base (52b8fdc) to head (7a6309a).

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18833      +/-   ##
============================================
+ Coverage     64.78%   64.81%   +0.02%     
  Complexity     1319     1319              
============================================
  Files          3392     3392              
  Lines        210988   211004      +16     
  Branches      33127    33132       +5     
============================================
+ Hits         136687   136758      +71     
+ Misses        63283    63206      -77     
- Partials      11018    11040      +22     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-21 64.81% <100.00%> (+0.02%) ⬆️
temurin 64.81% <100.00%> (+0.02%) ⬆️
unittests 64.81% <100.00%> (+0.02%) ⬆️
unittests1 56.96% <100.00%> (-0.04%) ⬇️
unittests2 37.23% <11.76%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants