Skip to content

feat(catalog): support column-level alter table#370

Open
TheR1sing3un wants to merge 1 commit into
apache:mainfrom
TheR1sing3un:pr/column-alter-table
Open

feat(catalog): support column-level alter table#370
TheR1sing3un wants to merge 1 commit into
apache:mainfrom
TheR1sing3un:pr/column-alter-table

Conversation

@TheR1sing3un

Copy link
Copy Markdown
Member

Purpose

Linked issue: close #368

Implement column-level alter table on both FileSystemCatalog and RESTCatalog, and align the SchemaChange JSON wire format with Java Paimon.

Brief change log

  • TableSchema::apply_changes: implement add / rename / drop column, update column type / nullability / comment / position, and update table comment, mirroring Java SchemaManager.generateTableSchema (field-id allocation, FIRST/AFTER/BEFORE/LAST moves, primary/partition-key guards). Operates on top-level columns; reuses ColumnAlreadyExist / ColumnNotExist. The method keeps its single-argument signature — the catalog fills in the table name on column errors.
  • Align SchemaChange JSON with Java Paimon: internally tagged by action, with fieldNames arrays, comment / newDataType / keepNullability / newNullability / newComment fields, referenceFieldName move anchors, and FIRST/AFTER/BEFORE/LAST move types.
  • Add AlterTableRequest + RESTApi::alter_table; implement RESTCatalog::alter_table.

Tests

  • New unit tests in spec::schema_change (Java-format (de)serialization), spec::schema, and catalog::filesystem (add+move, rename syncing PK refs, drop rejecting PK columns, type/nullability/comment updates, reposition, and the ColumnAlreadyExist / ColumnNotExist / TableNotExist error paths).
  • Existing paimon-datafusion alter-table tests pass.

API and Format

Changes the SchemaChange JSON wire format. This is effectively a fix: the previous format was incompatible with Java Paimon, and REST alter_table was never implemented, so no working interaction is broken. No public Rust API signature change (apply_changes keeps its arity).

Documentation

None required.

Out of scope (follow-ups)

Nested struct field paths; UpdateColumnType cast-compatibility validation; Java's dropPrimaryKey / updateColumnDefaultValue.

@TheR1sing3un

Copy link
Copy Markdown
Member Author

Pushed a fix for the integration job: the existing test_alter_table_add_column (in crates/integrations/datafusion/tests/sql_context_tests.rs) asserted that ALTER TABLE ... ADD COLUMN fails because AddColumn was unsupported. Since this PR implements it, the test now asserts the statement succeeds and the new column is appended to the schema. Verified locally: cargo test -p paimon-datafusion --test sql_context_tests test_alter_table passes.

Implement all column-level SchemaChange variants in
TableSchema::apply_changes (add/rename/drop column, update column
type/nullability/comment/position, update table comment), so
FileSystemCatalog::alter_table and RESTCatalog::alter_table can evolve
table schemas. Changes operate on top-level columns and reuse the
existing ColumnAlreadyExist/ColumnNotExist errors; apply_changes keeps
its single-argument signature (the catalog fills in the table name).

Align the SchemaChange JSON wire format with Java Paimon: internally
tagged by "action", with fieldNames arrays, comment / newDataType /
keepNullability / newNullability / newComment fields, referenceFieldName
move anchors, and FIRST/AFTER/BEFORE/LAST move types. Add
AlterTableRequest and RESTApi::alter_table, and implement
RESTCatalog::alter_table so the REST client can alter tables against a
Paimon REST server.
@TheR1sing3un TheR1sing3un force-pushed the pr/column-alter-table branch from 8d214bf to 44fa702 Compare June 10, 2026 02:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support column-level alter table in FileSystemCatalog and RESTCatalog

1 participant