Skip to content

feat(blob): add BlobType groundwork and preserve DDL semantics#250

Merged
JingsongLi merged 2 commits intoapache:mainfrom
QuakeWang:feat/blob-type-spec
Apr 16, 2026
Merged

feat(blob): add BlobType groundwork and preserve DDL semantics#250
JingsongLi merged 2 commits intoapache:mainfrom
QuakeWang:feat/blob-type-spec

Conversation

@QuakeWang
Copy link
Copy Markdown
Contributor

Purpose

This PR adds the first groundwork slice for BlobType in paimon-rust.

It introduces BlobType into the Rust type system, preserves BLOB in DataFusion DDL, and explicitly rejects unsupported write paths.

This PR is only a groundwork step for #228, and I will implement .blob read path for the next pr.

Brief change log

  • add DataType::Blob and BlobType
  • support BlobType schema serialization / deserialization and add serde fixtures
  • expose BlobType as Arrow Binary in the existing type conversion surface
  • preserve BLOB in DataFusion DDL schema translation, instead of silently degrading it to VARBINARY
  • add tests for CREATE TABLE / ALTER TABLE ... ADD COLUMN ... BLOB
  • explicitly reject BlobType in current write paths
  • explicitly keep BlobType unsupported for partition keys and parquet row filter literals

Tests

API and Format

API change:

  • add a new logical type BlobType / DataType::Blob

Format impact:

  • schema serde now supports BLOB
  • no data file format change in this PR
  • .blob file read support is not included in this PR

Documentation

///
/// DDL schema translation must use this function instead of going through Arrow,
/// because Arrow cannot preserve logical distinctions such as `BLOB` vs `VARBINARY`.
fn sql_data_type_to_paimon_type(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have this, we should remove sql_data_type_to_arrow.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JingsongLi Thanks for the review. sql_data_type_to_arrow has been removed, and DDL type translation now goes directly through sql_data_type_to_paimon_type.

Copy link
Copy Markdown
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi JingsongLi merged commit 4a2e657 into apache:main Apr 16, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants