Search before asking
Paimon version
master (also reproduces on 1.4.2)
Compute Engine
any (reproduces with the table API alone)
Minimal reproduce step
Create a primary-key table whose primary key contains a BYTES/VARBINARY field, enable a lookup strategy (deletion-vectors.enabled = true, or changelog-producer = lookup), write enough that lookup compaction must probe a higher level for an overwritten key (e.g. restore-style workloads where level-1+ files exist before new level-0 writes). The first sorted-lookup-file seek throws:
java.lang.ClassCastException: class [B cannot be cast to class java.lang.Comparable
at org.apache.paimon.data.serializer.RowCompactedSerializer$SliceComparator.compare(RowCompactedSerializer.java)
at org.apache.paimon.sst.BlockIterator.seekTo(BlockIterator.java:74)
at org.apache.paimon.lookup.sort.SortLookupStoreReader.lookup(SortLookupStoreReader.java:66)
at org.apache.paimon.mergetree.LookupLevels.lookup(...)
What doesn't meet your expectations?
Binary primary keys are legal: SchemaValidation admits them and TypeCheckUtils.isComparable returns true for BINARY/VARBINARY, so createSliceComparator's guard passes. But SliceComparator.compare casts every field value to Comparable, and BINARY/VARBINARY fields read back as byte[], which is not. The comparator should order binary fields the way BinaryRow does (unsigned lexicographic, SortUtil.compareBinary).
The bug stays hidden on small tables because compactions there tend to pick full merges (no lookup); it surfaces exactly when lookup compaction starts probing higher levels.
Anything else?
Fix incoming as a PR.
Are you willing to submit a PR?
Search before asking
Paimon version
master (also reproduces on 1.4.2)
Compute Engine
any (reproduces with the table API alone)
Minimal reproduce step
Create a primary-key table whose primary key contains a
BYTES/VARBINARYfield, enable a lookup strategy (deletion-vectors.enabled = true, orchangelog-producer = lookup), write enough that lookup compaction must probe a higher level for an overwritten key (e.g. restore-style workloads where level-1+ files exist before new level-0 writes). The first sorted-lookup-file seek throws:What doesn't meet your expectations?
Binary primary keys are legal:
SchemaValidationadmits them andTypeCheckUtils.isComparablereturns true for BINARY/VARBINARY, socreateSliceComparator's guard passes. ButSliceComparator.comparecasts every field value toComparable, and BINARY/VARBINARY fields read back asbyte[], which is not. The comparator should order binary fields the wayBinaryRowdoes (unsigned lexicographic,SortUtil.compareBinary).The bug stays hidden on small tables because compactions there tend to pick full merges (no lookup); it surfaces exactly when lookup compaction starts probing higher levels.
Anything else?
Fix incoming as a PR.
Are you willing to submit a PR?