Skip to content

[MHA] fix issue-2779#2780

Open
HanchiMed wants to merge 8 commits intoClickHouse:mainfrom
HanchiMed:issue-2779
Open

[MHA] fix issue-2779#2780
HanchiMed wants to merge 8 commits intoClickHouse:mainfrom
HanchiMed:issue-2779

Conversation

@HanchiMed
Copy link

@HanchiMed HanchiMed commented Mar 5, 2026

Description
Steps to reproduce
create a Clickhouse table with columns :
create table test
(
numliv UInt32,
cdc_changesequence UInt128
)
ENGINE = SharedReplacingMergeTree(cdc_changesequence)
ORDER BY (numliv)

create à PutDatabaseRecord on Nifi

Insert json with column cdc_changesequence filled by a BigInt

{"numliv" : 54878 , "cdc_changesequence " : 20260219145041580000000000403034229 }

Error Log or Exception StackTrace
Failed to put Records to database for FlowFile[filename=c797411d-7c71-4403-9f92-e204c26849a2]. Routing to failure.: java.io.IOException: Unable to setObject() with value 20260219145041580000000000403034240 at index 208 of type 1111

Caused by: java.sql.SQLException: Cannot convert OTHER to a ClickHouse one. Consider using java.sql.JDBCType or com.clickhouse.data.ClickHouseDataType

for clickHouse support (1)

Expected Behaviour
Insertion is functionnal in case of cdc_changesequence is typed as String or BigInt

cdc_changesequence as BigInt
{"numliv" : 54878 , "cdc_changesequence " : 20260219145041580000000000403034229 }

cdc_changesequence as BigInt
{"numliv" : 54878 , "cdc_changesequence " : "20260219145041580000000000403034229" }

Code Example
com.clickhouse.jdbc.JdbcTypeMapping:
public int toSqlType : .......
case UInt64:
case Int128:
case UInt128:
case Int256:
case UInt256:
sqlType = Types.NUMERIC;
break;

com.clickhouse.jdbc.internal.JdbcUtils:

private static Map<ClickHouseDataType, SQLType> generateTypeMap() { ...........
map.put(ClickHouseDataType.Int128, JDBCType.NUMERIC);
map.put(ClickHouseDataType.Int256, JDBCType.NUMERIC);
map.put(ClickHouseDataType.UInt64, JDBCType.NUMERIC);
map.put(ClickHouseDataType.UInt128, JDBCType.NUMERIC);
map.put(ClickHouseDataType.UInt256, JDBCType.NUMERIC);


Note

Medium Risk
Changes JDBC type/class mappings for large integer ClickHouse types, which can affect ResultSetMetaData and setObject/getObject behavior for existing consumers expecting OTHER/BigDecimal. Scope is limited to type mapping and conversion paths and is covered by new unit/integration tests.

Overview
Fixes large-integer handling by mapping ClickHouse UInt64/Int128/UInt128/Int256/UInt256 to JDBC NUMERIC and Java BigInteger (instead of OTHER or DECIMAL) in both the legacy JdbcTypeMapping and jdbc-v2 JdbcUtils.

Updates jdbc-v2 conversion to support BigInteger values and adjusts SQL_TYPE_TO_CLASS_MAP so NUMERIC resolves to BigInteger. Adds new unit tests for the mappings and a new integration test that validates inserts (including nullable columns), ResultSetMetaData types/class names, and getObject(..., BigInteger.class) for these columns.

Written by Cursor Bugbot for commit 3cd682e. This will update automatically on new commits. Configure here.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


mohamed.hanchi seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@chernser
Copy link
Contributor

chernser commented Mar 5, 2026

@HanchiMed
please add tests to verify that correct values are returned for ints and decimals.

@chernser
Copy link
Contributor

chernser commented Mar 5, 2026

@HanchiMed
please add some tests.

@chernser
Copy link
Contributor

chernser commented Mar 5, 2026

@HanchiMed
the tests fail:

Error:  Failures: 
Error:    R2DBCTestKitImplTest.segmentInsertEmitsUpdateCount expectation "insert of a single row" failed (expected value: 1; actual value: 0)
Error:    R2DBCTestKitImplTest.transactionCommit expectation "rows inserted" failed (expected value: 1; actual value: 0)

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

@HanchiMed
Copy link
Author

Hello @chernser

The tests were added.
However, I don’t understand why the tests under R2DBCTestKitImplTest are failing, and I don’t see any relation with the changes I made. Could you please help me to fix this?

Regards

@chernser
Copy link
Contributor

chernser commented Mar 6, 2026

@HanchiMed
I will take a look on R2DBC tests tomorrow.
As for added tests - they are unit and do not test match.
We need integration test to be sure that we can read this type of data. Please add them - at least for affected types.

Thanks!

@HanchiMed
Copy link
Author

@chernser
Integration Tests were added.

Regards

@chernser
Copy link
Contributor

chernser commented Mar 6, 2026

@HanchiMed
I will work on this PR next week as it requires some work for R2DBC client. Everything else looks fine.

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants