[FLINK-40217][core] Reject non-finite numbers in Variant JSON conversion#28808
Open
raminqaf wants to merge 2 commits into
Open
[FLINK-40217][core] Reject non-finite numbers in Variant JSON conversion#28808raminqaf wants to merge 2 commits into
raminqaf wants to merge 2 commits into
Conversation
snuyanzin
reviewed
Jul 22, 2026
snuyanzin
reviewed
Jul 22, 2026
snuyanzin
reviewed
Jul 22, 2026
snuyanzin
reviewed
Jul 22, 2026
Comment on lines
+126
to
+128
| @ValueSource(strings = {"NaN", "Infinity", "-Infinity", "1e400", "-1e400"}) | ||
| void testParseJsonRejectsNonFiniteNumbers(final String nonFiniteNumber) { | ||
| // NaN and the infinities are not valid JSON; 1e400 is valid JSON but overflows the double |
Contributor
There was a problem hiding this comment.
what about length of a number (not only for float/double)
in jackson it is configured with StreamReadConstraints.DEFAULT_MAX_NUM_LEN which is by default 1000.
Means there might be issues with longer numbers
Contributor
Author
There was a problem hiding this comment.
Added a new test that generates a number with 1001 digits.
Contributor
Author
There was a problem hiding this comment.
Removed the test. We investigated and realized that any number with a precision of >38 is parsed as double. If the number is then >Double.MAX_VALUE it will fall into the infinity exception.
PARSE_JSON accepted JSON numbers outside the double range, such as 1e400, and silently stored them as +/-Infinity. Variant.toJson() then emitted bare Infinity/-Infinity tokens, which are invalid JSON and cannot be parsed back by PARSE_JSON, so the round trip was broken. parseFloatingPoint now rejects a non-finite result from getDoubleValue() with a clear parse error. PARSE_JSON surfaces the failure and TRY_PARSE_JSON returns NULL, so a parsed Variant can never hold a non-finite value. As a defensive safeguard for the builder API, which can still inject non-finite values, toJson() now throws for non-finite DOUBLE and FLOAT values instead of emitting invalid tokens.
Cover the end-to-end SQL wiring for PARSE_JSON and TRY_PARSE_JSON: a JSON_STRING round trip, NULL handling, and the out-of-range number behavior. An overflowing number such as 1e400 makes PARSE_JSON fail with a TableRuntimeException, while TRY_PARSE_JSON returns NULL. The parsing semantics themselves stay covered by BinaryVariantInternalBuilderTest.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
PARSE_JSONaccepted JSON numbers outside the double range, such as1e400, and silently stored them as±Infinity.Variant.toJson()then emitted bareInfinity/-Infinitytokens, which are invalid JSON and cannot be parsed back byPARSE_JSON. This closes that gap by rejecting non-finite values on both the parse and the serialize side.Brief change log
BinaryVariantInternalBuilder.parseFloatingPoint()now rejects a non-finite result fromgetDoubleValue()with a clear parse error.PARSE_JSON('1e400')fails loudly andTRY_PARSE_JSON('1e400')returnsNULL, so a parsed Variant can never hold a non-finite value.BinaryVariant.toJson()now throws aVariantTypeExceptionfor non-finiteDOUBLEandFLOATvalues instead of appending invalidInfinity/NaNtokens. This guards the builder API, which is the only remaining way to inject a non-finite value.parseJsonreuses a single staticJsonFactoryinstead of allocating one per call.Verifying this change
This change added tests and can be verified as follows:
BinaryVariantInternalBuilderTest#testParseJsonRejectsNonFiniteNumbers, parameterized overNaN,Infinity,-Infinity,1e400, and-1e400, assertingparseJsonthrows (invalid JSON tokens and finite-but-overflowing numbers alike).BinaryVariantTest#testToJsonRejectsNonFiniteDoubleand#testToJsonRejectsNonFiniteFloat, parameterized over the infinities andNaN, assertingtoJson()throws instead of emitting invalid JSON.Does this pull request potentially affect one of the following parts:
@Public(Evolving): no (both changed classes are@Internal; the user-visible effect is a semantics change to thePARSE_JSON/TRY_PARSE_JSONSQL functions on out-of-range numbers)PARSE_JSON/toJson()paths; the only added cost is a constant-time finite check, and reusingJsonFactoryremoves a per-call allocation)Documentation
Was generative AI tooling used to co-author this PR?