Skip to content

[SPARK-58296][SQL] Fix to_time returning STRING type when the format is a foldable NULL#57486

Open
LuciferYang wants to merge 1 commit into
apache:masterfrom
LuciferYang:SPARK-58296-totime-null-format
Open

[SPARK-58296][SQL] Fix to_time returning STRING type when the format is a foldable NULL#57486
LuciferYang wants to merge 1 commit into
apache:masterfrom
LuciferYang:SPARK-58296-totime-null-format

Conversation

@LuciferYang

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

ToTime (the to_time function) is a RuntimeReplaceable, so its dataType comes from replacement.dataType. When the format argument is foldable and evaluates to NULL, the replacement fell back to Literal(null, expr.dataType), where expr is the format argument, so the whole expression reported the format's type (STRING) instead of TIME. This changes the fallback to Literal(null, TimeType()), so to_time reports TimeType() like all of its other branches.

Why are the changes needed?

to_time(str, fmt) with a foldable NULL fmt reports the wrong type. SELECT typeof(to_time('00:12:00', null)) returns string instead of time(6), and the analyzed plan carries STRING for that column, so schema inference and UNION type coercion (against a real TIME column) use the wrong type. The value is NULL either way, but the type is user-visible and wrong. The inconsistency is also internal: a non-foldable NULL format takes the invokeParser() branch and correctly yields time(6), so the result type depended on whether the format happened to be foldable.

Does this PR introduce any user-facing change?

Yes. to_time(str, fmt) with a foldable NULL fmt now has type time(6) instead of STRING, consistent with all other forms of to_time. The returned value (NULL) is unchanged. This is a bug fix: the wrong type has been present since to_time was introduced in 4.1.0.

How was this patch tested?

Added a TimeExpressionsSuite case asserting ToTime(...).dataType === TimeType() for both an untyped and a STRING-typed foldable NULL format; it fails on the unfixed tree with StringType did not equal TimeType(6). The same case also asserts the value is still NULL via checkEvaluation. Also added two golden queries in time.sql whose schema now locks in time(6).

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.8)

…is a foldable NULL

`ToTime` is a `RuntimeReplaceable`, so its `dataType` comes from `replacement.dataType`.
When the format argument is foldable and evaluates to NULL, the replacement fell back
to `Literal(null, expr.dataType)` where `expr` is the format argument, making `to_time`
report STRING instead of TIME. Change the fallback to `Literal(null, TimeType())` so it
matches every other branch.
@LuciferYang
LuciferYang requested a review from MaxGekk July 24, 2026 05:46
@LuciferYang

Copy link
Copy Markdown
Contributor Author

cc @MaxGekk @cloud-fan

@uros-b uros-b left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you @LuciferYang! cc @MaxGekk

@cloud-fan cloud-fan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 blocking, 0 non-blocking, 0 nits.
The implementation corrects the inconsistent replacement type at its source and includes focused expression-level and SQL-level regression coverage.

Verification

Independently traced every ToTime.replacement branch: both invokeParser paths declare TimeType(), and the foldable-NULL fallback now does the same. The added tests exercise typed and untyped foldable NULL formats, retain the NULL value assertion, and verify the externally visible type through typeof. No test command was run as part of this review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants