Skip to content

fix: array to array cast#2897

Open
manuzhang wants to merge 1 commit intoapache:mainfrom
manuzhang:test-array-cast
Open

fix: array to array cast#2897
manuzhang wants to merge 1 commit intoapache:mainfrom
manuzhang:test-array-cast

Conversation

@manuzhang
Copy link
Member

@manuzhang manuzhang commented Dec 13, 2025

Which issue does this PR close?

Part of #2766

Rationale for this change

What changes are included in this PR?

  1. Implement tests for cast array to array
  2. Fix casting array to array for some element types.

How are these changes tested?

Added tests.

@manuzhang manuzhang force-pushed the test-array-cast branch 4 times, most recently from 864a25f to 92a7867 Compare December 14, 2025 15:23
@codecov-commenter
Copy link

codecov-commenter commented Dec 14, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.53%. Comparing base (f09f8af) to head (5f7b017).
⚠️ Report is 815 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2897      +/-   ##
============================================
+ Coverage     56.12%   59.53%   +3.40%     
- Complexity      976     1376     +400     
============================================
  Files           119      167      +48     
  Lines         11743    15496    +3753     
  Branches       2251     2569     +318     
============================================
+ Hits           6591     9225    +2634     
- Misses         4012     4971     +959     
- Partials       1140     1300     +160     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@manuzhang
Copy link
Member Author

Created #2914 for the failed test

  "...The value -96833550.[]7BD of the type "DEC..." did not equal "...The value -96833550.[0]7BD of the type "DEC..." (CometCastSuite.scala:1328)

@manuzhang
Copy link
Member Author

Submitted #2916 to fix the failed test.

Ok(cast_with_options(&array, to_type, &CAST_OPTIONS)?)
(List(from), List(to))
if can_cast_types(from_type, to_type)
|| (matches!(from.data_type(), Decimal128(_, _))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the only case where we have a spark cast but no DF/arrow cast?

}

private def generateArrays(rowSize: Int, elementType: DataType): DataFrame = {
private def generateArrays(rowNum: Int, elementType: DataType): DataFrame = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit : might be an unintended change ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intended since rowNum (meaning the number of rows) makes more sense to me.

@github-actions
Copy link

Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.

@github-actions github-actions bot added the Stale label Mar 13, 2026
@github-actions github-actions bot removed the Stale label Mar 18, 2026
@manuzhang manuzhang force-pushed the test-array-cast branch 2 times, most recently from d9770c7 to fae74f6 Compare March 18, 2026 07:26
@coderfender
Copy link
Contributor

Taking a look

@manuzhang
Copy link
Member Author

@coderfender Sorry, there are more test failures that I need to fix first.

@manuzhang manuzhang force-pushed the test-array-cast branch 2 times, most recently from f0455fd to b93600c Compare March 18, 2026 09:12
@manuzhang
Copy link
Member Author

@coderfender It's ready for review now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants