Commit 005f2a3
[SPARK-56168][PS][TESTS] Relax groupby diff-length assertions for pandas 3
### What changes were proposed in this pull request?
This PR updates the pandas-on-Spark diff-frame groupby length test in `test_groupby_diff_len.py`.
The changes are test-only:
- restructure the test with `subTest` to make the failing case easier to identify
- use `almost=True` for pandas 3.x
- add a short comment explaining that pandas 3 includes external group keys for `as_index=False` and can widen their dtype after aligning mismatched lengths
### Why are the changes needed?
The test behavior differs across pandas versions for mismatched-length external group keys.
With pandas 3.x, `groupby(..., as_index=False)` can include the external group key in the result and widen its dtype after alignment. That makes the strict equality used by this test fail due to dtype differences even though the grouped values still match.
This patch keeps the existing pandas 2 behavior untouched and relaxes the assertions only for pandas 3.x in this localized test.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Updated the related tests.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-5)
Closes #54969 from ueshin/issuse/SPARK-56168/diff_len.
Authored-by: Takuya Ueshin <ueshin@databricks.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>1 parent 75757e1 commit 005f2a3
1 file changed
Lines changed: 36 additions & 31 deletions
Lines changed: 36 additions & 31 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
| |||
46 | 47 | | |
47 | 48 | | |
48 | 49 | | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
80 | 85 | | |
81 | 86 | | |
82 | 87 | | |
| |||
0 commit comments