Skip to content

Commit

Permalink
[SPARK-45588][SPARK-45640][SQL][TESTS][3.5] Fix flaky ProtobufCatalys…
Browse files Browse the repository at this point in the history
…tDataConversionSuite

### What changes were proposed in this pull request?
The pr aims to fix flaky ProtobufCatalystDataConversionSuite, include:
- Fix the type check (when the random value was empty array, we didn't skip it. Original intention is to skip default values for types.) [SPARK-45588]
- When data.get(0) is null, data.get(0).asInstanceOf[Array[Byte]].isEmpty will be thrown java.lang.NullPointerException. [SPARK-45640]

Backport above to branch 3.5.
Master branch pr: #43424 & #43493

### Why are the changes needed?
Fix flaky ProtobufCatalystDataConversionSuite.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
- Pass GA
- Manually test

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes #43521 from panbingkun/branch-3.5_SPARK-45640.

Authored-by: panbingkun <pbk1982@gmail.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
panbingkun authored and HyukjinKwon committed Oct 25, 2023
1 parent ddccf5a commit 26f6663
Showing 1 changed file with 2 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,8 @@ class ProtobufCatalystDataConversionSuite
while (
data != null &&
(data.get(0) == defaultValue ||
(dt == BinaryType &&
(dt.fields(0).dataType == BinaryType &&
data.get(0) != null &&
data.get(0).asInstanceOf[Array[Byte]].isEmpty)))
data = generator().asInstanceOf[Row]

Expand Down

0 comments on commit 26f6663

Please sign in to comment.