Skip to content

Commit

Permalink
[SPARK-45640][SQL][TESTS] Fix flaky ProtobufCatalystDataConversionSuite
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
The pr aims to fix flaky ProtobufCatalystDataConversionSuite.

As can be seen from the GA test below
https://github.com/panbingkun/spark/actions/runs/6612141762/job/17982780917
<img width="988" alt="image" src="https://github.com/apache/spark/assets/15246973/fd1931ba-d858-4c59-9d5c-3a1353e3e005">

When `data.get(0)` is null, `data.get(0).asInstanceOf[Array[Byte]].isEmpty` will be thrown `java.lang.NullPointerException`.

<img width="491" alt="image" src="https://github.com/apache/spark/assets/15246973/4cf83e36-1c4a-4b65-b119-1152b026ed29">

### Why are the changes needed?
Fix flaky ProtobufCatalystDataConversionSuite.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Pass GA.
Manually test:
```
./build/sbt "protobuf/testOnly org.apache.spark.sql.protobuf.ProtobufCatalystDataConversionSuite"
```
<img width="763" alt="image" src="https://github.com/apache/spark/assets/15246973/95e76b2e-5b09-4268-81bf-df20dd262b66">

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes #43493 from panbingkun/SPARK-45640.

Authored-by: panbingkun <pbk1982@gmail.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
panbingkun authored and HyukjinKwon committed Oct 24, 2023
1 parent 4eac024 commit a6503b2
Showing 1 changed file with 1 addition and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@ class ProtobufCatalystDataConversionSuite
data != null &&
(data.get(0) == defaultValue ||
(dt.fields(0).dataType == BinaryType &&
data.get(0) != null &&
data.get(0).asInstanceOf[Array[Byte]].isEmpty)))
data = generator().asInstanceOf[Row]

Expand Down

0 comments on commit a6503b2

Please sign in to comment.