Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-48943][SQL][TESTS][FOLLOWUP] Fix the h2 filter push-down test case failure with ANSI mode off #47472

Closed
wants to merge 1 commit into from

Conversation

wayneguow
Copy link
Contributor

@wayneguow wayneguow commented Jul 24, 2024

What changes were proposed in this pull request?

This PR aims to fix the h2 filter push-down test case failure with ANSI mode off.

Why are the changes needed?

Fix test failure.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manually test of the whole JDBCV2Suite with ANSI mode off and on.

  1. Method One: with IDEA.
  • ANSI mode off: with SPARK_ANSI_SQL_MODE=false
image
  • ANSI mode on: without SPARK_ANSI_SQL_MODE env variable
image
  1. Method Two: with commands.
  • ANSI mode off
SPARK_ANSI_SQL_MODE=false
$ build/sbt
> project sql
> testOnly org.apache.spark.sql.jdbc.JDBCV2Suite
  • ANSI mode on
UNSET SPARK_ANSI_SQL_MODE
$ build/sbt
> project sql
> testOnly org.apache.spark.sql.jdbc.JDBCV2Suite

Test results:

  1. The issue on current master branch
  • with SPARK_ANSI_SQL_MODE=false, test failed
  • without SPARK_ANSI_SQL_MODE env variable, test passed
  1. Fixed with new test code
  • with SPARK_ANSI_SQL_MODE=false, test passed
  • without SPARK_ANSI_SQL_MODE env variable, test passed

Was this patch authored or co-authored using generative AI tooling?

No.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to fail even in ANSI mode. Did you check both?

[info] - show tables *** FAILED *** (262 milliseconds)
[info]   Results do not match for query:

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-48943][SQL][TESTS] Fix the h2 filter push-down test case failure with ANSI mode off [SPARK-48943][SQL][TESTS][FOLLOWUP] Fix the h2 filter push-down test case failure with ANSI mode off Jul 24, 2024
@dongjoon-hyun
Copy link
Member

FYI, we need to use [FOLLOWUP] in the follow-up PR title because we reuse the JIRA ID, @wayneguow .

@wayneguow
Copy link
Contributor Author

wayneguow commented Jul 24, 2024

@dongjoon-hyun Thank you for your timely review. As I want to fix this issue as soon as possible, there are some minor issues with the previous code. I have just pushed the new version.

@dongjoon-hyun
Copy link
Member

NP. Thank you for the quick fix.

@wayneguow
Copy link
Contributor Author

And I will add some content to make it more clearly in How was this patch tested? part.

@HyukjinKwon
Copy link
Member

Merged to master.

@wayneguow
Copy link
Contributor Author

The new GA(Build / Non-ANSI (master, Hadoop 3, JDK 17, Scala 2.13) result seems ok: https://github.com/apache/spark/actions/runs/10086623495. Thank you all @dongjoon-hyun @HyukjinKwon .

ilicmarkodb pushed a commit to ilicmarkodb/spark that referenced this pull request Jul 29, 2024
…t case failure with ANSI mode off

### What changes were proposed in this pull request?

This PR aims to fix the `h2` filter push-down test case failure with ANSI mode off.

### Why are the changes needed?

Fix test failure.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manually test of the whole `JDBCV2Suite` with ANSI mode off and on.

1. Method One: with IDEA.
-  ANSI mode off: with `SPARK_ANSI_SQL_MODE=false`
<img width="1066" alt="image" src="https://github.com/user-attachments/assets/13ec8ff4-0699-4f3e-95c4-74f53d9824fe">

-  ANSI mode on: without `SPARK_ANSI_SQL_MODE` env variable
<img width="1066" alt="image" src="https://github.com/user-attachments/assets/8434bf0c-b332-4663-965c-0d17d60da78a">

2. Method Two: with commands.
- ANSI mode off
```
SPARK_ANSI_SQL_MODE=false
$ build/sbt
> project sql
> testOnly org.apache.spark.sql.jdbc.JDBCV2Suite
```

- ANSI mode on
```
UNSET SPARK_ANSI_SQL_MODE
$ build/sbt
> project sql
> testOnly org.apache.spark.sql.jdbc.JDBCV2Suite
```

Test results:
1. The issue on current `master` branch
-  with `SPARK_ANSI_SQL_MODE=false`, test failed
-  without `SPARK_ANSI_SQL_MODE` env variable, test passed
2. Fixed with new test code
-  with `SPARK_ANSI_SQL_MODE=false`, test passed
-  without `SPARK_ANSI_SQL_MODE` env variable, test passed

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#47472 from wayneguow/fix_h2.

Authored-by: Wei Guo <guow93@gmail.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
fusheng-rd pushed a commit to fusheng-rd/spark that referenced this pull request Aug 6, 2024
…t case failure with ANSI mode off

### What changes were proposed in this pull request?

This PR aims to fix the `h2` filter push-down test case failure with ANSI mode off.

### Why are the changes needed?

Fix test failure.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manually test of the whole `JDBCV2Suite` with ANSI mode off and on.

1. Method One: with IDEA.
-  ANSI mode off: with `SPARK_ANSI_SQL_MODE=false`
<img width="1066" alt="image" src="https://github.com/user-attachments/assets/13ec8ff4-0699-4f3e-95c4-74f53d9824fe">

-  ANSI mode on: without `SPARK_ANSI_SQL_MODE` env variable
<img width="1066" alt="image" src="https://github.com/user-attachments/assets/8434bf0c-b332-4663-965c-0d17d60da78a">

2. Method Two: with commands.
- ANSI mode off
```
SPARK_ANSI_SQL_MODE=false
$ build/sbt
> project sql
> testOnly org.apache.spark.sql.jdbc.JDBCV2Suite
```

- ANSI mode on
```
UNSET SPARK_ANSI_SQL_MODE
$ build/sbt
> project sql
> testOnly org.apache.spark.sql.jdbc.JDBCV2Suite
```

Test results:
1. The issue on current `master` branch
-  with `SPARK_ANSI_SQL_MODE=false`, test failed
-  without `SPARK_ANSI_SQL_MODE` env variable, test passed
2. Fixed with new test code
-  with `SPARK_ANSI_SQL_MODE=false`, test passed
-  without `SPARK_ANSI_SQL_MODE` env variable, test passed

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#47472 from wayneguow/fix_h2.

Authored-by: Wei Guo <guow93@gmail.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
attilapiros pushed a commit to attilapiros/spark that referenced this pull request Oct 4, 2024
…t case failure with ANSI mode off

### What changes were proposed in this pull request?

This PR aims to fix the `h2` filter push-down test case failure with ANSI mode off.

### Why are the changes needed?

Fix test failure.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manually test of the whole `JDBCV2Suite` with ANSI mode off and on.

1. Method One: with IDEA.
-  ANSI mode off: with `SPARK_ANSI_SQL_MODE=false`
<img width="1066" alt="image" src="https://github.com/user-attachments/assets/13ec8ff4-0699-4f3e-95c4-74f53d9824fe">

-  ANSI mode on: without `SPARK_ANSI_SQL_MODE` env variable
<img width="1066" alt="image" src="https://github.com/user-attachments/assets/8434bf0c-b332-4663-965c-0d17d60da78a">

2. Method Two: with commands.
- ANSI mode off
```
SPARK_ANSI_SQL_MODE=false
$ build/sbt
> project sql
> testOnly org.apache.spark.sql.jdbc.JDBCV2Suite
```

- ANSI mode on
```
UNSET SPARK_ANSI_SQL_MODE
$ build/sbt
> project sql
> testOnly org.apache.spark.sql.jdbc.JDBCV2Suite
```

Test results:
1. The issue on current `master` branch
-  with `SPARK_ANSI_SQL_MODE=false`, test failed
-  without `SPARK_ANSI_SQL_MODE` env variable, test passed
2. Fixed with new test code
-  with `SPARK_ANSI_SQL_MODE=false`, test passed
-  without `SPARK_ANSI_SQL_MODE` env variable, test passed

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#47472 from wayneguow/fix_h2.

Authored-by: Wei Guo <guow93@gmail.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants