-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added spark 3.4 to the project #346
Conversation
Missing documentation to make it final |
Test summary report 📊
|
eaa7afb
to
cfbd319
Compare
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #346 +/- ##
=======================================
Coverage 97.61% 97.61%
=======================================
Files 62 62
Lines 1253 1253
Branches 15 15
=======================================
Hits 1223 1223
Misses 30 30
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report in Codecov by Sentry.
|
ff49ef9
to
c34c99b
Compare
Co-authored-by: Eduardo Ruiz Alonso <edu.r26@gmail.com>
df5a3ad
to
a803eed
Compare
* Added spark 3.4 to the project * Documentation and CI * Update .github/labeler.yml Co-authored-by: Eduardo Ruiz Alonso <edu.r26@gmail.com> * Documentation and CI * Latest rebase * All "column not found" errors with builders * Default spark version setted to 3.4 * Documentation error --------- Co-authored-by: Eduardo Ruiz Alonso <edu.r26@gmail.com>
SPARK 3.4, what else
Description
This pr adds spark 3.4.0 to the CI and adds all needed changes to the schemas to use them in the project and tests.
To avoid problems with the new error messages in unfunded columns I've implemented a way to make it simple using ColumnNotFoundError builder
Related Issue and dependencies
How Has This Been Tested?