Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Doric for spark 2.4!!!!! #207

Merged
merged 8 commits into from
May 9, 2022
Merged

Doric for spark 2.4!!!!! #207

merged 8 commits into from
May 9, 2022

Conversation

alfonsorr
Copy link
Member

Doric for spark 2.4 in scala 2.11

Description

Split the logic in doric to allow create it for spark 2.4

Related Issue

#205
Draft Pull request until mrpowers-io/spark-fast-tests#106 is solved

Motivation and Context

Publish doric for spark 2.4 😄

How Has This Been Tested?

Tested using a local published version of spark-fast-tests

  • This pull request contain appropriate tests?

@github-actions github-actions bot added spark_3.0 PR changes to spark 3.0 spark_3.1 PR changes to spark 3.1 spark_3.2 PR changes to spark 3.2 labels Apr 17, 2022
@github-actions github-actions bot added the CI/CD Continuous integration and continuous delivery label Apr 18, 2022
@alfonsorr alfonsorr force-pushed the feature/split_24_logic branch 3 times, most recently from d0edad5 to 15ba770 Compare April 18, 2022 15:55
@codecov
Copy link

codecov bot commented Apr 18, 2022 •
edited
Loading

Codecov Report

Merging #207 (b4432af) into main (1a1bd8e) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #207      +/-   ##
==========================================
- Coverage   95.99%   95.97%   -0.02%     
==========================================
  Files          49       53       +4     
  Lines         798      794       -4     
  Branches       11       13       +2     
==========================================
- Hits          766      762       -4     
  Misses         32       32
Flag Coverage Δ
spark-2.4.x 93.04% <100.00%> (?)
spark-3.0.x 94.76% <100.00%> (-0.05%) ⬇️
spark-3.1.x 95.93% <100.00%> (-0.04%) ⬇️
spark-3.2.x 95.93% <100.00%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
core/src/main/scala/doric/sem/CollectOps.scala 100.00% <ø> (ø)
...ore/src/main/scala/doric/syntax/ArrayColumns.scala 100.00% <ø> (ø)
...re/src/main/scala/doric/syntax/CommonColumns.scala 100.00% <ø> (ø)
core/src/main/scala/doric/syntax/MapColumns.scala 100.00% <ø> (ø)
...re/src/main/scala/doric/syntax/StringColumns.scala 100.00% <ø> (ø)
core/src/main/scala/doric/syntax/DateColumns.scala 100.00% <100.00%> (ø)
.../src/main/scala/doric/types/LiteralSparkType.scala 100.00% <100.00%> (ø)
core/src/main/scala/doric/types/SparkType.scala 93.85% <100.00%> (+0.20%) ⬆️
..._2.4_mount/scala/doric/syntax/StringColumn24.scala 100.00% <100.00%> (ø)
....0_3.1_3.2/scala/doric/syntax/ArrayColumns3x.scala 100.00% <100.00%> (ø)
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a1bd8e...b4432af. Read the comment docs.

@alfonsorr
Spark 2.4 test for some versions.
9ff51de 
Skip mdoc for 2.4 builds
@eruizalo eruizalo added the spark_2.4 PR changes to spark 2.4 label May 5, 2022
@alfonsorr @eruizalo
Apply suggestions from code review
b4432af 
Co-authored-by: Eduardo Ruiz Alonso <edu.r26@gmail.com>
@alfonsorr alfonsorr marked this pull request as ready for review May 9, 2022 10:33
@alfonsorr alfonsorr requested a review from a team as a code owner May 9, 2022 10:33
@alfonsorr alfonsorr requested review from eruizalo May 9, 2022 10:37
@eruizalo eruizalo merged commit 59d6780 into hablapps:main May 9, 2022
@eruizalo eruizalo linked an issue May 9, 2022 that may be closed by this pull request
Build doric for scala 2.11 #205
Closed
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eruizalo eruizalo Awaiting requested review from eruizalo

Assignees
No one assigned
Labels
CI/CD Continuous integration and continuous delivery spark_2.4 PR changes to spark 2.4 spark_3.0 PR changes to spark 3.0 spark_3.1 PR changes to spark 3.1 spark_3.2 PR changes to spark 3.2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Build doric for scala 2.11
2 participants
@alfonsorr @eruizalo