Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(datasets): Unpin delta-spark #780

Merged
merged 4 commits into from
Aug 21, 2024
Merged

fix(datasets): Unpin delta-spark #780

merged 4 commits into from
Aug 21, 2024

Conversation

felipemonroy
Copy link
Contributor

Description

Development notes

Checklist

  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the relevant RELEASE.md file
  • Added tests to cover my changes

@felipemonroy felipemonroy marked this pull request as ready for review August 13, 2024 00:00
@merelcht merelcht mentioned this pull request Aug 19, 2024
Closed
16 tasks
astrojuanlu
astrojuanlu approved these changes Aug 21, 2024
View reviewed changes
Copy link
Member

@astrojuanlu astrojuanlu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry it took us so long, thanks @felipemonroy for this one too!

@astrojuanlu astrojuanlu merged commit d4b4767 into kedro-org:main Aug 21, 2024
14 checks passed
@rwpurvis rwpurvis mentioned this pull request Aug 22, 2024
Closed
@rwpurvis
Copy link
Contributor

rwpurvis commented Aug 22, 2024 •
edited
Loading

Hi guys, latest delta-spark is 3.2.0, any reason for pinning below 3.0.0? The current pin limits pyspark to <3.5, and latest databricks runtimes use 3.5

@astrojuanlu
Copy link
Member

Hi @rwpurvis, no particular reason. I see 3.x is already 10 months old, we didn't check when we approved the PR (which was a net improvement over what we had before). Feel free to open a new one.

@rwpurvis rwpurvis mentioned this pull request Aug 22, 2024
Merged
4 tasks
merelcht pushed a commit to galenseilis/kedro-plugins that referenced this pull request Aug 27, 2024
@felipemonroy @astrojuanlu @merelcht
fix(datasets): Unpin delta-spark (kedro-org#780)
abeafa4 
Signed-off-by: Felipe Monroy <felipe.m02@gmail.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@astrojuanlu astrojuanlu astrojuanlu approved these changes

@noklam noklam noklam approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@felipemonroy @rwpurvis @astrojuanlu @noklam