Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use UnaryLike instead of UnaryExpression [databricks] #11490

Merged
merged 1 commit into from
Sep 24, 2024

Conversation

abellina
Copy link
Collaborator

Closes #11479

This is a potential solution to #11479, where in Spark 4.0.0, Rand was changed to not subclass from UnaryExpression and instead it subclasses UnaryLike[Expression] (apache/spark#48004).

This required changing some of Cast as well since it also used UnaryExpression directly.

Draft to run all of CI and gather reviews on the approach.

@abellina
Copy link
Collaborator Author

build

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>
@abellina
Copy link
Collaborator Author

build

@abellina abellina changed the title Use UnaryLike instead of UnaryExpression/UnaryExecNode Use UnaryLike instead of UnaryExpression Sep 23, 2024
@abellina
Copy link
Collaborator Author

build

@abellina abellina marked this pull request as ready for review September 23, 2024 17:42
Copy link
Collaborator

@gerashegalov gerashegalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I would run it through Databricks tests as well

@abellina abellina changed the title Use UnaryLike instead of UnaryExpression Use UnaryLike instead of UnaryExpression [databricks] Sep 23, 2024
@abellina
Copy link
Collaborator Author

build

@abellina
Copy link
Collaborator Author

@gerashegalov good point, thanks! running them now.

@abellina
Copy link
Collaborator Author

build

@abellina abellina merged commit 6a9731f into NVIDIA:branch-24.10 Sep 24, 2024
45 checks passed
@abellina abellina deleted the unarylike branch September 24, 2024 13:13
@sameerz sameerz added the Spark 4.0+ Spark 4.0+ issues label Sep 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Spark 4.0+ Spark 4.0+ issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] spark400 build failed do not conform to class UnaryExprMeta's type parameter
4 participants