-
Notifications
You must be signed in to change notification settings - Fork 707
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement equals[T](a: TypedPipe[T], b: TypedPipe[T]) #1238
Comments
equals[T](a: TypedPipe[T], b: TypedPipe[T]): Execution[Boolean] = (a.asKeys.size).outerJoin(b.asKeys.size)
.mapValues { case (left, right) => left == right }
.values
.groupAll
.forall(identity)
.values
.toIterableExecution.map(_.headOption.getOrElse(false)) should be close |
This could include a way to compute the difference between two typed pipes as well. |
@isnotinvain can you think about releasing some of the code you wrote for this? Where should we put it? |
Yes, definitely. Well, I've said this before, but we don't have a great place for "recipes" or helper functions that don't belong in scalding-core, but use nothing but scalding. So, not sure, I guess somewhere in this repo :p |
Doesn't include equals, but equals is basically pipe1.diff(pipe2).isEmpty so we can add that on top if needed. |
You can create variants that return
Execution[Boolean]
andValuePipe[Boolean]
.related to #520
The text was updated successfully, but these errors were encountered: