Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement equals[T](a: TypedPipe[T], b: TypedPipe[T]) #1238

Closed
johnynek opened this issue Mar 30, 2015 · 5 comments
Closed

Implement equals[T](a: TypedPipe[T], b: TypedPipe[T]) #1238

johnynek opened this issue Mar 30, 2015 · 5 comments
Labels

Comments

@johnynek
Copy link
Collaborator

You can create variants that return Execution[Boolean] and ValuePipe[Boolean].

related to #520

@johnynek
Copy link
Collaborator Author

equals[T](a: TypedPipe[T], b: TypedPipe[T]): Execution[Boolean] = (a.asKeys.size).outerJoin(b.asKeys.size)
  .mapValues { case (left, right) => left == right }
  .values
  .groupAll
  .forall(identity)
  .values
  .toIterableExecution.map(_.headOption.getOrElse(false))

should be close

@isnotinvain
Copy link
Contributor

This could include a way to compute the difference between two typed pipes as well.

@johnynek
Copy link
Collaborator Author

@isnotinvain can you think about releasing some of the code you wrote for this? Where should we put it?

@isnotinvain
Copy link
Contributor

Yes, definitely. Well, I've said this before, but we don't have a great place for "recipes" or helper functions that don't belong in scalding-core, but use nothing but scalding. So, not sure, I guess somewhere in this repo :p

@isnotinvain
Copy link
Contributor

#1266

Doesn't include equals, but equals is basically pipe1.diff(pipe2).isEmpty so we can add that on top if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants