You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In our recent effort to polish and tighten up ADAM, the focus has been to use the ADAM repo to store primarily lower-level ETL tools that make it easy to load, store, filter, join, etc. all different kinds of genomics data. Ideally, we'd use the lowest level abstractions given to us, which are the Spark RDDs. As such, if you get back an RDD of genomics objects and want to filter them, you should call .filter() on the RDD, rather than some other ADAM-specific function that essentially wraps the same functionality. The only load functions that should take predicates are ones that push the predicate into the Parquet code.
The text was updated successfully, but these errors were encountered:
laserson
changed the title
Eliminate applyPredicate from a
Eliminate applyPredicate from ADAMContextMar 2, 2015
In our recent effort to polish and tighten up ADAM, the focus has been to use the ADAM repo to store primarily lower-level ETL tools that make it easy to load, store, filter, join, etc. all different kinds of genomics data. Ideally, we'd use the lowest level abstractions given to us, which are the Spark RDDs. As such, if you get back an RDD of genomics objects and want to filter them, you should call
.filter()
on the RDD, rather than some other ADAM-specific function that essentially wraps the same functionality. The only load functions that should take predicates are ones that push the predicate into the Parquet code.The text was updated successfully, but these errors were encountered: