Improve TypedParquetTuple #1302

[JiJiTang]

No description provided.

[JiJiTang]

Hi @johnynek here's my commit with macros to generate Parquet read/write support instances. And I'm wondering maybe need to create a module like "scalding-parquet-macro" to put all the macros there and use them from "TypedParquet". Thank you very much for reviewing it and please let me know your thoughts.

[johnynek]

Let's make one Injection and use that:

something like:

val injection: Injection[Any, String] = KryoInjection.andThen(Injection.connect[Array[Byte], GZippedBase64String, String])

[JiJiTang]

fixed, that's how injection should be used:P

[johnynek]

Thanks a ton! I really appreciate how much work this is. I think we can make it super easy to read and write parquet: just as easy as TypedTsv[(Int, Int, String)]("mydata") is, except with all the benefits of Parquet.

[JiJiTang]

Hi @johnynek thank you very much for your kind words and your code review and also your good advices. I am so glad to contribute to Scalding. Please check my latest commit.

[johnynek]

Looks great! thanks for making the changes so quickly. Looks very close to merging.

[johnynek]

Closes #1302

[JiJiTang]

Hi @johnynek thanks a lot. I've just added another commit. Please let me know if any problem before good to go.

[johnynek]

Looks good to me!

Will merge when green. Thanks a lot.

Next up: macros to create Filters from scala code:

Filter[SomeClass] {
  _.y == "foo"
}
// returns:
// FilterApi.eq(binaryColumn("y"), Binary.fromString("foo"))

[JiJiTang]

Hi @johnynek , thank you very much for merging the PR. In parquet-mr/parquet-scala, there's a DSL for defining filter more fluidly. And good idea, macro generated filters would be even more smoothie.