Skip to content
This repository has been archived by the owner on Dec 20, 2018. It is now read-only.

Feat no union #95

Closed
wants to merge 11 commits into from
Closed

Feat no union #95

wants to merge 11 commits into from

Commits on Oct 23, 2015

  1. instead of UnionRDD set multiple input paths on single RDD

    @koertkuipers
    koertkuipers committed Oct 23, 2015
    Configuration menu
    Copy the full SHA
    a8f51b7 View commit details
    Browse the repository at this point in the history

  2. make sure i can read from multiple paths with commas in them

    @koertkuipers
    koertkuipers committed Oct 23, 2015
    Configuration menu
    Copy the full SHA
    e5510fb View commit details
    Browse the repository at this point in the history

  3. also test for comma and glob together

    @koertkuipers
    koertkuipers committed Oct 23, 2015
    Configuration menu
    Copy the full SHA
    1a79272 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2015

  1. use avro reader schema to project required columns and force all data… 

    … to be of same type across part files
    @koertkuipers
    koertkuipers committed Nov 11, 2015
    Configuration menu
    Copy the full SHA
    b3b60f7 View commit details
    Browse the repository at this point in the history

  2. limit line length

    @koertkuipers
    koertkuipers committed Nov 11, 2015
    Configuration menu
    Copy the full SHA
    0330c95 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2015

  1. Merge branch 'master' into feat-no-union

    @koertkuipers
    koertkuipers committed Dec 12, 2015
    Configuration menu
    Copy the full SHA
    d01fc1c View commit details
    Browse the repository at this point in the history

  2. option to pick schema from last path. we pass in our paths in chronol… 

    …ogical order and this allows me to use most recent schema
    @koertkuipers
    koertkuipers committed Dec 12, 2015
    Configuration menu
    Copy the full SHA
    720ab68 View commit details
    Browse the repository at this point in the history

  3. i dont like scalastyle line length limits

    @koertkuipers
    koertkuipers committed Dec 12, 2015
    Configuration menu
    Copy the full SHA
    484fa8b View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2015

  1. add ability to specify data path or file to read avro schema from

    @koertkuipers
    koertkuipers committed Dec 18, 2015
    Configuration menu
    Copy the full SHA
    92f4aed View commit details
    Browse the repository at this point in the history

  2. i dont like scalastyle line length limits

    @koertkuipers
    koertkuipers committed Dec 18, 2015
    Configuration menu
    Copy the full SHA
    437b3f6 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2016

  1. Merge branch 'master' into feat-no-union

    @koertkuipers
    koertkuipers committed Apr 21, 2016
    Configuration menu
    Copy the full SHA
    2281f05 View commit details
    Browse the repository at this point in the history