Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-13749][SQL] Faster pivot implementation for many distinct values with two phase aggregation #11583

Closed
wants to merge 16 commits into from
Closed

[SPARK-13749][SQL] Faster pivot implementation for many distinct values with two phase aggregation #11583

wants to merge 16 commits into from

Commits on Feb 11, 2016

  1. sketch

    @aray
    aray committed Feb 11, 2016
    Configuration menu
    Copy the full SHA
    75a101a View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2016

  1. Merge branch 'master' of https://github.com/apache/spark into fast-pivot

    @aray
    aray committed Feb 12, 2016
    Configuration menu
    Copy the full SHA
    e42cb36 View commit details
    Browse the repository at this point in the history

Commits on Feb 19, 2016

  1. working version

    @aray
    aray committed Feb 19, 2016
    Configuration menu
    Copy the full SHA
    b65cfb2 View commit details
    Browse the repository at this point in the history

  2. Support other datatypes

    @aray
    aray committed Feb 19, 2016
    Configuration menu
    Copy the full SHA
    adbcd1b View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2016

  1. working analyzer rule

    @aray
    aray committed Feb 23, 2016
    Configuration menu
    Copy the full SHA
    4b33b47 View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2016

  1. Some cleanup and unit tests

    @aray
    aray committed Feb 26, 2016
    Configuration menu
    Copy the full SHA
    d0b0b2f View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2016

  1. disable partial agg since it may cause data expansion in some situati… 

    …ons and at best doesent do much data reduction.
    @aray
    aray committed Mar 7, 2016
    Configuration menu
    Copy the full SHA
    7a662ba View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/apache/spark into fast-pivot 

    # Conflicts:
#	sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
    @aray
    aray committed Mar 7, 2016
    Configuration menu
    Copy the full SHA
    66e69db View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2016

  1. Unit tests and code restructuring

    @aray
    aray committed Mar 8, 2016
    Configuration menu
    Copy the full SHA
    b3ccf61 View commit details
    Browse the repository at this point in the history

  2. fix decimal unit test and remove map from case class constructor to s… 

    …peed things up
    @aray
    aray committed Mar 8, 2016
    Configuration menu
    Copy the full SHA
    bc0571d View commit details
    Browse the repository at this point in the history

  3. support for multiple aggregations and other cleanup

    @aray
    aray committed Mar 8, 2016
    Configuration menu
    Copy the full SHA
    cc9f49f View commit details
    Browse the repository at this point in the history

  4. cleanup import

    @aray
    aray committed Mar 8, 2016
    Configuration menu
    Copy the full SHA
    bffc7aa View commit details
    Browse the repository at this point in the history

  5. scalastyle

    @aray
    aray committed Mar 8, 2016
    Configuration menu
    Copy the full SHA
    359a374 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2016

  1. Addresses code review comments 

    * Remove threshold of 10 for pivot values in Analyzer
* Change updateFunction into a partial function so support can be checked without try/catch
* Scaladoc for PivotFirst
* Move children, inputTypes, nullable, and dataType to beginning
* Added comments
    @aray
    aray committed Apr 18, 2016
    Configuration menu
    Copy the full SHA
    28bbbef View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/apache/spark into fast-pivot 

    # Conflicts:
#	sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
    @aray
    aray committed Apr 18, 2016
    Configuration menu
    Copy the full SHA
    32e97a2 View commit details
    Browse the repository at this point in the history

  3. fixes scalastyle issue

    @aray
    aray committed Apr 18, 2016
    Configuration menu
    Copy the full SHA
    1723046 View commit details
    Browse the repository at this point in the history