Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform #30957

Closed
wants to merge 25 commits into from
Closed

[SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform #30957

wants to merge 25 commits into from

Commits on Dec 29, 2020

  1. [SPARK-31937][SQL] Support processing array and map type using spark … 

    …noserde mode
    @AngersZhuuuu
    AngersZhuuuu committed Dec 29, 2020
    Configuration menu
    Copy the full SHA
    adc9ded View commit details
    Browse the repository at this point in the history

  2. Update CatalystTypeConverters.scala

    @AngersZhuuuu
    AngersZhuuuu committed Dec 29, 2020
    Configuration menu
    Copy the full SHA
    6a7438b View commit details
    Browse the repository at this point in the history

  3. fix failed UT

    @AngersZhuuuu
    AngersZhuuuu committed Dec 29, 2020
    Configuration menu
    Copy the full SHA
    d3b9cec View commit details
    Browse the repository at this point in the history

  4. Update SparkScriptTransformationSuite.scala

    @AngersZhuuuu
    AngersZhuuuu committed Dec 29, 2020
    Configuration menu
    Copy the full SHA
    fdd5225 View commit details
    Browse the repository at this point in the history

  5. Update BaseScriptTransformationSuite.scala

    @AngersZhuuuu
    AngersZhuuuu committed Dec 29, 2020
    Configuration menu
    Copy the full SHA
    aa16c8f View commit details
    Browse the repository at this point in the history

  6. Update BaseScriptTransformationExec.scala

    @AngersZhuuuu
    AngersZhuuuu committed Dec 29, 2020
    Configuration menu
    Copy the full SHA
    092c927 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'master' into SPARK-31937

    @AngersZhuuuu
    AngersZhuuuu committed Dec 29, 2020
    Configuration menu
    Copy the full SHA
    9761c0e View commit details
    Browse the repository at this point in the history

  8. Update BaseScriptTransformationSuite.scala

    @AngersZhuuuu
    AngersZhuuuu committed Dec 29, 2020
    Configuration menu
    Copy the full SHA
    28ad7fa View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2021

  1. Merge branch 'master' into SPARK-31937

    @AngersZhuuuu
    AngersZhuuuu committed Jan 4, 2021
    Configuration menu
    Copy the full SHA
    9ac75fc View commit details
    Browse the repository at this point in the history

  2. Update BaseScriptTransformationExec.scala

    @AngersZhuuuu
    AngersZhuuuu committed Jan 4, 2021
    Configuration menu
    Copy the full SHA
    33d8b5b View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2021

  1. follow comment

    @AngersZhuuuu
    AngersZhuuuu committed Feb 4, 2021
    Configuration menu
    Copy the full SHA
    63f07eb View commit details
    Browse the repository at this point in the history

  2. Update BaseScriptTransformationExec.scala

    @AngersZhuuuu
    AngersZhuuuu committed Feb 4, 2021
    Configuration menu
    Copy the full SHA
    b631b70 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2021

  1. follow comment

    @AngersZhuuuu
    AngersZhuuuu committed Feb 5, 2021
    Configuration menu
    Copy the full SHA
    b7e7f92 View commit details
    Browse the repository at this point in the history

  2. follow comment

    @AngersZhuuuu
    AngersZhuuuu committed Feb 5, 2021
    Configuration menu
    Copy the full SHA
    8dec5a1 View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2021

  1. Update BaseScriptTransformationExec.scala

    @AngersZhuuuu
    AngersZhuuuu committed Feb 6, 2021
    Configuration menu
    Copy the full SHA
    529d54d View commit details
    Browse the repository at this point in the history

  2. Avoid construct JsonToStructs repeated

    @AngersZhuuuu
    AngersZhuuuu committed Feb 6, 2021
    Configuration menu
    Copy the full SHA
    4f0e78f View commit details
    Browse the repository at this point in the history

  3. remove unused UT

    @AngersZhuuuu
    AngersZhuuuu committed Feb 6, 2021
    Configuration menu
    Copy the full SHA
    ed8c54c View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2021

  1. Update sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScr… 

    …iptTransformationExec.scala

Co-authored-by: Hyukjin Kwon <gurwls223@gmail.com>
    @AngersZhuuuu @HyukjinKwon
    AngersZhuuuu and HyukjinKwon authored Apr 16, 2021
    Configuration menu
    Copy the full SHA
    520f4b8 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into SPARK-31937

    @AngersZhuuuu
    AngersZhuuuu committed Apr 16, 2021
    Configuration menu
    Copy the full SHA
    97f9d58 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2021

  1. [SPARK-35097][SQL] Add column name to SparkUpgradeException about anc… 

    …ient datetime
    @AngersZhuuuu
    AngersZhuuuu committed Apr 18, 2021
    Configuration menu
    Copy the full SHA
    b5a4268 View commit details
    Browse the repository at this point in the history

  2. Revert "[SPARK-35097][SQL] Add column name to SparkUpgradeException a… 

    …bout ancient datetime"

This reverts commit b5a4268.
    @AngersZhuuuu
    AngersZhuuuu committed Apr 18, 2021
    Configuration menu
    Copy the full SHA
    76a746e View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2021

  1. fix UT

    @AngersZhuuuu
    AngersZhuuuu committed Apr 19, 2021
    Configuration menu
    Copy the full SHA
    6aa05fc View commit details
    Browse the repository at this point in the history

  2. Revert "fix UT" 

    This reverts commit 6aa05fc.
    @AngersZhuuuu
    AngersZhuuuu committed Apr 19, 2021
    Configuration menu
    Copy the full SHA
    9e3f808 View commit details
    Browse the repository at this point in the history

  3. fix UT

    @AngersZhuuuu
    AngersZhuuuu committed Apr 19, 2021
    Configuration menu
    Copy the full SHA
    3f51d27 View commit details
    Browse the repository at this point in the history

  4. Update sql-migration-guide.md

    @AngersZhuuuu
    AngersZhuuuu committed Apr 19, 2021
    Configuration menu
    Copy the full SHA
    adf8a66 View commit details
    Browse the repository at this point in the history