Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-47411][SQL] Support StringInstr & FindInSet functions to work with collated strings #45643

Closed
wants to merge 53 commits into from
Closed

[SPARK-47411][SQL] Support StringInstr & FindInSet functions to work with collated strings #45643

wants to merge 53 commits into from

Commits on Mar 21, 2024

  1. Add support for instr and unit test in CollationStringExpressionsSuit… 

    …e.scala
    @miland-db
    miland-db committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    a4d3592 View commit details
    Browse the repository at this point in the history

  2. Correct code style

    @miland-db
    miland-db committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    eb2d7c5 View commit details
    Browse the repository at this point in the history

  3. Remove blank line from CollationStringExpressionsSuite.scala

    @miland-db
    miland-db committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    9340831 View commit details
    Browse the repository at this point in the history

  4. Correct comment indentation

    @miland-db
    miland-db committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    465e814 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. Add unit tests for INSTR operation

    @miland-db
    miland-db committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    f3f30d8 View commit details
    Browse the repository at this point in the history

  2. Add doGenCode for FindInSet

    @miland-db
    miland-db committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    9cb92d3 View commit details
    Browse the repository at this point in the history

  3. Rewrite unit tests for INSTR and FIND_IN_SET

    @miland-db
    miland-db committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    834be70 View commit details
    Browse the repository at this point in the history

  4. Correct return value when substr is not found in INSTR method

    @miland-db
    miland-db committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    db2453a View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. Update unit tests for StringInStr and FindInSet

    @miland-db
    miland-db committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    91b648a View commit details
    Browse the repository at this point in the history

  2. Remove tests on non-explicit default collation

    @miland-db
    miland-db committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    1062521 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'apache:master' into miland-db/substr-functions

    @miland-db
    miland-db authored Mar 25, 2024
    Configuration menu
    Copy the full SHA
    42700c7 View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2024

  1. Improve signature of testInStr

    @miland-db
    miland-db committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    427ea25 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into miland-db/substr-functions

    @miland-db
    miland-db authored Mar 26, 2024
    Configuration menu
    Copy the full SHA
    546b3b0 View commit details
    Browse the repository at this point in the history

  3. Remove E2E test for collation mismatch. This will be added in Implici… 

    …t Casting
    @miland-db
    miland-db committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    108d707 View commit details
    Browse the repository at this point in the history

  4. Resolve merge problems with master

    @miland-db
    miland-db committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    822ecd2 View commit details
    Browse the repository at this point in the history

  5. Improve scala style

    @miland-db
    miland-db committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    f730d05 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2024

  1. Solve whitespace scala style problem

    @miland-db
    miland-db committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    de7b591 View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. Add lazy val collationId

    @miland-db
    miland-db committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    f0ee8fd View commit details
    Browse the repository at this point in the history

  2. Remove repeated code

    @miland-db
    miland-db committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    4ac6885 View commit details
    Browse the repository at this point in the history

  3. Improve test format

    @miland-db
    miland-db committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    b931333 View commit details
    Browse the repository at this point in the history

  4. Improve indexOf method

    @miland-db
    miland-db committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    0fd51d5 View commit details
    Browse the repository at this point in the history

  5. Remove checks in return statement of collatedIndexOf method

    @miland-db
    miland-db committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    28fa7f0 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2024

  1. Merge branch 'master' into substr-functions

    @miland-db
    miland-db committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    bab96ac View commit details
    Browse the repository at this point in the history

  2. Add branch for collated findInSet

    @miland-db
    miland-db committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    4666aff View commit details
    Browse the repository at this point in the history

  3. Add branch for collation check in StringInstr

    @miland-db
    miland-db committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    ca8a37c View commit details
    Browse the repository at this point in the history

  4. Improve naming of collation aware methods

    @miland-db
    miland-db committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    4ffab78 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2024

  1. Improve java style

    @miland-db
    miland-db committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    037d6be View commit details
    Browse the repository at this point in the history

  2. Improve collationAwareIndexOf performance

    @miland-db
    miland-db committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    0a22909 View commit details
    Browse the repository at this point in the history

  3. Fix indentation

    @miland-db
    miland-db committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    b3be85d View commit details
    Browse the repository at this point in the history

  4. Add more tests for instr

    @miland-db
    miland-db committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    c4c0fe7 View commit details
    Browse the repository at this point in the history

  5. Add more tests

    @miland-db
    miland-db committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    5b29f76 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2024

  1. Remove collation match type checks

    @miland-db
    miland-db committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    877828e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into substr-functions 

    # Conflicts:
#	sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala
    @miland-db
    miland-db committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    038a071 View commit details
    Browse the repository at this point in the history

  3. Merge with the latest master

    @miland-db
    miland-db committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    b35a8ac View commit details
    Browse the repository at this point in the history

  4. Remove checkInputDataTypes

    @miland-db
    miland-db committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    8b06014 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2024

  1. Merge branch 'master' into substr-functions

    @miland-db
    miland-db committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    2c454af View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into substr-functions 

    # Conflicts:
#	sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala
    @miland-db
    miland-db committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    4dbc26e View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2024

  1. Refactor code and move it to CollationSupport

    @miland-db
    miland-db committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    fbd1c00 View commit details
    Browse the repository at this point in the history

  2. Improve codegen and run tests

    @miland-db
    miland-db committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    960af54 View commit details
    Browse the repository at this point in the history

  3. Unify collationAwareIndexOf for return value to have same semantics a… 

    …s indexOf
    @miland-db
    miland-db committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    05cd6c4 View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2024

  1. Break line at 100 chars

    @miland-db
    miland-db committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    053efa0 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. Add new version of getStringSearch

    @miland-db
    miland-db committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    c65d68e View commit details
    Browse the repository at this point in the history

  2. Rename StringInstr params and class in CollationSupport

    @miland-db
    miland-db committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    ae33a38 View commit details
    Browse the repository at this point in the history

  3. Go from nullSafeCodeGen to defineCodeGen

    @miland-db
    miland-db committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    be1b52c View commit details
    Browse the repository at this point in the history

  4. Refactor testing

    @miland-db
    miland-db committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    5894d2f View commit details
    Browse the repository at this point in the history

  5. Remove empty lines

    @miland-db
    miland-db committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    75dc0bd View commit details
    Browse the repository at this point in the history

  6. Improve CollationAware indexOf to have the same semantics as UTF8Stri… 

    …ng indexOf
    @miland-db
    miland-db committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    c712b4b View commit details
    Browse the repository at this point in the history

  7. Add new e2e test

    @miland-db
    miland-db committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    3c37f35 View commit details
    Browse the repository at this point in the history

  8. Revert unused import deletion

    @miland-db
    miland-db committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    cd860b9 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Fix codegen

    @miland-db
    miland-db committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    3fa2502 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. Merge branch 'master' into substr-functions 

    # Conflicts:
#	common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationSupport.java
#	sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala
    @miland-db
    miland-db committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    b12f176 View commit details
    Browse the repository at this point in the history

  2. Remove unused import

    @miland-db
    miland-db committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    b35d718 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. Add new tests

    @miland-db
    miland-db committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    1ee5ad6 View commit details
    Browse the repository at this point in the history