Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Cast function's support status #5936

Open
PHILO-HE opened this issue May 31, 2024 · 2 comments
Open

[VL] Cast function's support status #5936

PHILO-HE opened this issue May 31, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@PHILO-HE
Copy link
Contributor

PHILO-HE commented May 31, 2024

Description

Not any pair of types can be converted from each other. Spark has a check here.
Gluten cpp has some pre-check logic for cast expression. See Gluten code here.

This is the output from a tool proposed by #6392 ("!!" marks not supported cast case).

## cast validation passes: cast from BooleanType to ByteType
## cast validation passes: cast from BooleanType to ShortType
## cast validation passes: cast from BooleanType to IntegerType
## cast validation passes: cast from BooleanType to LongType
## cast validation passes: cast from BooleanType to FloatType
## cast validation passes: cast from BooleanType to DoubleType
## cast validation passes: cast from BooleanType to DecimalType(5,1)
## cast validation passes: cast from BooleanType to StringType
!! cast validation fails: cast from BooleanType to TimestampType

## cast validation passes: cast from ByteType to BooleanType
## cast validation passes: cast from ByteType to ShortType
## cast validation passes: cast from ByteType to IntegerType
## cast validation passes: cast from ByteType to LongType
## cast validation passes: cast from ByteType to FloatType
## cast validation passes: cast from ByteType to DoubleType
## cast validation passes: cast from ByteType to DecimalType(5,1)
## cast validation passes: cast from ByteType to StringType
## cast validation passes: cast from ByteType to BinaryType
!! cast validation fails: cast from ByteType to TimestampType

## cast validation passes: cast from ShortType to BooleanType
## cast validation passes: cast from ShortType to ByteType
## cast validation passes: cast from ShortType to IntegerType
## cast validation passes: cast from ShortType to LongType
## cast validation passes: cast from ShortType to FloatType
## cast validation passes: cast from ShortType to DoubleType
## cast validation passes: cast from ShortType to DecimalType(5,1)
## cast validation passes: cast from ShortType to StringType
## cast validation passes: cast from ShortType to BinaryType
!! cast validation fails: cast from ShortType to TimestampType

## cast validation passes: cast from IntegerType to BooleanType
## cast validation passes: cast from IntegerType to ByteType
## cast validation passes: cast from IntegerType to ShortType
## cast validation passes: cast from IntegerType to LongType
## cast validation passes: cast from IntegerType to FloatType
## cast validation passes: cast from IntegerType to DoubleType
## cast validation passes: cast from IntegerType to DecimalType(5,1)
## cast validation passes: cast from IntegerType to StringType
## cast validation passes: cast from IntegerType to BinaryType
!! cast validation fails: cast from IntegerType to TimestampType

## cast validation passes: cast from LongType to BooleanType
## cast validation passes: cast from LongType to ByteType
## cast validation passes: cast from LongType to ShortType
## cast validation passes: cast from LongType to IntegerType
## cast validation passes: cast from LongType to FloatType
## cast validation passes: cast from LongType to DoubleType
## cast validation passes: cast from LongType to DecimalType(5,1)
## cast validation passes: cast from LongType to StringType
## cast validation passes: cast from LongType to BinaryType
!! cast validation fails: cast from LongType to TimestampType

## cast validation passes: cast from FloatType to BooleanType
## cast validation passes: cast from FloatType to ByteType
## cast validation passes: cast from FloatType to ShortType
## cast validation passes: cast from FloatType to IntegerType
## cast validation passes: cast from FloatType to LongType
## cast validation passes: cast from FloatType to DoubleType
## cast validation passes: cast from FloatType to DecimalType(5,1)
## cast validation passes: cast from FloatType to StringType
!! cast validation fails: cast from FloatType to TimestampType

## cast validation passes: cast from DoubleType to BooleanType
## cast validation passes: cast from DoubleType to ByteType
## cast validation passes: cast from DoubleType to ShortType
## cast validation passes: cast from DoubleType to IntegerType
## cast validation passes: cast from DoubleType to LongType
## cast validation passes: cast from DoubleType to FloatType
## cast validation passes: cast from DoubleType to DecimalType(5,1)
## cast validation passes: cast from DoubleType to StringType
!! cast validation fails: cast from DoubleType to TimestampType

## cast validation passes: cast from DecimalType(5,1) to BooleanType
## cast validation passes: cast from DecimalType(5,1) to ByteType
## cast validation passes: cast from DecimalType(5,1) to ShortType
## cast validation passes: cast from DecimalType(5,1) to IntegerType
## cast validation passes: cast from DecimalType(5,1) to LongType
## cast validation passes: cast from DecimalType(5,1) to FloatType
## cast validation passes: cast from DecimalType(5,1) to DoubleType
## cast validation passes: cast from DecimalType(5,1) to StringType
!! cast validation fails: cast from DecimalType(5,1) to TimestampType

## cast validation passes: cast from StringType to BooleanType
## cast validation passes: cast from StringType to ByteType
## cast validation passes: cast from StringType to ShortType
## cast validation passes: cast from StringType to IntegerType
## cast validation passes: cast from StringType to LongType
## cast validation passes: cast from StringType to FloatType
## cast validation passes: cast from StringType to DoubleType
## cast validation passes: cast from StringType to DecimalType(5,1)
## cast validation passes: cast from StringType to BinaryType
!! cast validation fails: cast from StringType to DateType
!! cast validation fails: cast from StringType to TimestampType

## cast validation passes: cast from BinaryType to BooleanType
## cast validation passes: cast from BinaryType to ByteType
## cast validation passes: cast from BinaryType to ShortType
## cast validation passes: cast from BinaryType to IntegerType
## cast validation passes: cast from BinaryType to LongType
## cast validation passes: cast from BinaryType to FloatType
## cast validation passes: cast from BinaryType to DoubleType
## cast validation passes: cast from BinaryType to DecimalType(5,1)
## cast validation passes: cast from BinaryType to StringType
!! cast validation fails: cast from BinaryType to DateType
!! cast validation fails: cast from BinaryType to TimestampType

!! cast validation fails: cast from DateType to BooleanType
!! cast validation fails: cast from DateType to ByteType
!! cast validation fails: cast from DateType to ShortType
!! cast validation fails: cast from DateType to IntegerType
!! cast validation fails: cast from DateType to LongType
!! cast validation fails: cast from DateType to FloatType
!! cast validation fails: cast from DateType to DoubleType
!! cast validation fails: cast from DateType to DecimalType(5,1)
!! cast validation fails: cast from DateType to StringType
!! cast validation fails: cast from DateType to TimestampType

!! cast validation fails: cast from TimestampType to BooleanType
!! cast validation fails: cast from TimestampType to ByteType
!! cast validation fails: cast from TimestampType to ShortType
!! cast validation fails: cast from TimestampType to IntegerType
!! cast validation fails: cast from TimestampType to LongType
!! cast validation fails: cast from TimestampType to FloatType
!! cast validation fails: cast from TimestampType to DoubleType
!! cast validation fails: cast from TimestampType to DecimalType(5,1)
!! cast validation fails: cast from TimestampType to StringType
!! cast validation fails: cast from TimestampType to DateType

## cast validation passes: cast from NullType to BooleanType
## cast validation passes: cast from NullType to ByteType
## cast validation passes: cast from NullType to ShortType
## cast validation passes: cast from NullType to IntegerType
## cast validation passes: cast from NullType to LongType
## cast validation passes: cast from NullType to FloatType
## cast validation passes: cast from NullType to DoubleType
## cast validation passes: cast from NullType to DecimalType(5,1)
## cast validation passes: cast from NullType to StringType
## cast validation passes: cast from NullType to BinaryType
## cast validation passes: cast from NullType to DateType
!! cast validation fails: cast from NullType to TimestampType
## cast validation passes: cast from NullType to ArrayType(IntegerType,true)
## cast validation passes: cast from NullType to MapType(StringType,IntegerType,true)
## cast validation passes: cast from NullType to StructType(StructField(a,StringType,true), StructField(b,IntegerType,true))

!! cast validation fails: cast from ArrayType(IntegerType,true) to StringType
!! cast validation fails: cast from MapType(StringType,IntegerType,true) to StringType
!! cast validation fails: cast from StructType(StructField(a,StringType,true), StructField(b,IntegerType,true)) to StringType
@PHILO-HE PHILO-HE added the enhancement New feature or request label May 31, 2024
@FelixYBW
Copy link
Contributor

We do have the table to support cast between data types but not fully tested. The goal of this issue is to track the support and test status of all cast cases.

@zhli1142015 @Yohahaha @ulysses-you @jackylee-ch @kecookier @WangGuangxin

@PHILO-HE
Copy link
Contributor Author

Just updated the list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants