Skip to content

Commit

Permalink
[SPARK-49311][SQL] Make it possible for large 'interval second' value…
Browse files Browse the repository at this point in the history
…s to be cast to decimal

### What changes were proposed in this pull request?

Prior to this PR, `interval second` values where the number of microseconds needed to be represented by 19 digits could not be cast to decimal. This PR removes this gap.

```
scala> sql("select 1000000000000.000000::interval second").show(false)
+---------------------------------------------+
|CAST(1000000000000.000000 AS INTERVAL SECOND)|
+---------------------------------------------+
|INTERVAL '1000000000000' SECOND              |
+---------------------------------------------+

scala> sql("select 1000000000000.000000::interval second::decimal(38, 10)").show(false)
org.apache.spark.SparkArithmeticException: [NUMERIC_VALUE_OUT_OF_RANGE.WITH_SUGGESTION]  0 cannot be represented as Decimal(18, 6). If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error, and return NULL instead. SQLSTATE: 22003
```

### Why are the changes needed?

This change adds additional coverage.

### Does this PR introduce _any_ user-facing change?

Yes, users couldn't cast large second intervals to decimals earlier. Now, they can.

### How was this patch tested?

Unit test in `IntervalExpressionsSuite`.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#47808 from harshmotw-db/harshmotw-db/interval_decimal_fix.

Authored-by: Harsh Motwani <harsh.motwani@databricks.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
  • Loading branch information
harshmotw-db authored and MaxGekk committed Aug 26, 2024
1 parent ccef4df commit e10a789
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -902,7 +902,7 @@ object IntervalUtils extends SparkIntervalUtils {
case DAY => Decimal(v / MICROS_PER_DAY)
case HOUR => Decimal(v / MICROS_PER_HOUR)
case MINUTE => Decimal(v / MICROS_PER_MINUTE)
case SECOND => Decimal(v, Decimal.MAX_LONG_DIGITS, 6)
case SECOND => Decimal(v, Decimal.MAX_LONG_DIGITS + 1, 6)
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,16 @@ class IntervalExpressionsSuite extends SparkFunSuite with ExpressionEvalHelper {
checkEvaluation(ExtractIntervalSeconds("61 seconds 1 microseconds"), Decimal(1000001, 8, 6))
}

test("cast large seconds to decimal") {
checkEvaluation(
Cast(
Cast(Literal(Decimal("9223372036854.775807")), DayTimeIntervalType(3, 3)),
DecimalType(19, 6)
),
Decimal("9223372036854.775807")
)
}

test("multiply") {
def check(
interval: String,
Expand Down

0 comments on commit e10a789

Please sign in to comment.