[R] as.Date fails going from timestamp[us] to timestamp[s]

[asfimport]

Using as.Date to convert from timestamp to date fails in Arrow even though this is fine in R.

library(arrow)
library(dplyr)
library(lubridate)

tf <- tempfile()
dir.create(tf)
tbl <- tibble::tibble(x = as_datetime('2022-05-05T00:00:01.676632'))
write_dataset(tbl, tf)
open_dataset(tf) %>%
  mutate(date = as.Date(x)) %>%
  collect()
#> Error in `collect()`:
#> ! Invalid: Casting from timestamp[us, tz=UTC] to timestamp[s, tz=UTC] would lose data: 1651708801676632
#> /home/nic2/arrow/cpp/src/arrow/compute/exec.cc:799  kernel_->exec(kernel_ctx_, input, out)
#> /home/nic2/arrow/cpp/src/arrow/compute/exec.cc:767  ExecuteSingleSpan(input, &output)
#> /home/nic2/arrow/cpp/src/arrow/compute/exec/expression.cc:597  executor->Execute( ExecBatch(std::move(arguments), all_scalar ? 1 : input.length), &listener)
#> /home/nic2/arrow/cpp/src/arrow/compute/exec/expression.cc:579  ExecuteScalarExpression(call->arguments[i], input, exec_context)
#> /home/nic2/arrow/cpp/src/arrow/compute/exec/project_node.cc:91  ExecuteScalarExpression(simplified_expr, target, plan()->exec_context())
#> /home/nic2/arrow/cpp/src/arrow/compute/exec/exec_plan.cc:573  iterator_.Next()
#> /home/nic2/arrow/cpp/src/arrow/record_batch.cc:337  ReadNext(&batch)
#> /home/nic2/arrow/cpp/src/arrow/record_batch.cc:351  ToRecordBatches()

tbl %>%
  mutate(date = as.Date(x))
#> # A tibble: 1 × 2
#>   x                   date      
#>   <dttm>              <date>    
#> 1 2022-05-05 00:00:01 2022-05-05

Reporter: Nicola Crane / @thisisnic
Assignee: Dewey Dunnington / @paleolimbot

PRs and other links:

• GitHub Pull Request #14935

_{Note: This issue was originally created as ARROW-17637. Please see the migration documentation for further details.}

[asfimport]

Neal Richardson / @nealrichardson:
The naive cast to date32() works:

> Array$create(lubridate::as_datetime('2022-05-05T00:00:01.676632'))
Array
<timestamp[us, tz=UTC]>
[
  2022-05-05 00:00:01.676632
]
> Array$create(lubridate::as_datetime('2022-05-05T00:00:01.676632'))$cast(date32())
Array
<date32[day]>
[
  2022-05-05
]

The issue looks to be in this extra cast, something about handling timezones: https://github.com/apache/arrow/blob/master/r/R/dplyr-funcs-datetime.R#L329

Basically, if x is timestamp type, we either need to keep the same unit from x (it's a parameter to the type, default is "s", hence the error), or pass the right cast option to allow truncation. (And probably not cast at all if it's already the same timezone.)

[asfimport]

Dewey Dunnington / @paleolimbot:
Issue resolved by pull request 14935
#14935