<expr>$map_batches() may crash the session

[eitsupi]

It seems that if we run map_batches() consecutively as shown below, an error will eventually occur.

while(TRUE) {pl$select(pl$lit("a")$map_batches(\(...) 1)) |> print()}

Error: bad value
Error:
! Evaluation failed in `$select()`.
Caused by error:
! Evaluation failed in `$collect()`.
Caused by error in `PlRLazyFrame_filter()`:
! invalid formal argument list for "function"
Run `rlang::last_trace()` to see where the error occurred.

I think this indicates that something is wrong with inter-thread communication.
The same error occurs for the current r-polars, but it does not appear to occur when I set in_background=TRUE to have the UDF evaluated in a separate R session.

[eitsupi]

Example to confirm that the cause of the segmentation fault is in map_batches()

x <- 0
while(TRUE) {pl$select(pl$lit("a")); print(x <<- x + 1)}

x <- 0
while(TRUE) {pl$select(pl$lit("a")$map_batches(\(...) 1)); print(x <<- x + 1)}

[eitsupi]

May be related: pola-rs/r-polars#1253

The segmentation fault that occurs in this issue has nothing to do with the error, and according to my understanding has nothing to do with that issue.
See pola-rs/r-polars#1253 (comment)

[etiennebacher]

FYI I've run 5k iterations of this without any problem. Does it still appear for you ?

x <- 0
while(TRUE) {pl$select(pl$lit("a")$map_batches(\(...) 1)); print(x <<- x + 1)}

Edit: and of course it's right after I write this that the same error as in the first post occurs...