Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-13507: [R] LTO job on CRAN fails #10849

Closed
wants to merge 5 commits into from

Conversation

jonkeane
Copy link
Member

@jonkeane jonkeane commented Aug 2, 2021

A test to see if we can (for now) build r-debug before using it

@github-actions
Copy link

github-actions bot commented Aug 2, 2021

git clone https://github.com/nealrichardson/r-debug.git
git checkout lto
cd r-debug
./buildall.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A possibly faster way to do this is to add a new job like for the r-sanitizer, and add dockerfile like https://github.com/nealrichardson/r-debug/blob/lto/r-debug-5/Dockerfile:

FROM wch1/r-debug

# RDlto: R built with --enable-lto
RUN wget https://raw.githubusercontent.com/nealrichardson/r-debug/lto/r-devel/buildR.sh
RUN buildR.sh lto
RUN RDlto -q -e 'install.packages("pak", repos = "https://r-lib.github.io/p/pak/dev")'
RUN RDlto -q -e 'pak::pkg_install(c("devtools", "Rcpp", "roxygen2", "testthat", "memoise", "rmarkdown"))'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonkeane
Copy link
Member Author

jonkeane commented Aug 2, 2021

@github-actions crossbow submit test-r-r_hub-debian-gcc-devel-lto-latest

@jonkeane
Copy link
Member Author

jonkeane commented Aug 2, 2021

@github-actions crossbow submit test-r-rhub-debian-gcc-devel-lto-latest

This should fail with the CRAN error at https://www.stats.ox.ac.uk/pub/bdr/LTO/arrow.out

@github-actions
Copy link

github-actions bot commented Aug 2, 2021

Revision: e278e8a

Submitted crossbow builds: ursacomputing/crossbow @ actions-693

Task Status
test-r-rhub-debian-gcc-devel-lto-latest Azure

@jonkeane
Copy link
Member Author

jonkeane commented Aug 3, 2021

@github-actions crossbow submit test-r-rhub-debian-gcc-devel-lto-latest

@github-actions
Copy link

github-actions bot commented Aug 3, 2021

Revision: d27fd38

Submitted crossbow builds: ursacomputing/crossbow @ actions-703

Task Status
test-r-rhub-debian-gcc-devel-lto-latest Azure

@jonkeane
Copy link
Member Author

jonkeane commented Aug 3, 2021

I also ran this locally removing all of the -fno-lto flags that were added in ARROW-13199, and I get a segfault there as well (this is at the same point as the crossbow build above which tries to remove that flag only when we know we're on LTO):

16: tools:::.test_load_package("arrow", "/arrow/r/check/arrow.Rcheck/00LOCK-arrow/00new")
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault
ERROR: loading failed

So it looks like we need to do something more to be able to build/link on the CRAN LTO check.

@@ -75,6 +79,7 @@ ${CMAKE} -DARROW_BOOST_USE_SHARED=OFF \
-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON \
-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON \
-DCMAKE_UNITY_BUILD=${CMAKE_UNITY_BUILD:-ON} \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=${ARROW_LTO:-ON} \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be ${ARROW_LTO:-$ARROW_DEFAULT_PARAM}? As it currently is, it looks like LTO will always be turned on.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed this to -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=${ARROW_LTO:-OFF} Confusingly, $ARROW_DEFAULT_PARAM here is about adding extra features like compression libraries (based on $LIBARROW_MINIMAL above).

@@ -45,6 +45,10 @@ else
ARROW_DEFAULT_PARAM="OFF"
fi

if echo "$ARROW_R_CXXFLAGS" | grep -q "flto"; then
ARROW_LTO="ON"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this have an else clause setting ARROW_LTO to "OFF"?

@jonkeane
Copy link
Member Author

jonkeane commented Aug 3, 2021

@github-actions crossbow submit test-r-rhub-debian-gcc-devel-lto-latest

@github-actions
Copy link

github-actions bot commented Aug 3, 2021

Revision: 02bd56a

Submitted crossbow builds: ursacomputing/crossbow @ actions-707

Task Status
test-r-rhub-debian-gcc-devel-lto-latest Azure

@jonkeane
Copy link
Member Author

jonkeane commented Aug 3, 2021

@github-actions crossbow submit test-r-rhub-debian-gcc-devel-lto-latest

@github-actions
Copy link

github-actions bot commented Aug 3, 2021

Revision: 6f2f0f7

Submitted crossbow builds: ursacomputing/crossbow @ actions-709

Task Status
test-r-rhub-debian-gcc-devel-lto-latest Azure

@jonkeane
Copy link
Member Author

jonkeane commented Aug 3, 2021

@github-actions crossbow submit test-r-rhub-debian-gcc-devel-lto-latest

@github-actions
Copy link

github-actions bot commented Aug 3, 2021

Revision: 6f2f0f7

Submitted crossbow builds: ursacomputing/crossbow @ actions-711

Task Status
test-r-rhub-debian-gcc-devel-lto-latest Azure

@jonkeane jonkeane force-pushed the ARROW-13507-r-lto branch from 6f2f0f7 to 2a8fc19 Compare August 4, 2021 14:07
@jonkeane
Copy link
Member Author

jonkeane commented Aug 4, 2021

@github-actions crossbow submit test-r-rhub-debian-gcc-devel-lto-latest

@github-actions
Copy link

github-actions bot commented Aug 4, 2021

Revision: 2a8fc19

Submitted crossbow builds: ursacomputing/crossbow @ actions-713

Task Status
test-r-rhub-debian-gcc-devel-lto-latest Azure

nealrichardson pushed a commit to nealrichardson/arrow that referenced this pull request Aug 5, 2021
A test to see if we can (for now) build r-debug before using it

Closes apache#10849 from jonkeane/ARROW-13507-r-lto

Authored-by: Jonathan Keane <jkeane@gmail.com>
Signed-off-by: Neal Richardson <neal.p.richardson@gmail.com>
michalursa pushed a commit to michalursa/arrow that referenced this pull request Aug 17, 2021
A test to see if we can (for now) build r-debug before using it

Closes apache#10849 from jonkeane/ARROW-13507-r-lto

Authored-by: Jonathan Keane <jkeane@gmail.com>
Signed-off-by: Neal Richardson <neal.p.richardson@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants