Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[R-package] Add an R GPU job in CI #3780

Closed
jameslamb opened this issue Jan 18, 2021 · 9 comments
Closed

[R-package] Add an R GPU job in CI #3780

jameslamb opened this issue Jan 18, 2021 · 9 comments

Comments

@jameslamb
Copy link
Collaborator

Summary

CMake-based builds of the R package support building a GPU-accelerated version of LightGBM (https://github.com/microsoft/LightGBM/blob/master/R-package/README.md#installing-a-gpu-enabled-build).

This feature is not currently tested in a continuous integration (CI) job, but it should be.

Motivation

The lack of automated tests for the R GPU package means it can be broken by changes in this repo, and we might not find out until those become user-facing issues (like the one reported in #3765 ).

Description

To close this issue, add a job to https://github.com/microsoft/LightGBM/blob/706f2af7badc26f6ec68729469ec6ec79a66d802/.github/workflows/r_package.yml that installs with a command like Rscript build_r.R --use-gpu and then runs the R tests.

          - os: ubuntu-latest
            task: r-package-gpu
            compiler: gcc
            r_version: 4.0
            build_type: cmake

References

You might need to reuse of some of the GPU setup for the Python "gpu" tasks

@jameslamb
Copy link
Collaborator Author

Adding this to #2302 with other feature requests. Anyone is welcome to contribute this! Leave a comment if you'd like to contribute this and the issue can be re-opened.

@jaredlander
Copy link

While I can't contribute, I'd like to encourage further development (I know, I know, asking but not helping, but I think cheerleading is a form of helping, right?). While I know getting the GPU version onto CRAN would be hard, if there are prebuilt binaries the way {catboost} installs using devtools::install_url() that would be a step in the right direction.

On the other hand, if you're making progress toward getting the GPU version on the CRAN that will be awesome.

And yes @jameslamb, I'm putting together a course (not just a slideshow) and want this for the "more speed" part of the course.

@jameslamb
Copy link
Collaborator Author

Thanks for stopping by @jaredlander !

Unless another maintainer here or an outside contributor comes to help us with this, this issue is not going to be picked up for at least a month. Personally, my focus right now is on the new Dask integration (#3872 (comment)).

You're welcome to build the GPU version of the R package yourself, following https://github.com/microsoft/LightGBM/blob/master/R-package/README.md#installing-a-gpu-enabled-build

@jaredlander
Copy link

Those build instructions don't seem to be too bad compared to what I've from other GPU packages. I'm certainly game for trying but I don't know if I could expect all of my attendees to do that.

I bet this could be added to GitHub Actions (or maybe this one) pretty easily. I would try but I'm just getting started with Actions and don't know enough about them. So I am hoping another maintainer could add it.

@jaredlander
Copy link

Oh, btw, maybe someone from the team should give a talk at the meetup about this package.

@jameslamb
Copy link
Collaborator Author

jameslamb commented Jan 28, 2021

I bet this could be added to GitHub Actions (or maybe this one) pretty easily

I expect it will be harder than just running R CMD INSTALL --build, to generate binaries that we feel confident will work well on a wide range of GPUs. It took significant effort to do something similar in LightGBM's Python package (#3403, #3144), and even after that effort those wheels are only supported on Windows.

It's a valuable feature, but I just want to be sure we set good expectations about the amount of effort required to add it.

If I'm wrong and this is straightforward, we'd welcome a PR to add this support. So if you know some people who have experience with building R packages that take advantage of GPUs, we'd be grateful if you'd encourage them to come help out on this.

Oh, btw, maybe someone from the team should give a talk at the meetup about this package.

Thanks. I'd be happy to come speak if you'd like! You can email me at redacted if you want to talk about possible topics and scheduling.

@jaredlander

This comment was marked as resolved.

@jameslamb

This comment was marked as resolved.

@jameslamb
Copy link
Collaborator Author

Just adding for context... having this CI job would help with cases like this: #6599

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants