Skip to content

Custom R Edition

Pre-release
Pre-release
Compare
Choose a tag to compare
@djw8605 djw8605 released this 30 Aug 22:45
· 32 commits to master since this release
v0.9.8
938b851

Summary

In this release, easily the most complicated changes since the beginning of the modifications, we have made some much requested additions:

  • #3 - It possible for users to specify custom packages to be installed on the remote machine.
  • #4 - A user may specify a custom R URL to download the R binary from.
  • #7 - The R bootstrap can now download a newer version of the R binaries if they are available.
  • #6 - When running a batched job, the outputs in the list will be updated asynchronously as they complete.

Updated Default Download

Since more people are using GridR, we have updated the default download URL to one that can handle more traffic.

Custom Packages

When the user initializes GridR, they are now able to specify packages that will be installed on the remote machine.

> grid.init(service="bosco.direct", localTmpDir="tmp", 
   remotePackages=c("<path to package.tar.gz>", "<package2>", "<package3>"))

The packages listed in the remotePackages argument will be sent with the jobs and installed on the remote cluster. They will automatically be available in the R environment when your processing begins.

The packages must be in source form. They will be installed with the command:

> R CMD INSTALL --build <package>

Custom R Installation

In addition to installing custom packages, users may also want to have a custom version of R itself. In this case, the user can specify the HTTP URL of another R binary tar ball. It will be downloaded on the worker node and the user's processing will be executed using this custom R installation.

User will give the argument Rurl to grid.init. The Rurl will then be used to download and start R on the worker node.

For example, the user would do:

> library("GridR")
> grid.init(service="bosco.direct", localTmpDir="tmp", Rurl="http://asdf/R-new.tar.gz")
> grid.apply("x", a)

Creating the R tarball

The R tar ball needs to be created to include the user's custom packages. Additionally, the R executable needs to be made portable. Documentation can be found on the Wiki. You may view this blog post. A working example can be seen from Dropbox.

Installing custom packages

Installing custom packages can be done by:

  1. Download or create a functional, portable, R tar ball.
  2. Install the custom package(s).
  3. Tar back the package.
Assets 4
Loading