CI pipelines to support GraalVM

[srnagar]

We are looking into supporting GraalVM for Java client libraries. One of the things we'd like to do as part of supporting this is to build native images in Windows, macOS, and Linux and run samples to ensure the image configuration works across all platforms.

So, creating this issue to track the tasks involved in how this can be done in CI pipelines and address any concerns with doing this from the EngSys side.

Setting up GraalVM on Windows

Pre-requisites

• Install Visual Studio Code Build Tools with Windows 10 SDK
• Install Visual Studio Code with Windows 10 SDK

Install GraalVM JDK

• Navigate to the GraalVM Releases repository on GitHub. Select Java 11 based or Java 17 based distribution for Windows, and download.

• Change the directory to the location where you want to install GraalVM, then move the .zip archive file to it.

• Unzip the archive to your file system.

• There can be multiple JDKs installed on the machine. The next step is to configure the runtime environment. Setting environment variables via the command line will work the same way for Windows 7, 8 and 10.

• Point the PATH environment variable to the GraalVM bin directory:
  setx /M PATH "C:\Progra~1\Java<graalvm>\bin;%PATH%"

• Set the JAVA_HOME environment variable to resolve to the GraalVM installation directory:
  setx /M JAVA_HOME "C:\Progra~1\Java<graalvm>"

• Note that the /M flag, equivalent to -m, requires elevated user privileges.

• Restart Command Prompt to reload the environment variables. Then use the following command to check whether the variables were set correctly:

echo %PATH%
echo %JAVA_HOME%

Setting up GraalVM on Linux

• Navigate to the GraalVM Releases repository on GitHub. Select Java 11 based or Java 17 based distribution for the Linux AMD64 architecture, and download.
• Change the directory to the location where you want to install GraalVM, then move the .tar.gz archive to it.
  Unzip the archive:
  tar -xzf graalvm-ce-java-linux-amd64-.tar.gz
• There can be multiple JDKs installed on the machine. The next step is to configure the runtime environment:
  • Point the PATH environment variable to the GraalVM Enterprise bin directory:

  export PATH=/path/to/<graalvm>/bin:$PATH

  • Set the JAVA_HOME environment variable to resolve to the installation directory:

  export JAVA_HOME=/path/to/<graalvm>

• To check whether the installation was successful, run the java -version command.

Setting up GraalVM on MacOS

• Navigate to GraalVM Releases repository on GitHub. Select Java 11 based or Java 17 based distribution for macOS, and download.
• Unzip the archive.

tar -xzf graalvm-ce-java<version>-darwin-amd64-<version>.tar.gz

Alternatively, open the file in Finder.

Note: If you are using macOS Catalina and later you may need to remove the quarantine attribute. See Installation Notes below.

• Move the downloaded package to its proper location, the /Library/Java/JavaVirtualMachines directory. Since this is a system directory, sudo is required:

  sudo mv graalvm-ce-java<version>-<version> /Library/Java/JavaVirtualMachines

To verify if the move is successful and to get a list of all installed JDKs, run /usr/libexec/java_home -V.

• There can be multiple JDKs installed on the machine. The next step is to configure the runtime environment:
  • Point the PATH environment variable to the GraalVM bin directory:

export PATH=/Library/Java/JavaVirtualMachines/<graalvm>/Contents/Home/bin:$PATH

• Set the JAVA_HOME environment variable to resolve to the GraalVM installation directory:

export JAVA_HOME=/Library/Java/JavaVirtualMachines/<graalvm>/Contents/Home

• To check whether the installation was successful, run the java -version command.

Install Native Image (common for all platforms)

• On Windows the native-image tool only works when it is executed from the x64 Native Tools Command Prompt.
• Run this command

gu install native-image

NOTE: The maven build is memory intensive. For e.g. building an app that takes a dependency on Storage Blob requires about 6GB of memory.

Release:
Client libraries will have config files to build native images included in the jar as resource files. So, this should not require any additional setup to release the libraries.

[kurtzeborn]

Putting this on the EngSys backlog, but if this is needed sooner than several months from now, someone on the Java team will need to enable this with guidance from @benbp.

[benbp]

The azure pipelines ubuntu 20.04 images recently added Graal CE 21.1.0: https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu2004-README.md#graalvm

Assuming that's a workable version to use, then the infra side of things likely won't involve too much more than provisioning a new high mem pool.

Tracking issue for GraalVM on other OS images: actions/runner-images#2197

[benbp]

Related issue: #21192

[xlf12]

Are there any new developments here?

I have seen that there is GraalVM support for the old Java 11, but no support for the current Java 17.

Do you always have to fall back on installing the SDK yourself?

[xlf12]

Well, finally I have found this obvious solution ...

# Maven
# Build your Java project and run tests with Apache Maven.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/java

trigger:
  - master

pool:
  vmImage: ubuntu-22.04

variables:
  MAVEN_CACHE_FOLDER: $(Pipeline.Workspace)/.m2/repository
  MAVEN_OPTS: '-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)'
  GRAALVM_VERSION: '22.2.0'

steps:
  - task: Cache@2
    inputs:
      key: 'maven | "$(Agent.OS)" | **/pom.xml'
      restoreKeys: |
        maven | "$(Agent.OS)"
        maven
      path: $(MAVEN_CACHE_FOLDER)
    displayName: Cache Maven local repo

  - script: |
      echo "Install GraalVM"
      cd /tmp
      curl -o GraalVM.tar.gz -L https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-$(GRAALVM_VERSION)/graalvm-ce-java17-linux-amd64-$(GRAALVM_VERSION).tar.gz
      tar -xzf GraalVM.tar.gz
      export PATH=/tmp/graalvm-ce-java17-$(GRAALVM_VERSION)/bin:$PATH
      export JAVA_HOME=/tmp/graalvm-ce-java17-$(GRAALVM_VERSION)
      java -version

  - task: Maven@3
    inputs:
      mavenPomFile: 'pom.xml'
      mavenOptions: '-Xmx3072m $(MAVEN_OPTS)'
      javaHomeOption: '/tmp/graalvm-ce-java17-$(GRAALVM_VERSION)'
      jdkDirectory: '/tmp/graalvm-ce-java17-$(GRAALVM_VERSION)'
      # jdkVersionOption: '17'
      # jdkArchitectureOption: 'x64'
      publishJUnitResults: true
      testResultsFiles: '**/surefire-reports/TEST-*.xml'
      goals: 'spring-boot:build-image -Pnative'

The installation (the script part) runs in 12 seconds, which is not a problem.

I wonder if this is the best approach to do such things. Well, as long as we don't have Docker images to build a better build runtime stack, this could be a hypothetical question...

[srnagar]

CI pipelines are enabled for GraalVM native image creation and testing - https://github.com/Azure/azure-sdk-for-java/blob/main/eng/pipelines/templates/stages/archetype-sdk-native-tests.yml