Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore building/running TornadoVM from IntelliJ & update of documentation #582

Merged
merged 12 commits into from
Oct 23, 2024

Conversation

stratika
Copy link
Collaborator

@stratika stratika commented Oct 22, 2024

Description

This PR (is an update of #445) and provides the following:

  1. a fix in the tornadovm-installer script to install cmake and maven in the path that is compliant with the IDE xml configurations.
  2. a fix in the post-installation script for building TornadoVM from IntelliJ in Windows OS. This is to trigger the pyInstaller for building the tornado executables.
  3. update the documentation for the TornadoVM read-the-docs.

The idea is that now TornadoVM users can:

  1. build TornadoVM (with one or more backends) from IntelliJ in all OSs (Linux, macOS, Windows).
    A user can use the Python application (TornadoVM-Build) that is in the repository, and update the Python interpreter and all configurations as described in the instructions (ide-integration.rst).
Screenshot 2024-10-22 at 11 43 33
  1. run/debug TornadoVM tests from IntelliJ
    A user can use the Python application (TornadoVM-Build) to run the tornado-test script. The default run invokes with the "--quickPass" argument, but it is configurable from IDE.
Screenshot 2024-10-22 at 11 50 09
  1. run/debug Java applications that use TornadoVM from IntelliJ
Screenshot 2024-10-22 at 11 53 05

Problem description

The instructions of building/running with IntelliJ in the TornadoVM documentation page are old and not applicable.

Backend/s tested

Mark the backends affected by this PR.

  • OpenCL
  • PTX
  • SPIRV

No backends are affected. This PR does not touch the compiler.

OS tested

Mark the OS where this PR is tested.

  • Linux
  • OSx
  • Windows

Did you check on FPGAs?

If it is applicable, check your changes on FPGAs.

  • Yes
  • No

How to test the new patch?

Instructions are provided in the instructions (ide-integration.rst)


@stratika stratika self-assigned this Oct 22, 2024
@stratika stratika added enhancement New feature or request documentation Documentation labels Oct 22, 2024
@stratika stratika requested a review from mairooni October 22, 2024 10:59
@jjfumero
Copy link
Member

jjfumero commented Oct 22, 2024

When I build, I see the following diff in the repo.

diff --git a/.build/TornadoVM-Tests.run.xml b/.build/TornadoVM-Tests.run.xml
index a755f7263..8f5317e98 100644
--- a/.build/TornadoVM-Tests.run.xml
+++ b/.build/TornadoVM-Tests.run.xml
@@ -5,14 +5,14 @@
     <option name="INTERPRETER_OPTIONS" value="" />
     <option name="PARENT_ENVS" value="true" />
     <envs>
-      <env name="BACKEND" value="opencl" />
       <env name="PYTHONUNBUFFERED" value="1" />
+      <env name="BACKEND" value="opencl" />
       <env name="selected_backends" value="opencl-backend" />
       <env name="TORNADO_SDK" value="$PROJECT_DIR$/bin/sdk" />
       <env name="JAVA_HOME" value="$PROJECT_DIR$/etc/dependencies/TornadoVM-graal-jdk-21/graalvm-community-openjdk-21.0.1+12.1" />
     </envs>
     <option name="SDK_HOME" value="" />
-    <option name="SDK_NAME" value="Python 3.10" />
+    <option name="SDK_NAME" value="Python 3.9 (tornado)" />
     <option name="WORKING_DIRECTORY" value="$PROJECT_DIR$" />
     <option name="IS_MODULE_SDK" value="false" />
     <option name="ADD_CONTENT_ROOTS" value="true" />
@@ -27,4 +27,4 @@
     <option name="INPUT_FILE" value="" />
     <method v="2" />
   </configuration>
-</component>
+</component>
\ No newline at end of file
diff --git a/.build/_internal_TornadoVM_Maven-cleanAndinstall.run.xml b/.build/_internal_TornadoVM_Maven-cleanAndinstall.run.xml
index 908ce38c8..dcf22bcb0 100644
--- a/.build/_internal_TornadoVM_Maven-cleanAndinstall.run.xml
+++ b/.build/_internal_TornadoVM_Maven-cleanAndinstall.run.xml
@@ -5,17 +5,18 @@
         <MavenGeneralSettings>
           <option name="alwaysUpdateSnapshots" value="false" />
           <option name="checksumPolicy" value="NOT_SET" />
+          <option name="customMavenHome" value="$PROJECT_DIR$/etc/dependencies/TornadoVM-jdk21/apache-maven-3.9.3" />
           <option name="emulateTerminal" value="true" />
           <option name="failureBehavior" value="NOT_SET" />
           <option name="localRepository" value="" />
-          <option name="mavenHome" value="$PROJECT_DIR$/etc/dependencies/apache-maven-3.9.3" />
+          <option name="mavenHome" value="$PROJECT_DIR$/etc/dependencies/TornadoVM-jdk21/apache-maven-3.9.3" />
+          <option name="mavenHomeTypeForPersistence" value="CUSTOM" />
           <option name="nonRecursive" value="false" />
           <option name="outputLevel" value="INFO" />
           <option name="printErrorStackTraces" value="false" />
           <option name="showDialogWithAdvancedSettings" value="false" />
           <option name="threads" />
           <option name="useMavenConfig" value="true" />
-          <option name="usePluginRegistry" value="false" />
           <option name="userSettingsFile" value="" />
           <option name="workOffline" value="false" />
         </MavenGeneralSettings>
@@ -33,10 +34,11 @@
               <option value="install" />
             </list>
           </option>
+          <option name="multimoduleDir" />
           <option name="pomFileName" />
           <option name="profilesMap">
             <map>
-              <entry key="graal-jdk-21" value="true" />
+              <entry key="jdk21" value="true" />
               <entry key="opencl-backend" value="true" />
             </map>
           </option>

Shall we add an external repo that pulls/builds these config files under demand?

@@ -1,5 +1,5 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="TornadoVM-Full-OpenCL-Build" type="PythonConfigurationType"
<configuration default="false" name="TornadoVM-Build" type="PythonConfigurationType"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build is till specific to OpenCL, shall we keep the name?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but, in the instructions I have added the text how to extend for SPIRV and PTX. so the same application is working, it is not OpenCL-specific. the default configuration is OpenCL because it is compatible to most of the target hardware devices.

<env name="PYTHONUNBUFFERED" value="1" />
<env name="selected_backends" value="opencl-backend" />
<env name="TORNADO_SDK" value="$PROJECT_DIR$/bin/sdk" />
<env name="JAVA_HOME" value="$PROJECT_DIR$/etc/dependencies/TornadoVM-graal-jdk-21/graalvm-community-openjdk-21.0.1+12.1" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we don't use the same JDK?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be working, but user must be careful to define the same JAVA_HOME when TornadoVM is built and run.

@jjfumero
Copy link
Member

It works with a bit of work in custom configuration. Keeping in mind that The JAVA_HOME selected during the build must be the same as the one selected for the launch.

I suggest to move forward with this PR, and have an extension of the PR to generate the xml config files as part of a new option with the TornadoVM script. Something like tornado --ideinit. I recall Graal used to have a similar approach.

@stratika
Copy link
Collaborator Author

It works with a bit of work in custom configuration. Keeping in mind that The JAVA_HOME selected during the build must be the same as the one selected for the launch.

I suggest to move forward with this PR, and have an extension of the PR to generate the xml config files as part of a new option with the TornadoVM script. Something like tornado --ideinit. I recall Graal used to have a similar approach.

yes, makes sense to me. I am not also a fun of having user-specific configurations that are being tracked by git vcs.

Copy link
Collaborator

@mairooni mairooni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@stratika
Copy link
Collaborator Author

thank you, I fixed an inconsistent name of file in the docs and will merge it. I have prepared a follow-up PR to implement the generation of the xml files for IDE so that they will not be in the repository.

@stratika stratika merged commit eafcbdf into beehive-lab:develop Oct 23, 2024
2 checks passed
@stratika stratika deleted the fix/idea-settings branch October 29, 2024 10:41
jjfumero added a commit to jjfumero/TornadoVM that referenced this pull request Dec 20, 2024
Improvements
=============

- beehive-lab#573: Enhanced output of unit-tests with a summary  of pass-rates and fail-rates.
- beehive-lab#576: Extended support for 3D matrices.
- beehive-lab#580: Extended debug information for execution plans.
- beehive-lab#584: Added helper menu for the ``tornado`` launcher script when no arguments are passed.
- beehive-lab#589: Enable partial loop unrolling for all backends.
- beehive-lab#594: Added RISC-V 64 CPU port support to run OpenCL with vector instructions RVV 1.0 (using the Codeplay OCK Toolkit).
- beehive-lab#598: OpenCL low-level buffers tagged as read, write and read/write based on the data dependency analysis.
- beehive-lab#601: Feature to select an immutable task graph to execute from a multi-task graph execution plan.

Compatibility
=============

- beehive-lab#570:  Extended timeout for all suite of unit-tests.
- beehive-lab#579: Removed legacy JDK 8 and JDK11 build options from the TornadoVM installer.
- beehive-lab#582: Restored tornado runner scripts for IntellIJ.
- beehive-lab#583: Automatic generation of IDE IntelliJ configuration runner files from the TornadoVM command.
- beehive-lab#597: Updated white-list of unit-test and checkstyle improved.

Bug Fixes
=============

- beehive-lab#571: Fix issues with bracket closing for if/loops conditions.
- beehive-lab#572: Fix for printing default execution plans (execution plans with default parameters).
- beehive-lab#575: Fix the Level Zero version used for building the SPIR-V backend.
- beehive-lab#577: Fix checkstyle.
- beehive-lab#587: Fix thread scheduler for new NVIDIA Drivers.
- beehive-lab#592: Fix ``Float.POSITIVE_INFINITY`` and ``Float.NEGATIVE_INFINITIVE`` constants for the OpenCL, CUDA and SPIR-V backends.
- beehive-lab#596: Fix extra closing bracket during the code-generation for the FPGAs.
- Remove the intermediate CUDA pinned memory regions in the JNI code: [link](beehive-lab@9c3f8ce)
- Fix bitwise negation operations for the PTX backend:  [link](beehive-lab@0db1cd3)
- `GetBackendImpl::getAllDevices` thread-safe: [link](beehive-lab@0d44252)
- Check size elements for memory segments: [link](beehive-lab@4360385)
@jjfumero jjfumero mentioned this pull request Dec 20, 2024
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Documentation enhancement New feature or request
Projects
Development

Successfully merging this pull request may close these issues.

3 participants