Extension of the TornadoExecutionPlan API to allow backend and device filters from the API #357

jjfumero · 2024-03-20T11:12:40Z

Description

This PR adds an extension to the TornadoExecutionPlan API in TornadoVM to allow developers to query backends and devices by applying filters. This API is designed as a consequence of the inconsistencies when obtaining a device using different APIs (more details in the next Section).

This API fixes this inconsistencies by giving the user a new Data Structure, called TornadoDeviceMap in which developers can query all backends, devices and apply filters.

Examples of use:

// Obtain an instance of the TornadoDeviceMap via the execution plan
TornadoDeviceMap tornadoDeviceMap = TornadoExecutionPlan.getTornadoDeviceMap();

// Obtain the number of backends installed
int numBackends = tornadoDeviceMap.getNumBackends();


// Query all backends installed
List<TornadoBackend> backends = tornadoDeviceMap.getAllBackends();

// Obtain the number of devices for the first backend
int numDevicesBackendZero = backends.getFirst().getDeviceCount();

We can apply more complex queries as follows:

TornadoDeviceMap tornadoDeviceMap = TornadoExecutionPlan.getTornadoDeviceMap();

// Obtain the OpenCL Backend
List<TornadoBackend> openCLBackend = tornadoDeviceMap.getBackendsWithPredicate(backend -> backend.getBackendType() == TornadoVMBackendType.OPENCL);

// Obtain Backends with at least, two devices
List<TornadoBackend> multiDeviceBackends = tornadoDeviceMap.getBackendsWithPredicate(backend -> backend.getDeviceCount() > 1);


// Obtain All backends that can access to an NVIDIA GPU
List<TornadoBackend> backendsWithNVIDIAAccess = tornadoDeviceMap.getBackendsWithDevicePredicate(device -> device.getDeviceName().toLowerCase().contains("nvidia"));

As part of this API, we needed to refactor some internal classes. The concept of Driver is not the same as we have in TornadoVM compared to the OS driver. Thus, all old driver classes became the Backend classes.

Problem description

The main problem is that to query the number of devices, developers need to access through the TornadoRuntime.getRuntime class. However, to select a device or a backend, developers need to the ExecutionPlan.

This PR fixes this inconsistencies by providing a more concise and expressive API to filter devices and backends without the need to instantiate or interact with the TornadoVM Core Runtime (internal class).

Backend/s tested

Mark the backends affected by this PR.

OpenCL
PTX
SPIRV

OS tested

Mark the OS where this PR is tested.

Linux
OSx
Windows

Did you check on FPGAs?

If it is applicable, check your changes on FPGAs.

Yes
No

How to test the new patch?

$ make 
$ tornado-test  -V uk.ac.manchester.tornado.unittests.api.TestDevices 

$ make BACKEND=spirv
$ tornado-test -V uk.ac.manchester.tornado.unittests.api.TestDevices 

$ make BACKEND-ptx
$ tornado-test -V uk.ac.manchester.tornado.unittests.api.TestDevices

… the execution plan

…consistency

mairooni

LGTM, just have a look at the files that were changed to make sure that the date in the license header is updated to 2024. E.g. in the TornadoBackend.java, TornadoExecutionPlan.java etc.

stratika

LGTM, some minor comments:

Regarding the update of the year in the copyright headers for the files that have been altered.
Java doc description for the new methods. Especially to clarify the use of predicate for the methods that use the TornadoBackend and the TornadoDevice as predicates.

stratika · 2024-03-20T14:53:01Z

tornado-api/src/main/java/uk/ac/manchester/tornado/api/TornadoBackend.java

@@ -17,11 +17,14 @@
 */
 package uk.ac.manchester.tornado.api;

+import java.util.List;


Update the year in the copyright header

stratika · 2024-03-20T14:53:55Z

tornado-api/src/main/java/uk/ac/manchester/tornado/api/TornadoBackend.java

@@ -31,6 +34,10 @@ public interface TornadoDriver {

    TornadoDevice getDevice(int index);

+    List<TornadoDevice> getAllDevices();
+
+    List<TornadoDevice> getDevicesWithPredicate(Predicate<? super TornadoDevice> predicate);


Add a java doc for this method. What is the context of the predicate in this case?

I removed this method. We don't need it in this interface.

tornado-api/src/main/java/uk/ac/manchester/tornado/api/TornadoDeviceMap.java

stratika · 2024-03-20T14:56:30Z

tornado-api/src/main/java/uk/ac/manchester/tornado/api/TornadoDeviceMap.java

+    private final List<TornadoBackend> backends;
+
+    public TornadoDeviceMap() {
+        numBackends = coreRuntime.getNumDrivers();


Shall we refactor the name of that method to coreRuntime.getNumBackends? Or does it break consistency within the runtime class? If it breaks consistency we can do it in a later PR.

good point. I think we can apply this refactor now.

tornado-api/src/main/java/uk/ac/manchester/tornado/api/TornadoDeviceMap.java

stratika · 2024-03-20T15:15:52Z

tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/common/TornadoTestBase.java

@@ -20,7 +20,7 @@

 import org.junit.Before;

-import uk.ac.manchester.tornado.api.TornadoDriver;
+import uk.ac.manchester.tornado.api.TornadoBackend;
