Enable Under Demand Data Transfers from Device to Host

[jjfumero]

Description

This PR enables under-demand sub-copies from the device to the host.
This is useful for iterative algorithms in which we do not have to copy all data back and forth between
iterations. Rather. we can keep the data on the device, and perform the transfer under demand.

This functionality was already available since version 0.15 of TornadoVM. This PR extends this functionality by allowing developers to copy sub-regions under demand.

There is a new method in the TornadoExecutionResult to perform partial copies.
The following code snippet shows an example:

// Build a Task Graph and declare the copy out (transfer to host) as a UNDER_DEMAND copy
taskGraph.transferToDevice(DataTransferMode.FIRST_EXECUTION, data) //
                .task("t0", TestArrays::addAccumulator, data, 1) //
                .transferToHost(DataTransferMode.UNDER_DEMAND, data);

// Create an immutable task graph
ImmutableTaskGraph immutableTaskGraph = taskGraph.snapshot();

// Create an execution plan
TornadoExecutionPlan executionPlan = new TornadoExecutionPlan(immutableTaskGraph);

// Run the execution plan and obtain the execution result.
TornadoExecutionResult executionResult = executionPlan.execute();

// From the Execution Result, we can build a DataRange and specify the range to copy out
DataRange dataRange = new DataRange(data);


// Copy under demand from offset 0 (0 is the default is not specified) until size = N/2.
executionResult.transferToHost(dataRange.withSize(N / 2));

// Perform a second copy from offset = N/2 with size = N/2. 
executionResult.transferToHost(dataRange.withOffset(N / 2).withSize(N / 2));

Using the data range, developers specify the offset and the size to copy.
Both of these parameters are optional (withOffset(x) and withSize(y)).

• If the offset is not specified, the TornadoVM Runtime selects offset 0
• If the size if not specified, the TornadoVM runtime copies from the selected (or default) offset until the end of the array.

Backend/s tested

Mark the backends affected by this PR.

• OpenCL
• PTX
• SPIRV

OS tested

Mark the OS where this PR is tested.

• Linux
• OSx
• Windows

Did you check on FPGAs?

If it is applicable, check your changes on FPGAs.

• Yes
• No

How to test the new patch?

$ make
$ tornado-test -V uk.ac.manchester.tornado.unittests.api.TestAPI#testLazyPartialCopyOut

[mairooni]

As far as I understand, the UNDER_DEMAND mode can be used to perform a lazy D2H copy of the whole dataset or a subset of the dataset if a DataRange object is included in the transferToHost, right? Does the partial copy need to be lazy? If not, do you think - not right now, but at a later point - that it would make sense to decouple the two? To have the UNDER_DEMAND mode just to define lazy copies and to have a let's say a PARTIAL mode or something similar to specify partial copies?

[jjfumero]

As far as I understand, the UNDER_DEMAND mode can be used to perform a lazy D2H copy of the whole dataset or a subset of the dataset if a DataRange object is included in the transferToHost, right?

Yes, this is correct.

Does the partial copy need to be lazy? If not, do you think - not right now, but at a later point - that it would make sense to decouple the two? To have the UNDER_DEMAND mode just to define lazy copies and to have a let's say a PARTIAL mode or something similar to specify partial copies?

This is an interesting point. In my view, if the user needs a partial copy, the way to do it is via DataRanges using the under demand mode. So, by default, it operates using all copy in/out. Having said this, I think is a nice to have if we can also specify custom DataRages for copy-in in a task-graph, and quite likely, this needs to be done at the ExecutionPlan level, rather than the ExecutionResult.

[mikepapadim]

LGTM, thanks

[stratika]

LGTM