Add support to TornadoNativeArray data types to get Memory Segments without header offset #350

mikepapadim · 2024-03-11T11:21:06Z

Description

This PR adds support to obtain a MemorySegment from TornadoNativeArray types with and without the header bytes.

MemorySegment getSegment() returns a memory segment excluding the Tornado header.
MemorySegment getSegmentWithHeader() returns a memory segment as allocated for TornadoVM including the Tornado header (24bytes).
long getNumBytesOfSegmentWithHeader() returns Segment's number of bytes including the Tornado header.
long getNumBytesOfSegment() returns Segment's number of bytes excluding the Tornado header.

By having distinct funtion in the above auxiliry methods we can have interoperoability when using Segments for GPU execution through the Taskgraph API and Vector API when using any of the TornadoNative types.

Also, refactor internally methods that operate on memory segments to be named explictly as getSegmentWithHeader.

Backend/s tested

Mark the backends affected by this PR.

OpenCL
PTX
SPIRV

OS tested

Mark the OS where this PR is tested.

Linux
OSx
Windows

Did you check on FPGAs?

If it is applicable, check your changes on FPGAs.

Yes
No

How to test the new patch?

make jdk21  BACKEND=openc
make tests

Implemented a new method called getSegmentWithHeader in various array classes in tornado-api package. This method returns the underlying MemorySegment of the array instance, including the header offset. As a result, users can now access the whole segment which includes both the header and data, rather than data only.

The code has been updated to use "getSegmentWithHeader()" method instead of "getSegment()" across various OpenCL, SPIRV and PTX driver files. This adjustment ensures the appropriate header attached memory segment is used with operations like buffer reading, writing, allocation, etc. Reduces potential for errors in memory management and data handling.

…apadim/segment_slice

Revise SPIRVVectorWrapper to handle memory segments in a safer manner. Replaced usages of "getSegment()" with "getSegmentWithHeader()" for buffer actions such as enqueueReadBuffer, enqueueWriteBuffer, readBuffer, and writeBuffer. The change helps mitigate potential concurrency issues and ensures better safety in multithreaded environments.

Renamed methods related to array size calculations in TornadoNativeArray. The method getNumBytesOfSegment() is now getNumBytesOfSegmentWithHeader() and the method getNumBytesWithoutHeader() is now getNumBytesOfSegment(). All implementations and references to these methods have been updated accordingly across various classes.

The documentation has been updated to reflect changes in method names related to the off-heap array size calculations in TornadoNative

The error message for TornadoMemoryException in SPIRVVectorWrapper has been updated to dynamically display the bufferSize. This minor modification can improve debugging by indicating the exact size that caused the exception.

stratika

I noticed that in many of the classes, the year in the copyright header has not been updated to 2024.

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/arrays/TornadoNativeArray.java

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/arrays/ByteArray.java

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/arrays/CharArray.java

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/arrays/DoubleArray.java

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/arrays/ShortArray.java

stratika · 2024-03-13T08:41:25Z

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/images/TornadoImagesInterface.java

@@ -31,4 +31,6 @@ public sealed interface TornadoImagesInterface<T extends Buffer> //

    MemorySegment getSegment();

+    MemorySegment getSegmentWithHeader();


The getNumBytes returns always the number of bytes with the header, right?

If someone will use the getNumBytes() for the segment returned via getSegment, will it work?

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/matrix/Matrix2DInt.java

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/matrix/TornadoMatrixInterface.java

...do-api/src/main/java/uk/ac/manchester/tornado/api/types/volumes/TornadoVolumesInterface.java

…rrays/TornadoNativeArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…rrays/ByteArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…rrays/CharArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…rrays/DoubleArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…rrays/FloatArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…rrays/HalfFloatArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…rrays/ShortArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…rrays/IntArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…rrays/LongArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

…rnadoVM into mikepapadim/segment_slice

jjfumero · 2024-03-18T11:16:49Z

I am confused. The withHeader version, does it always return +24 bytes, or it returns the bytes specified in TornadoNativeArray.ARRAY_HEADER?

mikepapadim · 2024-03-18T11:27:28Z

I am confused. The withHeader version, does it always return +24 bytes, or it returns the bytes specified in TornadoNativeArray.ARRAY_HEADER?

withHeader returns always number of bytes including the TornadoNativeArray.ARRAY_HEADER

jjfumero · 2024-03-18T11:30:52Z

I am confused. The withHeader version, does it always return +24 bytes, or it returns the bytes specified in TornadoNativeArray.ARRAY_HEADER?

withHeader returns always number of bytes including the TornadoNativeArray.ARRAY_HEADER

So please, update the PR description with this.

mikepapadim · 2024-03-18T11:37:35Z

I am confused. The withHeader version, does it always return +24 bytes, or it returns the bytes specified in TornadoNativeArray.ARRAY_HEADER?

withHeader returns always number of bytes including the TornadoNativeArray.ARRAY_HEADER

So please, update the PR description with this.

done

docs/source/offheap-types.rst

jjfumero · 2024-03-18T11:32:12Z

tornado-api/src/main/java/uk/ac/manchester/tornado/api/DataRange.java

@@ -29,23 +29,18 @@
 public class DataRange {

    private final TornadoNativeArray tornadoNativeArray;
+    private final long totalSizeInBytes;
+    private final int elementSize;


Out of curiosity, if we all use the same IntelliJ templates for code formatting, why is gets shuffled every time?

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/arrays/ByteArray.java

tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/arrays/TornadoNativeArray.java

jjfumero · 2024-03-18T11:47:08Z

Unittests are passing on OSx 14.4

…rrays/TornadoNativeArray.java Co-authored-by: Juan Fumero <jjfumero@gmail.com>

mikepapadim · 2024-03-18T11:56:43Z

@jjfumero done

jjfumero · 2024-03-18T11:57:47Z

ok, thanks. the PR looks good to me.

Improvements ~~~~~~~~~~~~~~~~~~ - `beehive-lab#344 <https://github.com/beehive-lab/TornadoVM/pull/344>`_: Support for Multi-threaded Execution Plans. - `beehive-lab#347 <https://github.com/beehive-lab/TornadoVM/pull/347>`_: Enhanced examples. - `beehive-lab#350 <https://github.com/beehive-lab/TornadoVM/pull/350>`_: Obtain internal memory segment for the Tornado Native Arrays without the object header. - `beehive-lab#357 <https://github.com/beehive-lab/TornadoVM/pull/357>`_: API extensions to query and apply filters to backends and devices from the ``TornadoExecutionPlan``. - `beehive-lab#359 <https://github.com/beehive-lab/TornadoVM/pull/359>`_: Support Factory Methods for FFI-based array collections to be used/composed in TornadoVM Task-Graphs. Compatibility ~~~~~~~~~~~~~~~~~~ - `beehive-lab#351 <https://github.com/beehive-lab/TornadoVM/pull/351>`_: Compatibility of TornadoVM Native Arrays with the Java Vector API. - `beehive-lab#352 <https://github.com/beehive-lab/TornadoVM/pull/352>`_: Refactor memory limit to take into account primitive types and wrappers. - `beehive-lab#354 <https://github.com/beehive-lab/TornadoVM/pull/354>`_: Add DFT-sample benchmark in FP32. - `beehive-lab#356 <https://github.com/beehive-lab/TornadoVM/pull/356>`_: Initial support for Windows 11 using Visual Studio Development tools. - `beehive-lab#361 <https://github.com/beehive-lab/TornadoVM/pull/361>`_: Compatibility with the SPIR-V toolkit v0.0.4. - `beehive-lab#366 <https://github.com/beehive-lab/TornadoVM/pull/363>`_: Level Zero JNI Dependency updated to 0.1.3. Bug Fixes ~~~~~~~~~~~~~~~~~~ - `beehive-lab#346 <https://github.com/beehive-lab/TornadoVM/pull/346>`_: Computation of local-work group sizes for the Level Zero/SPIR-V backend fixed. - `beehive-lab#360 <https://github.com/beehive-lab/TornadoVM/pull/358>`_: Fix native tests to check the JIT compiler for each backend. - `beehive-lab#355 <https://github.com/beehive-lab/TornadoVM/pull/355>`_: Fix custom exceptions when a driver/device is not found.

mikepapadim added 2 commits March 11, 2024 13:02

mikepapadim added the API label Mar 11, 2024

mikepapadim requested review from jjfumero, mairooni and stratika March 11, 2024 11:21

mikepapadim self-assigned this Mar 11, 2024

mikepapadim added 5 commits March 11, 2024 14:02

Merge branch 'develop' of github.com:beehive-lab/TornadoVM into mikep…

9118b81

…apadim/segment_slice

Update method names in offheap-types documentation

ca9dd1d

The documentation has been updated to reflect changes in method names related to the off-heap array size calculations in TornadoNative

Refactor error message in SPIRVVectorWrapper

73816b3

The error message for TornadoMemoryException in SPIRVVectorWrapper has been updated to dynamically display the bufferSize. This minor modification can improve debugging by indicating the exact size that caused the exception.

mikepapadim mentioned this pull request Mar 11, 2024

Test VectorAPI with TornadoNative types through Memory Segments for 64 to 512Bit vector #351

Merged

6 tasks

stratika reviewed Mar 13, 2024

View reviewed changes

mikepapadim and others added 10 commits March 13, 2024 12:16

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

13cbe68

…rrays/TornadoNativeArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

9599a73

…rrays/TornadoNativeArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

f2b055f

…rrays/ByteArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

f488d3e

…rrays/CharArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

70dcecb

…rrays/DoubleArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

6f78970

…rrays/FloatArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

151d626

…rrays/HalfFloatArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

879edfd

…rrays/ShortArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

30d495d

…rrays/IntArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

9fa307b

…rrays/LongArray.java Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>

mikepapadim changed the title ~~Add support to TornadoNativeArray data types to get Memory Segments without base offset~~ Add support to TornadoNativeArray data types to get Memory Segments without header offset Mar 13, 2024

mikepapadim added 5 commits March 17, 2024 15:35

Update copyright year in array files

d61964a

Update copyright years in various files

4c960ee

Merge branch 'mikepapadim/segment_slice' of github.com:mikepapadim/To…

7e50d14

…rnadoVM into mikepapadim/segment_slice

Add getNumBytesWithHeader method to various interfaces and classes

5d72bc9

jjfumero requested changes Mar 18, 2024

View reviewed changes

mikepapadim and others added 2 commits March 18, 2024 13:49

Update tornado-api/src/main/java/uk/ac/manchester/tornado/api/types/a…

b74d120

…rrays/TornadoNativeArray.java Co-authored-by: Juan Fumero <jjfumero@gmail.com>

Refactor API documentation to use '{@link ClassName}' references

44f27d3

jjfumero approved these changes Mar 18, 2024

View reviewed changes

mairooni approved these changes Mar 19, 2024

View reviewed changes

mikepapadim merged commit 26691bb into beehive-lab:develop Mar 19, 2024
2 checks passed

mikepapadim deleted the mikepapadim/segment_slice branch March 19, 2024 08:57

jjfumero mentioned this pull request Mar 27, 2024

[release] TornadoVM 1.0.3 #365

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support to TornadoNativeArray data types to get Memory Segments without header offset #350

Add support to TornadoNativeArray data types to get Memory Segments without header offset #350

mikepapadim commented Mar 11, 2024 •

edited

Loading

stratika left a comment

stratika Mar 13, 2024

stratika Mar 13, 2024

jjfumero commented Mar 18, 2024

mikepapadim commented Mar 18, 2024

jjfumero commented Mar 18, 2024

mikepapadim commented Mar 18, 2024

jjfumero Mar 18, 2024

jjfumero commented Mar 18, 2024

mikepapadim commented Mar 18, 2024

jjfumero commented Mar 18, 2024

		@@ -31,4 +31,6 @@ public sealed interface TornadoImagesInterface<T extends Buffer> //

		MemorySegment getSegment();

		MemorySegment getSegmentWithHeader();

Add support to TornadoNativeArray data types to get Memory Segments without header offset #350

Add support to TornadoNativeArray data types to get Memory Segments without header offset #350

Conversation

mikepapadim commented Mar 11, 2024 • edited Loading

Description

Backend/s tested

OS tested

Did you check on FPGAs?

How to test the new patch?

stratika left a comment

Choose a reason for hiding this comment

stratika Mar 13, 2024

Choose a reason for hiding this comment

stratika Mar 13, 2024

Choose a reason for hiding this comment

jjfumero commented Mar 18, 2024

mikepapadim commented Mar 18, 2024

jjfumero commented Mar 18, 2024

mikepapadim commented Mar 18, 2024

jjfumero Mar 18, 2024

Choose a reason for hiding this comment

jjfumero commented Mar 18, 2024

mikepapadim commented Mar 18, 2024

jjfumero commented Mar 18, 2024

mikepapadim commented Mar 11, 2024 •

edited

Loading