Skip to content

Commit

Permalink
Added documentation of memory types (#770)
Browse files Browse the repository at this point in the history
  • Loading branch information
cliffburdick authored Oct 17, 2024
1 parent 4e602cb commit c1ed734
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 6 deletions.
19 changes: 19 additions & 0 deletions docs_input/basics/memory.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
.. _memory:

Memory
======

MatX allows tensors to be allocated in several different spaces either corresponding to physical or logical
allocations. The space where the memory is allocated dictates where a user is allowed to access the memory.
For example, allocating device memory on some system is not accessible by the CPU and will result in a SEGFAULT
when trying to access.

The type of allocation may behave differently across systems. For example, on a Grace-Hopper system (GH200)
standard host memory from malloc is accessible from the GPU, but may not be on other platforms. Some types
may not be available in certain environments. On WSL2 CUDA Unified Memory (UM) or managed memory is not fully
supported and may result in slow code or other issues.

The memory type is typically chosen when creating a tensor with `make_tensor`. The memory *may* be allocated
immediately, but it is not guaranteed. The memory is guaranteed to be available before it used used, however.

.. doxygenenum:: matxMemorySpace_t
12 changes: 6 additions & 6 deletions include/matx/core/allocator.h
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,12 @@ namespace matx {
*
*/
enum matxMemorySpace_t {
MATX_MANAGED_MEMORY,
MATX_HOST_MEMORY,
MATX_HOST_MALLOC_MEMORY,
MATX_DEVICE_MEMORY,
MATX_ASYNC_DEVICE_MEMORY,
MATX_INVALID_MEMORY
MATX_MANAGED_MEMORY, ///< CUDA managed memory or CUDA Unified Memory (UM) from cudaMallocManaged
MATX_HOST_MEMORY, ///< CUDA host-pinned memory from cudaHostAlloc
MATX_HOST_MALLOC_MEMORY, ///< Host-alloced memory (pageable) from malloc
MATX_DEVICE_MEMORY, ///< CUDA device memory from cudaMalloc
MATX_ASYNC_DEVICE_MEMORY, ///< CUDA asynchronous device memory corresponding to a stream from cudaMallocAsync
MATX_INVALID_MEMORY ///< Sentinel value
};

namespace detail {
Expand Down

0 comments on commit c1ed734

Please sign in to comment.