YAKL Multi Dimensional Array Classes

Multi-Dimensional Array Accesses

YAKL multi-dimensional Arrays (like most C++ MD Array classes) store the data itself as a flat 1-D array and then do integer arithmetic when you access the data as a multi-dimensional array. For instance, if you had an array of type T with dimensions nz,ny,nx, a YAKL Array object will allocate an internal 1-D array of size nz*ny*nx. Then, when you access the data as arr(k,j,i), the YAKL Array object will then give you the data as return myData[k*ny*nx + j*nx + i]; (this assumes the array is styleC, meaning the right-most index varies the fastest). So the data is always a flat 1-D contiguous array, and the multi-dimensional indexing is done via integer arithmetic under the hood. This is in contrast to C multi-dimensional arrays, which are pointers to pointers to pointers and less efficient because of the indirect addressing that needs to be done.

Reshaping the Array

You could potentially reshape this data by creating a non-owned YAKL Array with different dimensions. Using the afformentioned Array object, we could have:

Array<float,2,memDevice,styleC> arr2d("arr2d",arr.data(),nz,ny*nx);

This can now be addressed as level,column. Note that the underlying data hasn't changed. Only the metadata of the Array object has changed to accommodate a different method of indexing, which is still valid. Wrapping pointers in non-owned YAKL Array objects is less safe because you have to be sure the dimensions you ascribe to the pointer are indeed correct. There is potential for user error there.

Switching from C-style to Fortran-style

Like reshaping, when switching from C to Fortran style, the underlying data will remain the same. You just change the Array metadata to accommodate a different style of indexing. In this case, again using the first example of an Array object, we will use:

// An example of wrapping a C-style Array's data in a Fortran-style Array
// The C-style Array object is dimensioned nz,ny,nx
Array<float,3,memDevice,styleFortran> arrFortran("arrFortran",arr.data(),nx,ny,nz);

In this case, because Fortran expects the left-most index to vary the fastest ("column-major" indexing), we now put nx on the left and nz on the right. The underlying data is the same (nx varies the fastest), but the way we're indexing it has changed because the style of the Array object has changed.

Similarly, if you have a Fortran Array indexed as above, you can wrap it in a C-style Array object as follows:

// An example of wrapping a Fortran-style Array's data in a C-style Array
// The Fortran-style Array object is dimensioned nx,ny,nz
Array<float,3,memDevice,styleC> arrC("arrC",arrFortran.data(),nz,ny,nx);

Interoperating with Kokkos View objects

There are a couple of gotchas in dealing with a Kokkos View. Kokkos isn't as transparent as YAKL regarding the underlying data layout, and it's on purpose. Kokkos allows you to more flexibly switch how the data is laid out for performance purposes. This does, however, make it a bit more difficult to deal with other frameworks and with libraries. When wrapping a Kokkos View object in a YAKL Array object, the Kokkos LayoutRight is synonymous with YAKL's styleC, and Kokkos LayoutLeft is synonymous with YAKL's styleFortran. Any other layout (or an unspecified layout) will have undefined behavior. For instance, Kokkos View objects are supposedly LayoutRight on the CPU by default and LayoutLeft on the GPU (as a default), so it's not always easy to know what kind of data you have under the hood in Kokkos. When in doubt, write a small test driver to make sure you know what the View object's layout is.

If the Kokkos View object is LayoutRight, then you should treat it as if it were a YAKL Array object of styleC in the examples above. If it is LayoutLeft, then you should treat it as if it were a YAKL Array object of styleFortran in the examples above.