Would it be more scalable and have better performance if ndarray.tobytes() returned a Span<byte>?

[lintao185]

    public static IList GetList<T>(ndarray ndarray) where T : struct
    {
        var values = MemoryMarshal.Cast<byte, T>(ndarray.tobytes());
        var list = GetList(values, 0, values.Length, ndarray.shape.iDims);
        return list;
    }
      private static IList GetList<T>(Span<T> values, int start, int end, long[] shapeIDims)
    {
        if (shapeIDims.Length == 1)
        {
            var listr=new List<T>(end-start);
            listr.AddRange(values[start..end]);
            return listr;
        }
        var genericType = typeof(List<>);
        var argType = typeof(T);
        for (int i = 0; i < shapeIDims.Length; i++)
        {
            argType = genericType.MakeGenericType(argType);
        }

        var list = (IList)Activator.CreateInstance(argType);
        var valueLength = end - start;
        var length = (int)(valueLength / shapeIDims[0]);
        for (int i = 0; i < shapeIDims[0]; i++)
        {
            var newStart = start + (i * length);
            var newEnd  = start + ((i + 1) * length);
            newEnd = newEnd >= values.Length ? values.Length : newEnd;
            list.Add(GetList(values, newStart, newEnd, shapeIDims[1..]));
        }

        return list;
    }
  var gn = np.ones((5, 10000, 10000)).astype(np.Float32);
  var gnLs = GetList<float>(gn);

In this example, the conversion is particularly slow.

[KevinBaselinesw]

I don't have time right now, but I will experiment with this when I get a chance.

Question: is this code MemoryMarshal.Cast<byte, T>(ndarray.tobytes()); even valid in a .NET standard application? In other words , will it work on .net versions for linux and mac too?

I welcome assistance. If you want branch the code and try to turn this into a generic API for different data types, please fell free. I will review it and possibly accept it into the main branch.

[KevinBaselinesw]

I tried to add a unit test to experiment with your sample. I does not compile cleanly for me. I have not used Span before so it may be user error on my part.

These lines do not compile cleanly:

values[start..end];
shapeIDims[1..]

Can you help?

    [TestMethod]
        public void test_lintao185_1()
        {
            var gn = np.ones((5, 10000, 10000)).astype(np.Float32);
            var gnLs = GetList<float>(gn);


        }

        public static IList GetList<T>(ndarray ndarray) where T : struct
        {
            var values = MemoryMarshal.Cast<byte, T>(ndarray.tobytes());
            var list = GetList(values, 0, values.Length, ndarray.shape.iDims);
            return list;
        }
        private static IList GetList<T>(Span<T> values, int start, int end, long[] shapeIDims)
        {
            if (shapeIDims.Length == 1)
            {
                var listr = new List<T>(end - start);
                listr.AddRange(values[start..end]);
                return listr;
            }
            var genericType = typeof(List<>);
            var argType = typeof(T);
            for (int i = 0; i < shapeIDims.Length; i++)
            {
                argType = genericType.MakeGenericType(argType);
            }

            var list = (IList)Activator.CreateInstance(argType);
            var valueLength = end - start;
            var length = (int)(valueLength / shapeIDims[0]);
            for (int i = 0; i < shapeIDims[0]; i++)
            {
                var newStart = start + (i * length);
                var newEnd = start + ((i + 1) * length);
                newEnd = newEnd >= values.Length ? values.Length : newEnd;
                list.Add(GetList(values, newStart, newEnd, shapeIDims[1..]));
            }

            return list;
        }

[xela-trawets]

Hi Kevin,

I just happened to click on this and am interested... so please pardon me for butting in.

The two lines that you mention
values[start..end];
shapeIDims[1..]

The compile error is not specific to Spans,
Those lines use a feature from c# version 8 called "range",
(https://learn.microsoft.com/en-us/dotnet/csharp/tutorials/ranges-indexes)

    list.Add(GetList(values, newStart, newEnd, shapeIDims[1..]));

can be read as
list.Add(GetList(values, newStart, newEnd, shapeIDims.Skip(1).ToArray()));// Allocates new array

    listr.AddRange(values[start..end]);

can be read as
listr.AddRange(values.Slice(start,end - start)); //slice() itself does not allocate

With regards to your other question..
Spans and MemoryMarshal.Cast are ".Net standard 2.0" onward and work fine on Linux and all "dotnet" (core). (The implementation of Spans on ".NET Framework 4.8" was slightly different, but it works there too).

A mini primer on Spans:

1. Spans are basically pointers with a length field.
2. Spans are (typed) references stored in a "ref struct" - ( not an object ).
3. Spans refer to memory already allocated - in an Array, stackalloc, native allocation, mapped file etc.
4. Spans cannot be safely stored in objects fields or properties
   -specifically no List and no Array Span[] can exist.
5. Spans cannot be used in code with await (due to the fourth law).

Memory is more amenable to storing and async code, Memory is also a typed reference with a length, but is an object and not a struct.

if you have some time, (and dont mind watching videos) the following recent presentation does a reasonable job at explaining how revolutionary the introduction of Span is:
https://learn.microsoft.com/en-us/shows/on-net/a-complete-dotnet-developers-guide-to-span-with-stephen-toub#time=31m51s

Alex

[KevinBaselinesw]

I appreciate you butting in :) Thanks for the tips. I appreciate you pointing the info.

[KevinBaselinesw]

As I interpret your question, the answer is that YES, converting an existing array to a Span via MemoryMarshal.Cast is much faster than using the existing "tobytes" method. In this unit test below, it took 7743 ms to convert using tobytes and only 205ms using the MemoryMarshal.Cast method.

  [TestMethod]
        public void test_lintao185_1()
        {
            var gn = np.ones((100000000)).astype(np.Float32);

            System.Diagnostics.Stopwatch sw1 = new System.Diagnostics.Stopwatch();
            sw1.Restart();

            var bytes1 = gn.tobytes();
            var gn1 = np.array(bytes1);

            sw1.Stop();

            //var gnLs = GetList<float>(gn);

            System.Diagnostics.Stopwatch sw2 = new System.Diagnostics.Stopwatch();
            sw2.Restart();

            var bytes2 = MemoryMarshal.Cast<float, byte>(gn.AsFloatArray());
            var gn2 = np.array(bytes2.ToArray());

            sw2.Stop();

            Console.WriteLine(sw1.ElapsedMilliseconds);  <--7743
            Console.WriteLine(sw2.ElapsedMilliseconds);  <--205


        }

[KevinBaselinesw]

In the unit test below, the call to tobytes returns a new array of bytes, copied from the source array. If you modify the bytes1 array it does not affect the original. If you use the MemoryMarshall.Cast method, you get a view into the original array and any modifications made to the bytes2 will be reflected in the original array. So, they are very different.

    [TestMethod]
        public void test_lintao185_2()
        {
            var gn = np.ones((10, 10, 10)).astype(np.Int32);

            var bytes1 = gn.tobytes();
            bytes1[0] = 99;


            var gn2 = np.ones((10, 10, 10)).astype(np.Int32);
            var bytes2 = MemoryMarshal.Cast<int, byte>(gn2.AsInt32Array());
            bytes2[0] = 99;

            return;


        }

[KevinBaselinesw]

Unless I misunderstood the question, I think the existing implementation of .tobytes() is closer to the python version (although that actually returns a string instead of a byte[]).

But this was a great exercise to show that anyone who wants to map an ndarray object to a different data type can do so much faster with MemoryMarshall.Cast as long as they don't want a copy.