[NDTensors] Start TensorAlgebra module

[mtfishman]

This is initial work on the TensorAlgebra module mentioned in #1250.

So far, it defines a simplified version of TTGT tensor contraction (https://arxiv.org/abs/1607.00145), which can be called with TensorAlgebra.contract(randn(2, 3), (:a, :b), randn(3, 4), (:b, :c); alg="matricize").

The code design is inspired by the one in TensorOperations.jl, in particular it makes use of what I call a BipartitionedPermutation which is a permutation of the form ((1, 3), (2, 4)), which defines a matricization of a tensor. The code logic revolves around analyzing the input labels and defining bipartitioned permutations/matricizations of the input tensors which define how they will be mapped to matrix multiplications.

Right now the logic misses some optimizations that are captured by the current ContractionProperties logic, namely it doesn't avoid certain permutations in cases which can be mapped to matrix multiplication through reshapes and transpositions, for example TensorAlgebra.contract(a, (:a, :b), b, (:c, :b)) can be mapped to a matrix multiplication a * transpose(b) but currently the code will eagerly permute b. This will be handled in future work, hopefully in a simpler way than the current ContractionProperties code (perhaps by simply analyzing the BipartitionedPermutation and swapping the two partitions and seeing if that corresponds to a trivial permutation, in which case it will be marked as a transposition).

Some of the code overlaps with code logic in #1246, for example both have matricize/fusedims functionality and some quasi-generic functionality for allocating the output of tensor operations. #1246 focused more on block sparse arrays but also has some more generic code. The goal will be to make the code in TensorAlgebra more generic so then a lot of the logic for matricization, along with matricized linear algebra (tensor contraction, tensor decompositions, etc.), can live just in TensorAlgebra and submodules like BlockSparseArrays can mostly make use of generic code and it will "just work" when certain overloads are defined, like information about how to fuse blocked axes/indices.

@kmp5VT @emstoudenmire

[emstoudenmire]

Great to see this. I think it will lead to a lot shorter code that can do more things (e.g. plug in more backends which can be simpler themselves).

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (408516d) 85.44% compared to head (e4afb2c) 54.70%.

❗ Current head e4afb2c differs from pull request most recent head 9c8558c. Consider uploading reports for the commit 9c8558c to get more accurate results

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1265       +/-   ##
===========================================
- Coverage   85.44%   54.70%   -30.75%     
===========================================
  Files          89       88        -1     
  Lines        8402     8349       -53     
===========================================
- Hits         7179     4567     -2612     
- Misses       1223     3782     +2559     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.