Check matmul types and error at compile-time if the backend doesn't support them

[cliffburdick]

Fixes #538

[cliffburdick]

/blossom-ci

[cliffburdick]

/blossom-ci

[AtomicVar]

@cliffburdick Great work! But I wonder why int8_t is listed as a supported GEMM type? I tried int8_t GEMM and it failed. cublasLtMatmulAlgoGetHeuristic() in ConfigureCublasLt() will return a CUBLAS_STATUS_NOT_SUPPORTED if we use int8_t.

Code to reproduce:

#include "matx.h"
#include <cassert>
#include <cstdio>

using namespace matx;

#define TYPE int8_t

int main() {
  MATX_ENTER_HANDLER();

  index_t M = 2;
  index_t N = 3;

  auto m = make_tensor<TYPE>({M, N});
  auto v = make_tensor<TYPE>({N, 1});

  m.SetVals({{1, 2, 3},
             {4, 5, 6}});
  v.SetVals({{1, 2, 3}});

  auto out = make_tensor<TYPE>({M, 1});

  (out = matmul(m, v)).run();

  cudaStreamSynchronize(0);

  printf("m:\n");
  print(m);
  printf("v:\n");
  print(v);
  printf("out:\n");
  print(out);

  CUDA_CHECK_LAST_ERROR();
  MATX_EXIT_HANDLER();
}

Output:

matxException (matxMatMulError: ret == CUBLAS_STATUS_SUCCESS) - /workspaces/MatX/include/matx/transforms/matmul.h:714

Stack Trace:
 examples/matmul_int : ()+0x8e4d
 examples/matmul_int : ()+0x1a3ee
 examples/matmul_int : ()+0x17ff0
 examples/matmul_int : ()+0x143f5
 examples/matmul_int : ()+0x10e16
 examples/matmul_int : ()+0x6664
 /lib/x86_64-linux-gnu/libc.so.6 : ()+0x29d90
 /lib/x86_64-linux-gnu/libc.so.6 : __libc_start_main()+0x80
 examples/matmul_int : ()+0x4ea5

[cliffburdick]

@cliffburdick Great work! But I wonder why int8_t is listed as a supported GEMM type? I tried int8_t GEMM and it failed. cublasLtMatmulAlgoGetHeuristic() in ConfigureCublasLt() will return a CUBLAS_STATUS_NOT_SUPPORTED if we use int8_t.

Code to reproduce:

#include "matx.h"
#include <cassert>
#include <cstdio>

using namespace matx;

#define TYPE int8_t

int main() {
  MATX_ENTER_HANDLER();

  index_t M = 2;
  index_t N = 3;

  auto m = make_tensor<TYPE>({M, N});
  auto v = make_tensor<TYPE>({N, 1});

  m.SetVals({{1, 2, 3},
             {4, 5, 6}});
  v.SetVals({{1, 2, 3}});

  auto out = make_tensor<TYPE>({M, 1});

  (out = matmul(m, v)).run();

  cudaStreamSynchronize(0);

  printf("m:\n");
  print(m);
  printf("v:\n");
  print(v);
  printf("out:\n");
  print(out);

  CUDA_CHECK_LAST_ERROR();
  MATX_EXIT_HANDLER();
}

Output:

matxException (matxMatMulError: ret == CUBLAS_STATUS_SUCCESS) - /workspaces/MatX/include/matx/transforms/matmul.h:714

Stack Trace:
 examples/matmul_int : ()+0x8e4d
 examples/matmul_int : ()+0x1a3ee
 examples/matmul_int : ()+0x17ff0
 examples/matmul_int : ()+0x143f5
 examples/matmul_int : ()+0x10e16
 examples/matmul_int : ()+0x6664
 /lib/x86_64-linux-gnu/libc.so.6 : ()+0x29d90
 /lib/x86_64-linux-gnu/libc.so.6 : __libc_start_main()+0x80
 examples/matmul_int : ()+0x4ea5

I'm going by the support matrix here:
https://docs.nvidia.com/cuda/cublas/#id98

I will try it out

[cliffburdick]

@AtomicVar the reason it's not working is the compute and scale types are wrong. Looking into it.

[cliffburdick]

@AtomicVar the reason it's not working is the compute and scale types are wrong. Looking into it.

Even with the correct scalar and compute types it's failing the heuristic check. Still investigating

[AtomicVar]

Even with the correct scalar and compute types it's failing the heuristic check. Still investigating

I'm also facing this weird issue. Don't know where's the problem.

[cliffburdick]

/blossom-ci