[cuda] Add CUDA version check

[0xzhang]

Related issue = fixes #3195

Issue
CUDADriver have two behaviours:

1. Load library;
2. Load CUDA functions.

In the past, error info throwed when loading function failure. Because lack of CUDA version's check. It assumes loading CUDA functions will be success.

After fix this, the behaviour of with_cuda will consistent with with_opengl. If cuda is not supported, arch will falling back to cpu directly and don't throw error info like before.

Implementation
The way I implemented because,

1. Version checking should be done after the library has been successfully loaded.
2. Version checking should be done before many CUDA functions are loaded.
3. The lower version library can load successfully, but the version check failed and the function cannot be loaded.
4. driver_get_version() is a one of the CUDA functions in file cuda_driver_functions.inc.h. Therefore I extracted this function from the file.
5. detected() will be called twice. First time, it used to check status of library loading when construct CUDADriver. Second time, it checks whether CUDA driver is ready. Thus, I set a member flag cuda_version_valid initialized with true. When construct CUDADriver, this flag is senseless. After instance constructed, the return value of detected() will dependent on actual CUDA driver status.

[netlify]

✔️ Deploy Preview for jovial-fermat-aa59dc canceled.

🔨 Explore the source changes: 5c4ac74

🔍 Inspect the deploy log: https://app.netlify.com/sites/jovial-fermat-aa59dc/deploys/61744aaaa833ef000799ae9d

[0xzhang]

taichi/python/taichi/lang/__init__.py

Lines 991 to 1009 in 53e04c6

	arch_table = {
	cuda: _ti_core.with_cuda,
	metal: _ti_core.with_metal,
	opengl: _ti_core.with_opengl,
	cc: _ti_core.with_cc,
	vulkan: lambda: _ti_core.with_vulkan(),
	wasm: lambda: True,
	cpu: lambda: True,
	}
	with_arch = arch_table.get(arch, lambda: False)
	try:
	return with_arch()
	except Exception as e:
	arch = _ti_core.arch_name(arch)
	_ti_core.warn(
	f"{e.__class__.__name__}: '{e}' occurred when detecting "
	f"{arch}, consider add `export TI_WITH_{arch.upper()}=0` "
	f" to environment variables to depress this warning message.")
	return False

For current archs, env variables TI_ENABLE_METAL/TI_ENABLE_OPENGL/TI_ENABLE_CUDA are avaliable. Env variable are not considered in arch cc and vulkan's implementation. They only use C++ macro TI_WITH_CC and TI_WITH_VULKAN to decide return value.

Currently, error info in file __init__.py is only work for metal, opengl and cuda. Support of env variables TI_ENABLE_VULKAN/TI_ENABLE_CC for vulkan and cc may be need to be implemented.

[k-ye]

Awesome! I only got a few nits

[k-ye]

Suggested change

	f"{arch}, consider add `TI_ENABLE_{arch.upper()}=0` "
	f"{arch}, consider adding `TI_ENABLE_{arch.upper()}=0` "

[k-ye]

// not your fault :-)

Suggested change

	f" to environment variables to depress this warning message.")
	f" to environment variables to suppress this warning message.")

[k-ye]

Let's show a bit more information here:

TI_WARN("The Taichi CUDA backend requires at least CUDA 10.0, got {}", version);

[k-ye]

FYI, our CPP guide is at https://docs.taichi.graphics/lang/articles/contribution/cpp_style

Suggested change

	bool cuda_version_valid = true;
	bool cuda_version_valid_{false};

[0xzhang]

Here I use true value to initialize.
😂 It's tricky. But I consider use this way to solves the problem.

The return value of detected() dependent on CUDA version and library loading together.

• If failed to load library. cuda_version_valid still be true. But always return false.
• If successed to load library. cuda_version_valid will be set according actual status. And return value is dependent on cuda_version_valid.

It's all because the function detected() be first called when construct singleton instance CUDADriver. 😂

Maybe I could decoupling detected() from CUDADriver constructor. The value of cuda_version_valid will be seemed more safer.

[k-ye]

Thanks for the detailed explanation!

The fact that detect() is called during initialization makes things a bit tricky -- Its value is supposed to be constant throughout the lifetime of CUDADriver. To improve the situation a bit, we can do this:

1. Cache the result of TI_ENABLE_CUDA in another bool member
2. Don't call detect() inside the ctor.

So the pseudo code can be something like this:

class CUDADriver {
 public:
  CUDADriver() {
    disabled_by_env_ = (get_environ_config("TI_ENABLE_CUDA", 1) == 0);
    if (disabled_by_env_) {
      // return directly
    }
    loader_ = ...;
    if (!loader_->loaded()) {
      // return directly
    }
    // Read version from the loader
    if (version < 10000) {
      cuda_version_valid_ = false;
      // rteturn directly
    }
  }

  bool detected() {
    return !disabled_by_env_ && valid_cuda_version_ && loader_->loaded();
  }
 private:
  bool disabled_by_env_{false};
  bool valid_cuda_version_{false};
};

[0xzhang]

Thanks very much!!
I changed the implementation.

[k-ye]

LGTM, thanks!