FluidX3D v2.0 (multi-GPU upgrade)

Big multi-GPU Update:

• Multi-GPU simulations are now possible on a single node (PC/laptop/server), allowing to pool VRAM from multiple GPUs.
  • Easy setup with minimal changes to the user: instead of LBM lbm(Nx, Ny, Nz, nu, ...);, use LBM lbm(Nx, Ny, Nz, Dx, Dy, Dz, nu, ...);, with Dx/Dy/Dz indicating how many domains (GPUs) in each spatial direction to use. By default, all identical GPUs will be automatically assigned their domains, however the GPUs can also be manually set with a list of their indices: ./make.sh 2 6 3 4 or /bin/FluidX3D 2 6 3 4.
  • All extensions are supported and validated to produce binary identical results compared to single-GPU simulations.
  • Multi-GPU also works with non-identical GPUs, regardless of vendor. Yes, you can run FluidX3D on unholy combinations of Nvidia/AMD/Intel GPUs/CPUs at the same time. I only recommend similar memory capacity and bandwidth, as the weakest GPU will bottleneck performance.
  • No SLI/Crossfire/NVLink/InfinityFabric is required. All communication runs over PCIe and is compatible with all hardware.
  • No MPI installation is required.
  • Total grid resolution must be equally divisible into domains, such that all domains are the same size.
  • The resolution of each domain is restricted to 4.29 billion grid points (2³², 225GB VRAM), but domain number and thus total grid resolution is unrestricted.
  • Under the hood: Complete re-write of C++ backend, to account for the domain decomposition architecture. The code is already fully optimized and shortened for maximum maintainability/upgradeability.
• Grid resolution can now be arbitrary and is not anymore restricted to the condition (Nx*Ny*Nz)%WORKGROUP_SIZE==0.

Known issues:

• Raytracing graphics are disabled for multi-GPU. The simulated light rays would have to travel through the entire simulation box, crossing domain boundaries. This is not easily possible, because each GPU only keeps its own domain in VRAM.