Skip to content
This repository has been archived by the owner on Dec 6, 2023. It is now read-only.

Latest commit

 

History

History
179 lines (138 loc) · 3.91 KB

README.md

File metadata and controls

179 lines (138 loc) · 3.91 KB
Raw

GPU for Science Day Mini-App

Hacking Competition Code for NERSC's GPU for Science Day, July 2019

Project Layout

  • ^/gpp
    • gpp.cpp
      • main function and the kernel to focus on
    • Makefile
  • ^/external
    • commonDefines.h
    • arrayMD/
      • host-device data array
    • ComplexClass/
      • custom complex number class
    • timemory
      • C++ template library for performance reporting
      • PGI compiler
        • v18.10 will fail to compile this library
        • v19.5 (default loaded by modules below) will compile this library but not report anything -- this is a compiler bug
    • cereal
      • Library used by timemory for serialization (not used)

Development Environment

Logging on to Cori

If you are already a NERSC user, your normal account should be enabled for the Cori GPU nodes.

ssh yournerscusername@cori.nersc.gov
password + OTP

If you are not a NERSC user, we have provided some test accounts.

ssh testXXX@cori.nersc.gov
testaccountpassword

Obtaining code

cd $HOME
git clone https://github.com/NERSC/gpu-for-science-day-july-2019.git
cd gpu-for-science-day-july-2019/gpp

Cori Modules [REQUIRED]

module use /usr/common/software/gpu4sci-jul-2019/modulefiles
Available modules
gpu4science/gcc
gpu4science/intel
gpu4science/cuda
gpu4science/kokkos
gpu4science/openmp
gpu4science/openacc

Info: To switch between available modules:

make clean
# make sure unload the current module
module unload gpu4science/{gcc,intel,cuda,kokkos,openmp,openacc}
# load the new module
module load gpu4science/{gcc,intel,cuda,kokkos,openmp,openacc}

Cori GPU

Get GPU node

module load gpu4science/required
salloc -A gpu4sci -C gpu -N 1 -t 04:00:00 -c 10 --gres=gpu:1

Build CPU (sequential) version

# setup
module load gpu4science/intel

# build
make COMP=intel

CUDA

# setup
module load gpu4science/cuda

# build
make COMP=cuda

OpenACC

# setup
module load gpu4science/openacc

# build
make COMP=openacc

OpenMP

# setup
module load gpu4science/openmp

# build
make COMP=openmp
export OMP_NUM_THREADS=10

Info: For the Cori GPU Skylake CPU, set OMP_NUM_THREADS=10. (This will not affect the GPU.)

Kokkos

# setup
module load gpu4science/kokkos

# build
make COMP=kokkos

Testing

Running test/debugging problem

  • Fast, good for debugging
srun ./gpp.ex test

Hint: Do NOT optimize the test problem, it runs so quickly it is not representative

Running benchmark problem

  • Slow, this is how we will determine the hackathon winner
srun ./gpp.ex benchmark

Hint: Optimize this problem

Competition Submission

  1. Decide on a team name, if you have not done so already
    • In the steps below, replace TEAM_NAME with this name
  2. Go to the top-level directory
  3. Create a branch for your team
    • git checkout -b TEAM_NAME
  4. Check to make sure your directory is clean
    • git status
    • remove any build files or outputs that show up
    • Do not add/commit any files other than code
  5. Stage the files you want to commit
    • git add gpp/gpp.cpp and any other relevant text files, e.g. gpp/gpp.cu
  6. Commit the code
    • git commit -m "Official submission of TEAM_NAME"
  7. Push the code upstream
    • git push
  8. Execute the PyCTest script
    • python ./pyctest-runner.py --team=TEAM_NAME --compiler=COMPILER
      • TEAM_NAME should be your team name... Please do not copy/paste and submit as TEAM_NAME
      • COMPILER should be one of openacc, openmp, cuda, or kokkos
    • This script will:
      • Build the code in the gpp folder
      • Execute the benchmark test
      • Submit the build and test logs to cdash.nersc.gov