Road to C++ and High Performance Computing

This repo contains things I'm going to try out to learn C++ and High Performance Computing

A lot of these are content I gathered from various places of the internet. Raise an issue if I missed attributing any external content

Chapters

Below each chapter noting down the things I learnt in the process

• Chapter 1 - Basics

  • 1.1 Learnt a ton of new features available
  • 1.2 Run Time Type Information
  • 1.3 Constructors
  • 1.3 Destructor order

• Chapter 2 - Game of Life

  • 2.1 Basic GOL with setting up LLDB with VSCode
  • 2.2 GOL with SSE, interesting thing I noticed is that the first execution was a bit slower compared to the others.

• Chapter 3 - SHA-1

  • 3.1Endian Conversion optimization with OSByteOrder.h for Mac, this is converted into bswap, checked it by inspecting the generated ASM
  • 3.2Image Convolution
  • 3.3Image Histogram Equalization
  • 3.4 Discrete Fourier Transform

• Chapter 4 - Perlin Noise

  • 4.1 Generated Perlin Noise map for a 10^8 points, parallelized it with OpenMP
  • 4.2OpenMP case studies

• Chapter 5 - Fractals

  • 5.1 Abelian sandpile model with SSE
  • 5.2 Mandelbrot set with OpenMP
  • 5.3 Abelian sandpile model with auto vectorization for non square grids

• Chapter Acceleration

  • OpenCL
    • exp-1 Get available device info
    • exp-2 Vector addition in kernel
    • exp-3 Game of Life
    • exp-4 Atomics
    • exp-5 Abelian Sandpile(not the best example)
  • Cuda
    • hello Hello World with Cuda
    • vector_add Vector addition in device
  • OpenMP / OpenACC
    • collide_gpu xxtea middle block collision on GPU, based on this gist
    • reduction reduction operation on GPU
    • vector_add - Adding two vectors

• Chapter CMake

• Chapter Flex Bison

• Chapter Multi Process

• Chapter Auto Vectorization

• Chapter Profiling

• Chapter Benchmarks

• Chaper CASA

• Paraphernalia

  • Basic spawning threads with pthread and <thread>
  • Basic forking process
  • SSE condition paraphernalia
  • Opening Ini file

Talks

• Tuning C++: Benchmarks, and CPUs, and Compilers! Oh My!
• LLVM Hybrid Datastructures
• std::allocator

• Reads
  • Undefined Behaviour
  • Hybrid Parallel Programming

Tools and Things to Explore

Links are to respective documentations

• perf
• gprof
• valgrind
• vtune
• OpenMP
• OpenMPI
• Google Benchmark
• Google Test
• rr
• lldb

Topics

• Intrinsics
• Stream Processing
• Domain Decomposition

Papers

• Selected Papers in ACM SIGPLAN 1979-1999
• What Every Computer Scientist Should Know About Floating-Point Arithmetic
• What Every Programmer Should Know About Memory
• Shortlist of Top 10 Algorithms and links to their papers

Books

• Introduction to High Performance Scientific Computing
• Computer Systems: A Programmer's Perspective
• Linux Kernel in a Nutshell
• Linux Device Drivers
• C Unleashed

Guides

• OpenMP Guide
• OpenMP Examples
• OpenMP 4.0 - 4.5
• CPP Tutor
• StreamProcessing Reading List
• Domain Decomposition
• Awesome C++
• Advanced OpenMP
• Introduction to HPC

Blogs

• NullProgram by skeeto C, C++, Performance, intrinsics
• Krister Walfridsson’s blog LLVM, GCC, Valgrind, nvptx
• Fredrik Johansson Blog scientific computing, python
• John Burkardt's Blog hpc, mpi, openmp
• Scott Meyers
• Frank Tetzel
• Wojciech Muła