Checking for Memory Leaks

The SLURM C API loads data structures into memory and must be freed at some point in order to avoid memory leaks. PySLURM uses Cython to wrap these C API functions and must also free memory where appropriate. This is especially important when writing scripts using PySLURM for long-lived daemons, as the memory used on the system can increase significantly.

Use Valgrind to help track down memory leaks. Valgrind can output quite a bit of possibly lost blocks of memory due to the Python interpreter. These are not related to PySLURM and can be ignored. Valgrind >= 3.9.0 has the --show-leak-kinds flag, which you can set to definite to only show definitely lost blocks of memory. This can be quite helpful in tracking down memory leaks. For example, to check if a PySLURM script called test-load_node.py is leaking memory, run the following valgrind command:

$ valgrind --leak-check=full --num-callers=30 --show-leak-kinds=definite python test-load-node.py

Pay attention to the summary output, in particular the definitely lost line. It should be 0 or there is a memory leak.

[output truncated...]
==21043== LEAK SUMMARY:
==21043==    definitely lost: 0 bytes in 0 blocks
==21043==    indirectly lost: 0 bytes in 0 blocks
==21043==      possibly lost: 310,835 bytes in 1,559 blocks
==21043==    still reachable: 771,917 bytes in 5,115 blocks
==21043==         suppressed: 0 bytes in 0 blocks

If you get any definitely lost memory blocks, report an issue. The problem is either in PySLURM or the underlying SLURM C API and requires some investigation.