PyProf is a tool that profiles and analyzes the GPU performance of PyTorch models. PyProf aggregates kernel performance from Nsight Systems or NvProf.

• Monkey patch support for APEX, fused Adam, and Layer Norm functions
• PyYAML requirement has been removed
• Error handling for non-existent parse file arguments has been added

• Identifies the layer that launched a kernel: e.g. the association of ComputeOffsetsKernel with a concrete PyTorch layer or API is not obvious.
• Identifies the tensor dimensions and precision: without knowing the tensor dimensions and precision, it's impossible to reason about whether the actual (silicon) kernel time is close to maximum performance of such a kernel on the GPU. Knowing the tensor dimensions and precision, we can figure out the FLOPs and bandwidth required by a layer, and then determine how close to maximum performance the kernel is for that operation.
• Forward-backward correlation: PyProf determines what the forward pass step is that resulted in the particular weight and data gradients (wgrad, dgrad), which makes it possible to determine the tensor dimensions required by these backprop steps to assess their performance.
• Determines Tensor Core usage: PyProf can highlight the kernels that use Tensor Cores.
• Correlate the line in the user's code that launched a particular kernel (program trace).

The current release of PyProf is 3.2.0 and is available in the 20.07 release of the PyTorch container on NVIDIA GPU Cloud (NGC). The branch for this release is r20.07.

A Quick Start Guide and Installation Guide provide instructions on how to install and use PyProf.

The User Guide can be found in the documentation for current release.

An FAQ provides answers for frequently asked questions.

Contributions to PyProf are more than welcome. To contribute make a pull request and follow the guidelines outlined in the Contributing document.

We appreciate any feedback, questions or bug reporting regarding this project. When help with code is needed, follow the process outlined in the Stack Overflow (https://stackoverflow.com/help/mcve) document. Ensure posted examples are:

• minimal – use as little code as possible that still produces the same problem
• complete – provide all parts needed to reproduce the problem. Check if you can strip external dependency and still show the problem. The less time we spend on reproducing problems the more time we have to fix it
• verifiable – test the code you're about to provide to make sure it reproduces the problem. Remove all other problems that are not related to your request/question.