pythonPerformance

Timing Framework

This is a helpful snippet in order to measure and compare different implementations.

import timeit
import pypiscout as sc
import datetime

FACTOR = 10**7+1

class aClass:
    def __init__(self):
        self.a = "a"
        self.b = "b"
        self.c = "c"
        self.d = "d"
        self.e = "e"
        self.f = "f"

anObj = aClass()


def function1ToMeasure():
    """
    YOUR CODE
    :return: None
    """
    pass


def function2ToMeasure():
    """
    YOUR CODE
    :return: None
    """
    pass


if __name__ == "__main__":
    sc.header("Let's do this!")

    timeStart_empty = timeit.default_timer()
    for i in range(0, FACTOR):
        pass
    timeEnd_empty = timeit.default_timer()
    sc.info("Job no load finished at:  {}  (duration: " "{:.12f} s)".format(datetime.datetime.now().strftime("%H:%M:%S"), (timeEnd_empty - timeStart_empty)))

    # ########################

    # Function 1
    timeStart = timeit.default_timer()
    for i in range(0, FACTOR):
        function1ToMeasure()
    timeEnd = timeit.default_timer()
    sc.info("Job 1 finished at:            {}  (duration: " "{:.12f} s)".format(datetime.datetime.now().strftime("%H:%M:%S"), (timeEnd - timeStart)))

    # Function 2
    timeStart = timeit.default_timer()
    for i in range(0, FACTOR):
        function1ToMeasure()
    timeEnd = timeit.default_timer()
    sc.info("Job 2 at:            {}  (duration: " "{:.12f} s)".format(datetime.datetime.now().strftime("%H:%M:%S"), (timeEnd - timeStart)))

Tracing Tools

• pyinstrument (statistical profiler (polls every 1 ms the call stack))
• Pythons build-in cProfile
  • Visualisation possible with
    • gprof2dot as a call graph
    • SnakeViz

Performance hints

Loops

• List comprehensions are generally faster than ordinary for-loops

Strings

Concatenation

F-strings are usually fastest (and (!) most readable). See also: https://stackoverflow.com/a/38362140/4773274

Additional Resources

• https://wiki.python.org/moin/PythonSpeed/PerformanceTips
• Map, for, list comprehensions, disassembly
• Looping over Pandas DataFrames

JIT

Cython

Actually this is not really JIT since you compile before you run.

• HowTo using setuptools | 1
• HowTo using setuptools | 2
• Set compiler via python.exe setup.py build_ext --inplace --compiler=msvc (Windows example with MSVC)

Linux

Process monitoring

top -p PID

-> However only live results (no history)

Windows

Windows Perfmon

• MS Windows perfmon cmd line usage
• Blog about perfmon

Perf equivalents

https://stackoverflow.com/questions/34641644/is-there-a-windows-equivalent-of-the-linux-command-perf-stat

Tracing

Windows performance toolkit and analyser: https://learn.microsoft.com/en-us/windows-hardware/test/wpt/