Profiling Decorators within Adagios

Why?

Running profiling can slow down your Adagios app quite a bit and is therefor not preferred in a production environment. But often you run into problems which only affect your production Adagios setup. Now you can profile single functions within Adagios with a simple decorator.

How?

It is quite simple, lets say I've found performance issues with adagios/rest/views.py:def javascript

Simply add the profile decorator to my function

@adagios.profiling.profile("rest-views-javascript")
def javascript(request, module_name, module_path):
     """ Create a javascript library that will wrap around module_path module """
     m = _load(module_path)

OK, where is the profiling data?

Profiles should be piling up in /var/lib/adagios which is configurable in settings.py:

PROFILE_LOG_BASE = "/var/lib/adagios"

Hotshot profile dumps

You can analyze the data using the hotshot module

#!/usr/bin/python

import hotshot.stats
import sys

stats = hotshot.stats.load(sys.argv[1])
stats.strip_dirs()
stats.sort_stats('time', 'calls')
stats.print_stats(20)

Output

$ ./analyze rest-views-javascript-20140515T1628209855 
         8249 function calls (8156 primitive calls) in 0.524 seconds

   Ordered by: internal time, call count
   List reduced from 245 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        5    0.344    0.069    0.430    0.086 /usr/lib/python2.6/site-packages/pynag/Parsers/__init__.py:2277(query)
       15    0.050    0.003    0.050    0.003 <string>:1(connect)
        6    0.035    0.006    0.035    0.006 /usr/lib64/python2.6/threading.py:116(acquire)
        5    0.029    0.006    0.029    0.006 /usr/lib64/python2.6/socket.py:409(readline)
        6    0.014    0.002    0.016    0.003 /usr/lib/python2.6/site-packages/pynag/Parsers/__init__.py:1568(_load_static_file)
        1    0.006    0.006    0.420    0.420 /usr/lib/python2.6/site-packages/pynag/Parsers/__init__.py:2209(test)
       32    0.005    0.000    0.006    0.000 /usr/lib64/python2.6/inspect.py:368(cleandoc)
        5    0.004    0.001    0.004    0.001 /usr/lib64/python2.6/socket.py:182(__init__)
        5    0.003    0.001    0.003    0.001 /usr/lib64/python2.6/socket.py:339(read)
        1    0.003    0.003    0.524    0.524 /usr/lib/python2.6/site-packages/adagios/rest/views.py:150(javascript)
     1597    0.002    0.000    0.003    0.000 /usr/lib/python2.6/site-packages/django/utils/encoding.py:54(force_unicode)
      387    0.002    0.000    0.015    0.000 /usr/lib/python2.6/site-packages/django/template/debug.py:82(render)
      352    0.002    0.000    0.003    0.000 /usr/lib/python2.6/site-packages/django/utils/html.py:32(escape)
      387    0.002    0.000    0.002    0.000 /usr/lib/python2.6/site-packages/django/utils/formats.py:132(localize)
        7    0.002    0.000    0.002    0.000 /usr/lib/python2.6/site-packages/pynag/Parsers/__init__.py:800(open)
        1    0.002    0.002    0.019    0.019 /usr/lib/python2.6/site-packages/django/template/defaulttags.py:132(render)
      369    0.002    0.000    0.005    0.000 /usr/lib/python2.6/site-packages/django/utils/functional.py:170(wrapper)
     38/1    0.001    0.000    0.019    0.019 /usr/lib/python2.6/site-packages/django/template/base.py:819(render)
      390    0.001    0.000    0.001    0.000 /usr/lib/python2.6/site-packages/django/utils/safestring.py:89(mark_safe)
      389    0.001    0.000    0.002    0.000 /usr/lib/python2.6/site-packages/django/template/base.py:738(_resolve_lookup)