Merge pull request #37 from MindscapeHQ/v3.1.0

V3.1.0

.coveragerc

@@ -4,6 +4,7 @@ omit =
     */site-packages/*
     */dist-packages/*
     */tests/*
+    /opt/*
 
  exclude_lines =
     except Exception as e:

.travis.yml

@@ -11,12 +11,17 @@ install:
   - pip install nose
   - pip install coveralls
   - pip install unittest2
+  - pip install mock
+  - pip install 'django==1.8.8'
 
 script:
   - pip install jsonpickle
-  - if [[ $TRAVIS_PYTHON_VERSION == 2* ]]; then coverage run -m unittest2 discover python2; fi
+  # Django 1.8 requires Python 2.7+
+  - if [[ $TRAVIS_PYTHON_VERSION != 2.6 && $TRAVIS_PYTHON_VERSION == 2* ]]; then coverage run -m unittest2 discover python2; fi
   # Coverage/Coveralls has dropped support for Python 3.2
   - if [[ $TRAVIS_PYTHON_VERSION != 3.2 && $TRAVIS_PYTHON_VERSION == 3* ]]; then coverage run -m unittest discover python3; fi
+  - if [[ $TRAVIS_PYTHON_VERSION == 'pypy' ]]; then coverage run -m unittest2 discover python2; fi
+  # - if [[ $TRAVIS_PYTHON_VERSION == 'pypy3' ]]; then coverage run -m unittest discover python3; fi
 
 after_success:
   coveralls

CHANGELOG.rst

@@ -1,3 +1,11 @@
+3.1.0
+
+- Add on_grouping_key for custom grouping logic
+- Add Django, Flask and WSGI middleware for Python 3
+- Allow Django settings.py config override
+- Surface more environment data (versions, interpreter location etc) for Python itself and Django/Flask
+- Fix WSGI close() implementation when an exception is raised while handling a prior one (match spec)
+
 3.0.3
 
 - Add 'httpTimeout' option to config

README.rst

@@ -85,14 +85,13 @@ This uses the built-in :code:`RaygunHandler`. You can provide your own handler i
 Web frameworks
 --------------
 
-Python 2
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Raygun4py in Python 2.x includes several middleware implementations for various frameworks to enable reporting out of the box:
+Raygun4py includes dedicated middleware implementations for Django and Flask, as well as generic WSGI frameworks (Tornado, Bottle, Ginkgo etc). These are available for both Python 2.6/2.7 and Python 3+.
 
 Django
 ++++++
 
+To configure Django to automatically send all exceptions that are raised in views to Raygun:
+
 settings.py
 
 .. code:: python
@@ -101,10 +100,28 @@ settings.py
       'raygun4py.middleware.django.Provider'
   )
 
-  RAYGUN4PY_API_KEY = 'your_apikey'
+  RAYGUN4PY_CONFIG = {
+      'api_key': 'paste_your_api_key_here'
+  }
 
-Exceptions that occur in views will be automatically sent to Raygun.
 
+The above configuration is the minimal required setup. The full set of options supported by the provider can be declared in the same way:
+
+.. code:: python
+
+  RAYGUN4PY_CONFIG = {
+      'api_key': 'paste_your_api_key_here',
+      'http_timeout': 10.0,
+      'proxy': None,
+      'before_send_callback': None,
+      'grouping_key_callback': None,
+      'filtered_keys': [],
+      'ignored_exceptions': [],
+      'transmit_global_variables': True,
+      'transmit_local_variables': True,
+      'userversion': "Not defined",
+      'user': None
+  }
 
 Flask
 +++++
@@ -163,21 +180,28 @@ Documentation
 Initialization options
 ----------------------
 
-:code:`RaygunSender` accepts a :code:`config` dict which is used to set options for the provider:
+:code:`RaygunSender` accepts a :code:`config` dict which is used to set options for the provider (the defaults are shown below):
 
 .. code:: python
 
   from raygun4py import raygunprovider
 
   client = raygunprovider.RaygunSender('your_apikey', config={
-    'transmitLocalVariables': True,
-    'transmitGlobalVariables': True,
-    'httpTimeout': 10
+      'http_timeout': 10.0,
+      'proxy': None,
+      'before_send_callback': None,
+      'grouping_key_callback': None,
+      'filtered_keys': [],
+      'ignored_exceptions': [],
+      'transmit_global_variables': True,
+      'transmit_local_variables': True,
+      'userversion': "Not defined",
+      'user': None
   })
 
-If either of the first two are set to False, the corresponding variables will not be sent with exception payloads. Both default to True.
+For the local/global variables, if their options are set to False the corresponding variables will not be sent with exception payloads.
 
-httpTimeout controls the maximum time the HTTP request can take when POSTing to the Raygun API, and is a float. It defaults to 10s.
+httpTimeout controls the maximum time the HTTP request can take when POSTing to the Raygun API, and is of type 'float'.
 
 Sending functions
 -----------------
@@ -252,6 +276,14 @@ Provide a list of exception types to ignore here. Any exceptions that are passed
 
 You can mutate the candidate payload by passing in a function that accepts one parameter using this function. This allows you to completely customize what data is sent, immediately before it happens.
 
++------------------+---------------+--------------------+
+| Function         | Arguments     | Type               |
++==================+===============+====================+
+| on_grouping_key  | callback      | Function           |
++------------------+---------------+--------------------+
+
+Pass a callback function to this method to configure custom grouping logic. The callback should take one parameter, an instance of RaygunMessage, and return a string between 1 and 100 characters in length (see 'Custom Grouping Logic' below for more details).
+
 +----------------+---------------+--------------------+
 | Function       | Arguments     | Type               |
 +================+===============+====================+
@@ -290,6 +322,30 @@ User data can be passed in which will be displayed in the Raygun web app. The di
 
 `identifier` should be whatever unique key you use to identify users, for instance an email address. This will be used to create the count of unique affected users. If you wish to anonymize it, you can generate and store a UUID or hash one or more of their unique login data fields, if available.
 
+Custom grouping logic
+---------------------
+
+You can create custom exception grouping logic that overrides the automatic Raygun grouping by passing in a function that accepts one parameter using this function. The callback's one parameter is an instance of RaygunMessage (python[2/3]/raygunmsgs.py), and the callback should return a string.
+
+The RaygunMessage instance contains all the error and state data that is about to be sent to the Raygun API. In your callback you can inspect this RaygunMessage, hash together the fields you want to group by, then return a string which is the grouping key.
+
+This string needs to be between 1 and 100 characters long. If the callback is not set or the string isn't valid, the default automatic grouping will be used.
+
+By example:
+
+.. code:: python
+
+    class MyClass(object):
+
+        def my_callback(self, raygun_message):
+            return raygun_message.get_error().message[:100] # Use naive message-based grouping only
+
+        def create_raygun_and_bind_callback(self):
+            sender = raygunprovider.RaygunSender('api_key')
+            sender.on_grouping_key(self.my_callback)
+
+The RaygunSender above will use the my_callback to execute custom grouping logic when an exception is raised. The above logic will use the exception message only - you'll want to use a more sophisticated approach, usually involving sanitizing or ignoring data.
+
 Chained exceptions
 ------------------
 

python2/raygun4py/middleware/django.py

@@ -1,19 +1,24 @@
 from __future__ import absolute_import
 
+import django
 from django.conf import settings
 
 from raygun4py import raygunprovider
 
+
 class Provider(object):
 
     def __init__(self):
-    	apiKey = getattr(settings, 'RAYGUN4PY_API_KEY', None)
-        self.sender = raygunprovider.RaygunSender(apiKey)
+        config = getattr(settings, 'RAYGUN4PY_CONFIG', {})
+        apiKey = getattr(settings, 'RAYGUN4PY_API_KEY', config.get('api_key', None))
+
+        self.sender = raygunprovider.RaygunSender(apiKey, config=config)
 
     def process_exception(self, request, exception):
-    	raygunRequest = self._mapRequest(request)
+        raygunRequest = self._mapRequest(request)
+        env = self._get_django_environment()
 
-        self.sender.send_exception(exception=exception, request=raygunRequest)
+        self.sender.send_exception(exception=exception, request=raygunRequest, extra_environment_data=env)
 
     def _mapRequest(self, request):
         headers = request.META.items()
@@ -32,3 +37,8 @@ def _mapRequest(self, request):
             'headers': _headers,
             'rawData': request.body if hasattr(request, 'body') else getattr(request, 'raw_post_data', {})
         }
+
+    def _get_django_environment(self):
+        return {
+            'frameworkVersion': django.get_version()
+        }

python2/raygun4py/middleware/flask.py

@@ -2,32 +2,37 @@
 
 import logging
 
+import flask
 from flask.signals import got_request_exception
 
 from raygun4py import raygunprovider
 
-
 log = logging.getLogger(__name__)
 
 
 class Provider(object):
+    def __init__(self, flaskApp, apiKey):
+        self.flaskApp = flaskApp
+        self.apiKey = apiKey
 
-	def __init__(self, flaskApp, apiKey):
-		self.flaskApp = flaskApp
-		self.apiKey = apiKey
+        got_request_exception.connect(self.send_exception, sender=flaskApp)
 
-		got_request_exception.connect(self.send_exception, sender=flaskApp)
+        flaskApp.extensions['raygun'] = self
 
-		flaskApp.extensions['raygun'] = self
+    def attach(self):
+        if not hasattr(self.flaskApp, 'extensions'):
+            self.flaskApp.extensions = {}
 
-	def attach(self):
-		if not hasattr(self.flaskApp, 'extensions'):
-			self.flaskApp.extensions = {}
+        self.sender = raygunprovider.RaygunSender(self.apiKey)
 
-		self.sender = raygunprovider.RaygunSender(self.apiKey)
+    def send_exception(self, *args, **kwargs):
+        if not self.sender:
+            log.error("Raygun-Flask: Cannot send as provider not attached")
 
-	def send_exception(self, *args, **kwargs):
-		if not self.sender:
-			log.error("Raygun-Flask: Cannot send as provider not attached")
+        env = self._get_flask_environment()
+        self.sender.send_exception(extra_environment_data=env)
 
-		self.sender.send_exception()
+    def _get_flask_environment(self):
+        return {
+            'frameworkVersion': 'Flask ' + getattr(flask, '__version__', '')
+        }

python2/raygun4py/middleware/wsgi.py

@@ -16,29 +16,26 @@ def __call__(self, environ, start_response):
         if not self.sender:
             log.error("Raygun-WSGI: Cannot send as provider not attached")
 
-        try:
-            chunk = self.app(environ, start_response)
-        except Exception as e:
-            request = self.build_request(environ)
-            self.sender.send_exception(exception=e, request=request)
-
-            raise
+        iterable = None
 
         try:
-            for event in chunk:
+            iterable = self.app(environ, start_response)
+            for event in iterable:
                 yield event
+
         except Exception as e:
-            request = build_request(environ)
+            request = self.build_request(environ)
             self.sender.send_exception(exception=e, request=request)
-
             raise
+
         finally:
-            if chunk and hasattr(chunk, 'close') and callable(chunk.close):
+            if hasattr(iterable, 'close'):
                 try:
-                    chunk.close()
+                    iterable.close()
                 except Exception as e:
-                    request = build_request(environ)
-                    self.send_exception(exception=e, request=request)
+                    request = self.build_request(environ)
+                    self.sender.send_exception(exception=e, request=request)
+                    raise
 
     def build_request(self, environ):
         request = {}

python2/raygun4py/raygunmsgs.py

@@ -1,7 +1,6 @@
-import traceback
-import inspect
-import os
 import sys
+import os
+import inspect
 
 try:
     import multiprocessing
@@ -28,17 +27,24 @@ def set_machine_name(self, name):
         self.raygunMessage.details['machineName'] = name
         return self
 
-    def set_environment_details(self):
+    def set_environment_details(self, extra_environment_data):
         self.raygunMessage.details['environment'] = {
             "processorCount": (
                 multiprocessing.cpu_count() if USE_MULTIPROCESSING else "n/a"
             ),
             "architecture": platform.architecture()[0],
             "cpu": platform.processor(),
             "oSVersion": "%s %s" % (platform.system(), platform.release()),
-            "environmentVariables": os.environ.data
+            "environmentVariables": os.environ.data,
+            "runtimeLocation": sys.executable,
+            "runtimeVersion": 'Python ' + sys.version
         }
 
+        if extra_environment_data is not None:
+            merged = extra_environment_data.copy()
+            merged.update(self.raygunMessage.details['environment'])
+            self.raygunMessage.details['environment'] = merged
+
         return self
 
     def set_exception_details(self, raygunExceptionMessage):
@@ -48,14 +54,14 @@ def set_exception_details(self, raygunExceptionMessage):
     def set_client_details(self):
         self.raygunMessage.details['client'] = {
             "name": "raygun4py",
-            "version": "2.2.0",
+            "version": "3.1.0",
             "clientUrl": "https://github.com/MindscapeHQ/raygun4py"
         }
         return self
 
-    def set_customdata(self, userCustomData):
-        if type(userCustomData) is dict:
-            self.raygunMessage.details['userCustomData'] = userCustomData
+    def set_customdata(self, user_custom_data):
+        if type(user_custom_data) is dict:
+            self.raygunMessage.details['userCustomData'] = user_custom_data
         return self
 
     def set_tags(self, tags):
@@ -72,7 +78,7 @@ def set_request_details(self, request):
               "queryString": request['queryString'],
               "form": request['form'],
               "headers": request['headers'],
-              "rawData": request['rawData'],
+              "rawData": request['rawData']
             }
 
             if 'ipAddress' in request: