pygelf allows to send logs into Graylog directly from your app using built-in logging module.
Each log message is transformed into GELF (Graylog Extended Log Format).
Currently TCP, UDP and TLS (encrypted TCP) logging handlers are supported.
pip install pygelf
from pygelf import GelfTcpHandler, GelfUdpHandler, GelfTlsHandler
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
logger.addHandler(GelfTcpHandler(host='127.0.0.1', port=9401, debug=True))
logger.addHandler(GelfUdpHandler(host='127.0.0.1', port=9402, compress=True, chunk_size=1350))
logger.addHandler(GelfTlsHandler(host='127.0.0.1', port=9403, validate=True, ca_certs='/etc/ssl/certs/ca-ceritficates.crt'))
logging.info('hello gelf')
Each handler has the following parameters:
- host: ip address of the GELF input
- port: port of the GELF input
- debug (false by default): if true, each log message will include debugging info: module name, file name, line number, method name
- version ('1.1' by default): GELF protocol version, can be overridden by a client
- extra_fields (False by default): if true, each log message will include all the extra fields set to LogRecord
In addition UDP and TLS handlers have some specific parameters.
UDP:
- chunk_size (1300 by default) - maximum length of the message. If log length exceeds this value, it splits into multiple chunks (see https://www.graylog.org/resources/gelf/ section "chunked GELF") with the length equals to this value. This parameter must be less than the MTU. If the logs don't seem to be delivered, try to reduce this value.
- compress (true by default): if true, compress log messages before send them to the server
TLS:
- validate (false by default) - if true, validate server certificate. If server provides a certificate that doesn't exist in ca_certs, you won't be able to send logs over TLS
- ca_certs (none by default) - path to CA bundle file. This parameter is required if validate is true.
If you need to include some static fields into your logs, simply pass them to the handler constructor. Each additional field shoud start with underscore. You can't add field '_id'.
Example:
handler = GelfUdpHandler(host='127.0.0.1', port=9402, _app_name='pygelf', _something=11)
logger.addHandler(handler)
If you need to include some dynamic fields into your logs, add them to record by using LoggingAdapter or logging.Filter and create handler with extra_fields set to True. All the non-trivial fields of the record will be sent to graylog2 with '_' added before the name
Example:
class ContextFilter(logging.Filter): def filter(self, record): record.job_id = threading.local().process_id # logging job_id of currently processed job return True logger.addFilter(ContextFilter()) handler = GelfUdpHandler(host='127.0.0.1', port=9402, extra_fields=True) logger.addHandler(handler)
To run tests, you'll need tox. After installing, simply run it:
tox
You can also specify interpreter version. For example:
tox -e py26 tox -e py35