Switch to use SSLContext

[berkerpeksag]

We need to use SSLContext in Python 3 (and perhaps 2.7.9+ too) to make Gunicorn users like easier. The old API is buggy and causes weird edge case bugs like reported in #1135.

[benoitc]

@berkerpeksag wanna want to do it? :)

[berkerpeksag]

Will do, but I'm busy with job interviews these days so it can take a while :)

[tilgovi]

Here's a diff that begins this work for anyone who wants to get started on this:

diff --git a/gunicorn/workers/sync.py b/gunicorn/workers/sync.py
index 15ac084..5fd5730 100644
--- a/gunicorn/workers/sync.py
+++ b/gunicorn/workers/sync.py
@@ -122,8 +122,16 @@ class SyncWorker(base.Worker):
         req = None
         try:
             if self.cfg.is_ssl:
-                client = ssl.wrap_socket(client, server_side=True,
-                    **self.cfg.ssl_options)
+                context = ssl.SSLContext(self.cfg.ssl_options['ssl_version'])
+                context.set_default_verify_paths()
+                #context.load_verify_locations(self.cfg.ssl_options['ca_certs'])
+                context.load_cert_chain(
+                    self.cfg.ssl_options['certfile'],
+                    keyfile=self.cfg.ssl_options['keyfile'])
+                context.set_ciphers(self.cfg.ssl_options['ciphers'])
+                context.options |= ssl.OP_CIPHER_SERVER_PREFERENCE
+                context.verify_mode = self.cfg.ssl_options['cert_reqs']
+                client = context.wrap_socket(client, server_side=True)

             parser = http.RequestParser(self.cfg, client)
             req = six.next(parser)

The ssl.OP_CIPHER_SERVER_PREFERENCE should probably be made a configuration option. I'm also not sure we should set the default verify paths or not and I have commented out the load_verify_locations call because I didn't fully digest the parameters here and wanted to be sure I was loading the CA certificates for client authentication. This was a rush, rough draft I did one evening last weekend just to start getting familiar with the ssl module.

We can also probably do this somewhere else, maybe in config.py, so that the SSLContext setup code can be shared by all the workers.

[benoitc]

SSLContext is available to python 2.7:

https://docs.python.org/2/library/ssl.html?highlight=sslcontext#ssl.SSLContext

[jamadden]

SSLContext is available to python 2.7

To share some experience from gevent, where this has been somewhat painful:

SSLContext is available in Python 2.7.9 and above, or where backported by vendors (RedHat has backported as far back as 2.7.5). But it's not generally available in Python 2.7 as such. The generic version of 'pypy' that you get on Travis for example is pypy-2.5 which implements Python 2.7.8 and doesn't have it. gevent (still) has users that are on 2.7.9+, <=2.7.8, and using vendor backports.

The bad thing about this is that it can lead to the python process actually crashing if you mix and match things in the wrong way.

[dumblob]

I'm using gunicorn (with Flask) for a smaller application and need to add client verification for just few URLs (not the whole application) while having a different client certificate for each client.

Apart from the fact, that gunicorn only seems to support authentication of all users or none disregarding URL they're requesting and assuming that each client has the same client certificate, it also seems to have difficulties araising from not using SSLContext (see e.g. https://stackoverflow.com/questions/53730374/gunicorn-two-way-ssl-error-ssl-error-unknown-ca-alert which seems to have a copy also here).

@tilgovi , @benoitc would you have any idea how to use SSLContext in a way to allow the client verification only for certain URL routes and having more client certificates (each client its own)?

[tilgovi]

I don't know that Gunicorn could provide route-specific authentication, though I may not have a good picture of how SSL client authentication works (now, or in the future with SSLContext). I expect the authentication would be on the SSL handshake, before any HTTP protocol request is read.

However, that does not mean your application needs to authorize access to routes without regard to the details of the authenticated client. Gunicorn would have to expose the peer certificate, following something like shown here: https://www.ajg.id.au/2018/01/01/mutual-tls-with-python-flask-and-werkzeug/

[dumblob]

Thank you, I'll take a look at it. If it won't work for my case, I might in the end change my mind and lean towards an application layer authentication providing a one-way verification (the server got verified by the client) was successful and there is an encrypted channel to communicate through.

[dumblob]

In the end I rather solved authentication in the application layer to ease maintenance.

There is one more thing I couldn't figure out with gunicorn. Is there a way how to exchange certificates (e.g. those passed to gunicorn through --keyfile --certfile --ca-certs) without restarting gunicorn? If not, will the new SSLContext make it somehow possible?

Use case is simple - using short-term certificates (e.g. Let's Encrypt) while maintaining "higher availability".

[pipeti]

An alternative solution for this could be instead of adding way too many ssl options and take the risk of changes in the python libraries, offer a server hook for the user to provide an SSL context.
Then gunicorn could use that instance to wrap socket and leaves all the configuration options (and responsibility) at the user who could consider an adapt the code to the used python version.
The hook could be invoked e.g. at server startup or preferably before handling a request. As e.g. the certificates might change, loading them once might be not good enough should they expire.

[tsaarni]

@benoitc, @berkerpeksag I can work with this issue if you don't mind?

[uedvt359]

Another use case for this (which we need at $work): gunicorn doesn't allow disabling of TLS session renegotiation, a potential DoS vector. With access to the SSLContext, one could disable renegotiation and avoid this vuln.