Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support tlsmode #555

Merged
merged 22 commits into from
Jul 19, 2024
Merged

Support tlsmode #555

Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
b01ae58
init
sitingren Jun 17, 2024
9b6fcc6
Update README.md
sitingren Jun 17, 2024
8c4f7e1
add tlsmode.py
sitingren Jun 18, 2024
b425471
update
sitingren Jun 18, 2024
b6f798d
update test
sitingren Jun 18, 2024
71579df
update prefer tls
sitingren Jun 18, 2024
90ebef8
fix test
sitingren Jun 19, 2024
08a4b33
update test
sitingren Jun 19, 2024
662ffc4
update test
sitingren Jun 19, 2024
8fba157
update
sitingren Jun 21, 2024
8597d70
test_TLSMode_verify_ca
sitingren Jun 21, 2024
cf46418
test_TLSMode_verify_ca
sitingren Jun 21, 2024
072b808
fix typo
sitingren Jun 21, 2024
ba67fea
client cert and key
sitingren Jun 24, 2024
0cbeabc
add test
sitingren Jun 24, 2024
d8616ca
update test
sitingren Jun 24, 2024
4fd13dc
add warning
sitingren Jun 24, 2024
6f8479d
update readme
sitingren Jun 24, 2024
c07c4bd
Update readme
sitingren Jun 24, 2024
b124af1
update test
sitingren Jun 26, 2024
82da3a8
Update README
sitingren Jul 4, 2024
c1df49d
wrap function
sitingren Jul 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 51 additions & 12 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,11 @@ with vertica_python.connect(**conn_info) as connection:
| oauth_access_token | See [OAuth Authentication](#oauth-authentication). <br>**_Default_**: "" |
| request_complex_types | See [SQL Data conversion to Python objects](#sql-data-conversion-to-python-objects). <br>**_Default_**: True |
| session_label | Sets a label for the connection on the server. This value appears in the client_label column of the _v_monitor.sessions_ system table. <br>**_Default_**: an auto-generated label with format of `vertica-python-{version}-{random_uuid}` |
| ssl | See [TLS/SSL](#tlsssl). <br>**_Default_**: False (disabled) |
| ssl | See [TLS/SSL](#tlsssl). <br>**_Default_**: None (tlsmode="prefer") |
| tlsmode | Controls whether the connection to the server uses TLS encryption. <br>See [TLS/SSL](#tlsssl). <br>**_Default_**: "prefer" |
| tls_cafile | The name of a file containing SSL certificate authority (CA) certificate(s). <br>See [TLS/SSL](#tlsssl). |
sitingren marked this conversation as resolved.
Show resolved Hide resolved
| tls_certfile | The name of a file containing client's certificate(s). <br>See [TLS/SSL](#tlsssl). |
| tls_keyfile | The name of a file containing client's private key. <br>See [TLS/SSL](#tlsssl). |
| unicode_error | See [UTF-8 encoding issues](#utf-8-encoding-issues). <br>**_Default_**: 'strict' (throw error on invalid UTF-8 results) |
| use_prepared_statements | See [Passing parameters to SQL queries](#passing-parameters-to-sql-queries). <br>**_Default_**: False |
| workload | Sets the workload name associated with this session. Valid values are workload names that already exist in a workload routing rule on the server. If a workload name that doesn't exist is entered, the server will reject it and it will be set to the default. <br>**_Default_**: "" |
Expand Down Expand Up @@ -141,22 +145,61 @@ with vertica_python.connect(dsn=connection_str, **additional_info) as conn:
```

#### TLS/SSL
You can pass `True` to `ssl` to enable TLS/SSL connection (equivalent to TLSMode=require).

There are two options to control client-server TLS: `tlsmode` and `ssl`. If both are set, `tlsmode` takes precedence.

`ssl` can be a bool or a `ssl.SSLContext` object. Here is the value mapping between `ssl` (exclude `ssl.SSLContext`) and `tlsmode`:

| `tlsmode` | `ssl` | Description |
| ------------- | ------------- | ---|
| 'disable' | False | only try a non-TLS connection. |
| 'prefer' | (not set) | (Default) first try a TLS connection; if TLS is disabled on the server, then fallback to a non-TLS connection. <br>Note: If TLS is enabled on the server and TLS connection fails, the client rejects the connection. |
| 'require' | True | connects using TLS without verifying certificates. If the TLS connection attempt fails, the client rejects the connection. |
| 'verify-ca' || connects using TLS and confirms that the server certificate has been signed by the certificate authority. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar nit, I would change the description slightly to say "...signed by a trusted certificate authority."

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and same change to the relevant part of verify-full

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

| 'verify-full' || connects using TLS, confirms that the server certificate has been signed by the certificate authority, and verifies that the host name matches the name provided in the server certificate. |

When `tlsmode` is 'verify-ca' or 'verify-full', these options take certificate/key files: `tls_cafile`, `tls_certfile` and `tls_keyfile`. Otherwise, these options are ignored.

`tlsmode` example:
```python
# [TLSMode: require]
import vertica_python
conn_info = {'host': '127.0.0.1',
'user': 'some_user',
'database': 'a_database',
'tlsmode': 'require'}
connection = vertica_python.connect(**conn_info)
```
```python
# [TLSMode: verify-ca]
import vertica_python
conn_info = {'host': '127.0.0.1',
'user': 'some_user',
'database': 'a_database',
'tlsmode': 'verify-ca',
'tls_cafile': '/path/to/ca_file.pem' # CA certificate used to verify server certificate
}
connection = vertica_python.connect(**conn_info)
```

```python
# [TLSMode: verify-full] + Mutual Mode
import vertica_python

# [TLSMode: require]
conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'some_user',
'password': 'some_password',
'database': 'a_database',
'ssl': True}
'tlsmode': 'verify-full',
'tls_cafile' = '/path/to/ca_file.pem' # CA certificate used to verify server certificate
'tls_certfile' = '/path/to/client.pem', # (for mutual mode) client certificate
'tls_keyfile' = '/path/to/client.key' # (for mutual mode) client private key
}
connection = vertica_python.connect(**conn_info)
```

You can pass an `ssl.SSLContext` to `ssl` to customize the SSL connection options. Server mode TLS examples:
You can pass an `ssl.SSLContext` object to `ssl` to customize the underlying SSL connection options. See more on SSL options [here](https://docs.python.org/3/library/ssl.html).

Server mode TLS examples:

```python
import vertica_python
Expand All @@ -171,7 +214,6 @@ ssl_context.verify_mode = ssl.CERT_NONE
conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'some_user',
'password': 'some_password',
'database': 'a_database',
'ssl': ssl_context}
connection = vertica_python.connect(**conn_info)
Expand All @@ -187,7 +229,6 @@ ssl_context.load_verify_locations(cafile='/path/to/ca_file.pem') # CA certificat
conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'some_user',
'password': 'some_password',
'database': 'a_database',
'ssl': ssl_context}
connection = vertica_python.connect(**conn_info)
Expand All @@ -204,7 +245,6 @@ ssl_context.load_verify_locations(cafile='/path/to/ca_file.pem') # CA certificat
conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'some_user',
'password': 'some_password',
'database': 'a_database',
'ssl': ssl_context}
connection = vertica_python.connect(**conn_info)
Expand All @@ -230,13 +270,12 @@ ssl_context.load_cert_chain(certfile='/path/to/client.pem', keyfile='/path/to/cl
conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'some_user',
'password': 'some_password',
'database': 'a_database',
'ssl': ssl_context}
connection = vertica_python.connect(**conn_info)
```

See more on SSL options [here](https://docs.python.org/3/library/ssl.html).


#### Kerberos Authentication
In order to use Kerberos authentication, install [dependencies](#using-kerberos-authentication) first, and it is the user's responsibility to ensure that an Ticket-Granting Ticket (TGT) is available and valid. Whether a TGT is available can be easily determined by running the `klist` command. If no TGT is available, then it first must be obtained by running the `kinit` command or by logging in. You can pass in optional arguments to customize the authentication. The arguments are `kerberos_service_name`, which defaults to "vertica", and `kerberos_host_name`, which defaults to the value of argument `host`. For example,
Expand Down
169 changes: 151 additions & 18 deletions vertica_python/tests/integration_tests/test_tls.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,12 @@


class TlsTestCase(VerticaPythonIntegrationTestCase):
SSL_STATE_SQL = 'SELECT ssl_state FROM sessions WHERE session_id=current_session()'

def tearDown(self):
if 'ssl' in self._conn_info:
del self._conn_info['ssl']
# Use a non-TLS connection here so cleanup can happen
# even if mutual mode is configured
self._conn_info['tlsmode'] = 'disable'
with self._connect() as conn:
cur = conn.cursor()
cur.execute("ALTER TLS CONFIGURATION server CERTIFICATE NULL TLSMODE 'DISABLE'")
Expand All @@ -36,8 +39,14 @@ def tearDown(self):
if hasattr(self, 'client_key'):
os.remove(self.client_key.name)
cur.execute("DROP KEY IF EXISTS vp_client_key CASCADE")
if hasattr(self, 'CA_cert'):
os.remove(self.CA_cert.name)
cur.execute("DROP KEY IF EXISTS vp_server_key CASCADE")
cur.execute("DROP KEY IF EXISTS vp_CA_key CASCADE")

for key in ('tlsmode', 'ssl', 'tls_cafile', 'tls_certfile', 'tls_keyfile'):
if key in self._conn_info:
del self._conn_info[key]
super(TlsTestCase, self).tearDown()

def _generate_and_set_certificates(self, mutual_mode=False):
Expand All @@ -52,6 +61,8 @@ def _generate_and_set_certificates(self, mutual_mode=False):
"VALID FOR 3650 EXTENSIONS 'nsComment' = 'Self-signed root CA cert' KEY vp_CA_key")
cur.execute("SELECT certificate_text FROM CERTIFICATES WHERE name='vp_CA_cert'")
vp_CA_cert = cur.fetchone()[0]
with NamedTemporaryFile(delete=False) as self.CA_cert:
self.CA_cert.write(vp_CA_cert.encode())

# Generate a server private key
cur.execute("CREATE KEY vp_server_key TYPE 'RSA' LENGTH 4096")
Expand Down Expand Up @@ -84,7 +95,7 @@ def _generate_and_set_certificates(self, mutual_mode=False):
# This CA certificate is used to verify client certificates
cur.execute('ALTER TLS CONFIGURATION server CERTIFICATE vp_server_cert ADD CA CERTIFICATES vp_CA_cert')
# Enable TLS. Connection succeeds if Vertica verifies that the client certificate is from a trusted CA.
# If the client does not present a client certificate, the connection uses plaintext.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good adjustment, more secure

# If the client does not present a client certificate, the connection is rejected.
cur.execute("ALTER TLS CONFIGURATION server TLSMODE 'VERIFY_CA'")

else:
Expand All @@ -100,42 +111,164 @@ def _generate_and_set_certificates(self, mutual_mode=False):

return vp_CA_cert

######################################################
#### Test 'ssl' and 'tlsmode' options are not set ####
######################################################

def test_TLSMode_disable(self):
def test_option_default_server_disable(self):
# TLS is disabled on the server
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'None')

def test_option_default_server_enable(self):
# Setting certificates with TLS configuration
self._generate_and_set_certificates()

# TLS is enabled on the server
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Server')

#######################################################
#### Test 'ssl' and 'tlsmode' options are both set ####
#######################################################

def test_tlsmode_over_ssl(self):
self._conn_info['tlsmode'] = 'disable'
self._conn_info['ssl'] = True
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'None')

###############################################
#### Test 'ssl' option with boolean values ####
###############################################

def test_ssl_false(self):
self._conn_info['ssl'] = False
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone('SELECT ssl_state FROM sessions WHERE session_id=(SELECT current_session())')
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'None')

def test_TLSMode_require_server_disable(self):
def test_ssl_true_server_disable(self):
# Requires that the server use TLS. If the TLS connection attempt fails, the client rejects the connection.
self._conn_info['ssl'] = True
self.assertConnectionFail(err_type=errors.SSLNotSupported,
err_msg='SSL requested but not supported by server')
err_msg='SSL requested but disabled on the server')

def test_TLSMode_require(self):
def test_ssl_true_server_enable(self):
# Setting certificates with TLS configuration
self._generate_and_set_certificates()

# Option 1
self._conn_info['ssl'] = True
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone('SELECT ssl_state FROM sessions WHERE session_id=(SELECT current_session())')
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Server')

###############################
#### Test 'tlsmode' option ####
###############################

def test_TLSMode_disable(self):
self._conn_info['tlsmode'] = 'disable'
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'None')

def test_TLSMode_prefer_server_disable(self):
# TLS is disabled on the server
self._conn_info['tlsmode'] = 'prefer'
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'None')

def test_TLSMode_prefer_server_enable(self):
# Setting certificates with TLS configuration
self._generate_and_set_certificates()

self._conn_info['tlsmode'] = 'prefer'
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Server')

# Option 2
def test_TLSMode_require_server_disable(self):
# Requires that the server use TLS. If the TLS connection attempt fails, the client rejects the connection.
self._conn_info['tlsmode'] = 'require'
self.assertConnectionFail(err_type=errors.SSLNotSupported,
err_msg='SSL requested but disabled on the server')

def test_TLSMode_require_server_enable(self):
# Setting certificates with TLS configuration
self._generate_and_set_certificates()

self._conn_info['tlsmode'] = 'require'
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Server')

def test_TLSMode_verify_ca(self):
# Setting certificates with TLS configuration
CA_cert = self._generate_and_set_certificates()

self._conn_info['tlsmode'] = 'verify-ca'
self._conn_info['tls_cafile'] = self.CA_cert.name
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Server')

def test_TLSMode_verify_full(self):
# Setting certificates with TLS configuration
CA_cert = self._generate_and_set_certificates()

self._conn_info['tlsmode'] = 'verify-full'
self._conn_info['tls_cafile'] = self.CA_cert.name
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Server')

def test_TLSMode_mutual_TLS(self):
# Setting certificates with TLS configuration
CA_cert = self._generate_and_set_certificates(mutual_mode=True)

self._conn_info['tlsmode'] = 'verify-full'
self._conn_info['tls_cafile'] = self.CA_cert.name # CA certificate used to verify server certificate
self._conn_info['tls_certfile'] = self.client_cert.name # client certificate
self._conn_info['tls_keyfile'] = self.client_key.name # private key used for the client certificate
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Mutual')

######################################################
#### Test 'ssl' option with ssl.SSLContext object ####
######################################################

def test_sslcontext_require(self):
# Setting certificates with TLS configuration
self._generate_and_set_certificates()

ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
self._conn_info['ssl'] = ssl_context
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone('SELECT ssl_state FROM sessions WHERE session_id=(SELECT current_session())')
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Server')

def test_TLSMode_verify_ca(self):
def test_sslcontext_verify_ca(self):
# Setting certificates with TLS configuration
CA_cert = self._generate_and_set_certificates()

Expand All @@ -147,10 +280,10 @@ def test_TLSMode_verify_ca(self):

with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone('SELECT ssl_state FROM sessions WHERE session_id=(SELECT current_session())')
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Server')

def test_TLSMode_verify_full(self):
def test_sslcontext_verify_full(self):
# Setting certificates with TLS configuration
CA_cert = self._generate_and_set_certificates()

Expand All @@ -162,10 +295,10 @@ def test_TLSMode_verify_full(self):
self._conn_info['ssl'] = ssl_context
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone('SELECT ssl_state FROM sessions WHERE session_id=(SELECT current_session())')
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Server')

def test_mutual_TLS(self):
def test_sslcontext_mutual_TLS(self):
# Setting certificates with TLS configuration
CA_cert = self._generate_and_set_certificates(mutual_mode=True)

Expand All @@ -178,7 +311,7 @@ def test_mutual_TLS(self):
self._conn_info['ssl'] = ssl_context
with self._connect() as conn:
cur = conn.cursor()
res = self._query_and_fetchone('SELECT ssl_state FROM sessions WHERE session_id=(SELECT current_session())')
res = self._query_and_fetchone(self.SSL_STATE_SQL)
self.assertEqual(res[0], 'Mutual')


Expand Down
8 changes: 7 additions & 1 deletion vertica_python/tests/unit_tests/test_parsedsn.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,20 @@ def test_str_arguments(self):
'session_label=vpclient&unicode_error=strict&'
'log_path=/home/admin/vClient.log&log_level=DEBUG&'
'oauth_access_token=GciOiJSUzI1NiI&'
'workload=python_test_workload&'
'workload=python_test_workload&tlsmode=verify-ca&'
'tls_cafile=tls/ca_cert.pem&tls_certfile=tls/cert.pem&'
'tls_keyfile=tls/key.pem&'
'kerberos_service_name=krb_service&kerberos_host_name=krb_host')
expected = {'database': 'db1', 'host': 'localhost', 'user': 'john',
'password': 'pwd', 'port': 5433, 'log_level': 'DEBUG',
'session_label': 'vpclient', 'unicode_error': 'strict',
'log_path': '/home/admin/vClient.log',
'oauth_access_token': 'GciOiJSUzI1NiI',
'workload': 'python_test_workload',
'tlsmode': 'verify-ca',
'tls_cafile': 'tls/ca_cert.pem',
'tls_certfile': 'tls/cert.pem',
'tls_keyfile': 'tls/key.pem',
'kerberos_service_name': 'krb_service',
'kerberos_host_name': 'krb_host'}
parsed = parse_dsn(dsn)
Expand Down
Loading