diff --git a/src/conf.py b/src/conf.py index 8d8dea85..2fda1859 100644 --- a/src/conf.py +++ b/src/conf.py @@ -83,7 +83,8 @@ 'auspice': ('https://docs.nextstrain.org/projects/auspice/page/', None), 'cli': ('https://docs.nextstrain.org/projects/cli/page/', None), 'nextclade': ('https://docs.nextstrain.org/projects/nextclade/page/', None), - 'ncov': ('https://docs.nextstrain.org/projects/ncov/page/', None) + 'ncov': ('https://docs.nextstrain.org/projects/ncov/page/', None), + 'python': ('https://docs.python.org/3/', None), } diff --git a/src/index.rst b/src/index.rst index fb708f02..790eb629 100644 --- a/src/index.rst +++ b/src/index.rst @@ -96,6 +96,7 @@ team and other Nextstrain users provide assistance. For private inquiries, reference/data-formats reference/data-files FAQ + reference/ca-certificates reference/snakemake-style-guide reference/documentation-style-guide reference/governance diff --git a/src/reference/ca-certificates.rst b/src/reference/ca-certificates.rst new file mode 100644 index 00000000..086e37e4 --- /dev/null +++ b/src/reference/ca-certificates.rst @@ -0,0 +1,248 @@ +=========================== +CA certificate trust stores +=========================== + +Secure network connections using TLS (or SSL), e.g. ``https://`` URLs, are +based on *certificates* issued to server operators by *certificate authorities* +(CAs). Each program which makes these secure connections ultimately needs to +have a list of CAs that are to be trusted and be able to trace the issuance of +a server's certificate back to a trusted CA's root certificate. + +This set of trusted CA root certificates is called a *trust store*. Every +computer system has its own OS-level trust store and usually also has several +other application-specific trust stores on the system. Typically these are +automatically managed and do not need configuration or modification. + +When your network environment includes a proxy, egress firewall, or +institutional CA used internally, however, it may be necessary to configure the +trust store used by an application or modify the trust store contents (e.g. +adding a institutional CA certificate). + +This document focuses on configuration and modification of trust stores used by +software that is in turn used by Nextstrain's software. It focuses only on +changes that can be made without modification of source or program code, e.g. +thru environment variables or config files. + +.. contents:: + :local: + :depth: 3 + + +Operating system +================ + +Linux uses :ref:`OpenSSL ` for its system-wide trust store. macOS and +Windows use their own platform-specific stores, Keychain (managed via +Keychain.app) and Certificate Store (managed via ``certmgr.msc``), +respectively. OpenSSL and some other software on macOS and Windows systems +(e.g. :ref:`curl`) is sometimes configured by default to use those +platform-specific stores, but not always. + +If you need to include an additional CA certificate in the trust store and that +certificate applies to *all* software on the computer (e.g. your web browser), +not just Nextstrain's, then consider adding the CA certificate to the +system-wide trust store and configuring Nextstrain's software to use the +system's trust store instead of any application-specific ones when necessary. +This blanket approach is generally more reliable over time than +application-by-application configuration. + + +Application software +==================== + +Most software provides a way to override the trust store by providing a path to +a file containing the the entire set of CA certificates to trust. Thus, if you +only need to include an additional CA certificate in the trust store, you +should ensure that the file you provide contains a standard set of CA +certificates as well as your addition. An exception here is if you know for +certain that all connections will use your CA certificate and only yours (e.g. +a mandatory egress proxy that removes (and adds back) TLS for inspection and +policy purposes). + +.. _openssl: + +OpenSSL library +--------------- + +OpenSSL is the most common library used to provide TLS/SSL support in +application software. Its `default locations of trusted CA certificates +`__ can be +overridden by setting the ``SSL_CERT_FILE`` and/or ``SSL_CERT_DIR`` environment +variables. + +Its final trust store is built from certificates in all default locations, so +to *comprehensively* override the defaults, all locations must be overridden. +This is typically unnecessary unless you need to ensure some CAs in the +defaults *aren't* trusted. + +.. note:: + Each Conda environment or Docker container has its own isolated copy of + OpenSSL and its own isolated default CA certificate trust store separate + from any OS-level OpenSSL trust store. + + +.. _node.js: + +Node.js +------- + +*Used by nextstrain.org and Auspice's standalone server component.* + +Node.js may use either its own bundled snapshot of `Mozilla's CA trust store`_ +or the :ref:`OpenSSL ` default CA trust store, with the default choice +depending on how it was compiled. + +This typically means that ``node`` from your official OS package repositories +defaults to the OpenSSL default CA trust store while ``node`` from `nodejs.org +`__ (e.g. via ``nvm``), `conda-forge +`__, and other third-party places +defaults to the bundled CA trust store. You can be explicit about the +preferred CA trust store by including |--use-openssl-ca|_ or +|--use-bundled-ca|_ in the |NODE_OPTIONS| environment variable or by passing +those options directly in your ``node`` invocation. + +To override the trust store, set the |NODE_OPTIONS|_ environment variable to +include ``--use-openssl-ca`` and then set OpenSSL's ``SSL_CERT_FILE`` or +``SSL_CERT_DIR`` environment variables. + +To add CA certificates to the trust store, set the |NODE_EXTRA_CA_CERTS|_ +environment variable. + +.. |--use-openssl-ca| replace:: ``--use-openssl-ca`` +.. _--use-openssl-ca: https://nodejs.org/api/cli.html#--use-bundled-ca---use-openssl-ca + +.. |--use-bundled-ca| replace:: ``--use-bundled-ca`` +.. _--use-bundled-ca: https://nodejs.org/api/cli.html#--use-bundled-ca---use-openssl-ca + +.. |NODE_OPTIONS| replace:: ``NODE_OPTIONS`` +.. _NODE_OPTIONS: https://nodejs.org/api/cli.html#node_optionsoptions + +.. |NODE_EXTRA_CA_CERTS| replace:: ``NODE_EXTRA_CA_CERTS`` +.. _NODE_EXTRA_CA_CERTS: https://nodejs.org/api/cli.html#node_extra_ca_certsfile + + +.. _python: + +Python +------ + +Uses the :ref:`OpenSSL library ` and its defaults. See +:py:meth:`python:ssl.SSLContext.load_default_certs` and +:py:meth:`python:ssl.SSLContext.set_default_verify_paths`. + + +.. _python-requests: + +``requests`` module +~~~~~~~~~~~~~~~~~~~ + +*Used by Nextstrain CLI and some pathogen workflows.* + +Uses a snapshot of `Mozilla's CA trust store`_ via the |certifi|_ Python +package. + +Set the |REQUESTS_CA_BUNDLE|_ environment variable to override. + +.. note:: + If a ``requests``-specific CA bundle isn't configured, ``requests`` falls + back to the |CURL_CA_BUNDLE|_ environment variable (if set). + +.. |REQUESTS_CA_BUNDLE| replace:: ``REQUESTS_CA_BUNDLE`` +.. _REQUESTS_CA_BUNDLE: https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification + + +.. _nextclade-cli: + +Nextclade CLI +------------- + +*Applies to Nextclade v3.* + +Uses its own bundled snapshot of `Mozilla's CA trust store`_ via the +|webpki-roots|_ Rust crate (by way of the ``reqwest`` crate's +|rustls-tls-webpki-roots feature|_). + +There is currently no way to configure or modify the trust store without +modifying the Nextclade source code. + +.. I have a fix in-flight for ↑ that. —trs, 10 Oct 2024 + +.. |webpki-roots| replace:: ``webpki-roots`` +.. _webpki-roots: https://docs.rs/webpki-roots/0.26.6/webpki_roots/ + +.. |rustls-tls-webpki-roots feature| replace:: ``rustls-tls-webpki-roots`` feature +.. _rustls-tls-webpki-roots feature: https://docs.rs/reqwest/0.12.8/reqwest/#optional-features + + + +.. _aws-cli: + +AWS CLI +------- + +*Used by some pathogen workflows.* + +The AWS CLI (v1 and v2), via `Botocore`_, uses a snapshot of `Mozilla's CA +trust store`_ via the |certifi|_ Python package when it's available, otherwise +it falls back to `its own bundled CA trust store `_. + +Set the |AWS_CA_BUNDLE|_ environment variable or |ca_bundle|_ profile +configuration to override. + +.. note:: + If an AWS-specific CA bundle isn't configured, the AWS CLI falls back to the + |REQUESTS_CA_BUNDLE|_ environment variable (if set). + +.. _aws-sdks: + +AWS SDKs +-------- + +*Used by Nextstrain CLI and nextstrain.org.* + +The AWS SDKs for JavaScript (v2 and v3) default to the :ref:`Node.js ` +trust store. + +Set the |AWS_CA_BUNDLE|_ environment variable or |ca_bundle|_ profile +configuration to override. + +.. _Botocore: https://pypi.org/project/botocore/ +.. _Mozilla's CA trust store: https://wiki.mozilla.org/CA/Included_Certificates +.. _botocore-cacerts: https://github.com/boto/botocore/blob/master/botocore/cacert.pem + +.. |certifi| replace:: ``certifi`` +.. _certifi: https://pypi.org/project/certifi/ + +.. |AWS_CA_BUNDLE| replace:: ``AWS_CA_BUNDLE`` +.. _AWS_CA_BUNDLE: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html#envvars-list-AWS_CA_BUNDLE + +.. |ca_bundle| replace:: ``ca_bundle`` +.. _ca_bundle: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html#cli-config-ca_bundle + + +.. _curl: + +curl +---- + +*Used by many pathogen workflows.* + +Curl can come with support for many different TLS implementations, which means +how to configure or modify the CA trust store in use can vary between copies of +``curl``, even when they're the same version. Refer to the `curl book`_ and +`curl docs`_ for more details about which trust stores are used when and +configuration basics. + +Set the |CURL_CA_BUNDLE|_ environment variable to override. + +.. note:: + Curl also respects the ``SSL_CERT_FILE`` and ``SSL_CERT_DIR`` environment + variables, even if the TLS implementation in use is not OpenSSL (with one + exception). This is handy if you're setting those already, as you can avoid + setting ``CURL_CA_BUNDLE`` too. + +.. _curl book: https://everything.curl.dev/usingcurl/tls/verify.html +.. _curl docs: https://curl.se/docs/sslcerts.html + +.. |CURL_CA_BUNDLE| replace:: ``CURL_CA_BUNDLE`` +.. _CURL_CA_BUNDLE: https://curl.se/docs/manpage.html#CURLCABUNDLE