- HashiCorp Vault PKI with Raft backend
- Introduction
- Purpose
- Prerequisites
- Quick Start
- Quick Start Explanation - short version
- Quick Start Explanation - long version
- Creating custom leaf certificates
- Web Browswer Testing with Docker
- Commands
- Revocation
- Vault Configuration
- Security Concerns
- Style Convention
- Why the name raft.sh?
- Improvements
- References
This repo demonstrates HashiCorp's Vault product as a Certificate Authority (CA) for Public Key Infrastructure (PKI).
Following the Quick Start below, you're able to quickly genearate self-signed certificates for learning or deployment in test environments.
This demo utilizes a raft as a storage backend as opposed to a file backend which is typical in many demos/tutorials. Deploying Vault with raft backend allows for simple backup and recovery of data via built-in commands. While there are other features with a raft deployment, convienent backup and restoration were significant factors for using raft as a backend. Technically, raft is a consensous algorithm, but that is a digression.
The primary purpose is to provide a development/test environment to learn PKI with HashiCorp Vault.
This repo (platform) could potentially be used for personal/private networks, if you accept the shortcomings. The limitations are the lack comprehensive features, security best practices and proper deployment of HashiCorp raft backend infrastructure.
- Developed on Ubuntu Linux 22.04 LTS.
- Tested with Bash Shell
Naturally this repo will work with other *nix Operating Systems and/or Shells with modification.
- HashiCorp Vault Note: Preferred to install Vault from Hashicorp, while Vault can be installed via Snap package, the Snap version does not include a GUI console.
- Jq
- OpenSSL
This enables convenient copy and paste of root token to login to Vault. (either CLI and/or GUI)
-
Basic understanding of TLS certificates. If knowledge is limited; this platform is great to learn and play with TLS certificates and Certificate Authority (CA)
-
Basic understanding of HashiCorp Vault.
-
Basic knowledge of Linux command line and BASH scripting.
Clone the Repo:
git clone https://github.com/richlamdev/vault-pki-raft.git
cd vault-pki-raft
Steps:
./raft.sh start
./create_root_inter_certs.sh
./issue_cert_template.sh
-
Creates a Certificate Authority from a single Vault instance.
-
Generates a signed leaf certificate, template.middleearth.test. This certificate can be deployed to a web server. Naturally, this is just a demo, read below for instruction on how to modify the output leaf certificate.
-
Optional: Import the root certificate generated by Vault, to your browser to allow the browser to trust the leaf certifcate presented.
The above will perform the following:
-
Deploys a single Vault instance with a raft backend. - [raft.sh]
a. This will unseal the vault and login in the current terminal (user) as the root user.
b. This will save the unseal key in the file unseal_key and root token in the file root_token in the current folder. Naturally, this not the most secure method to save the unseal_key and root_token. Actually, a single unseal_key is not the best method for provisioning. See Security Concerns section for more information.
c. After this is complete, the root token is saved to the system buffer. This enables convenient login to the vault GUI console. Visit
http://127.0.0.1:8200/via your browser. Choose Token method to login and paste the root token in the Token field to login. This step is optional, naturally the GUI provides a convenient way to explore the certificates and/or actions as resuilt of the CLI commands. -
Enables Vault PKI Engine and creates a CA - [create_root_inter_certs.sh]
a. Creates a root certificate and self sign the certificate. The root CA is designated by the variable ISSUER_NAME_CN. By default the ISSUER_NAME_CN is "Lord of the Rings". Change this value if you like.
b. Creates an intermediate certificate signing request, have the root authority sign this certificate and store it within the CA.
c. Creates a role designated by the variable VAULT_ROLE to sign leaf certificates. Note the value of VAULT_ROLE, itself, is not critical. However, the VAULT_ROLE value must be the same in both files, create_root_inter_certs.sh and issue_cert_template.sh. This role name is referenced (used) to sign leaf certificates. If they do not match, an error will occur. Change this value if you like, however keep them consistent. VAULT_ROLE is authorized to sign subdomains indicated by the variable DOMAIN, in this case the Second Level Domain (SLD) and Top Level Domain(TLD), the default value is "middleearth.test". Again, change this value as you like.
d. The self-signed CA root certificate and intermediate certificate chain are stored in the directory as designated by the variable $ROOT_INTER_DIR. The directory default is ./root_inter_certs. Import the root certificate from this folder to your Operating System Trusted Store or Web Browser. If you're unaware how to import the root certificate to either, a quick google search will help you.
-
Issues a "template" certificate with a default Subject Common Name (CN) template.middleearth.test - [issue_cert_template.sh]
a. The resulting public certificate, key file, as well as entire signed json blob is stored in directory designated by the variable SUBJECT_CN. Edit the HOST and DOMAIN variables to change the default value of SUBJECT_CN. Ensure the value of DOMAIN is the same in both files, create_root_inter_certs.sh and issue_cert_template.sh. In this example, the resulting certificate files will be stored in the directory template.middleearth.test.
Inspecting template.middleearth.test certificate via openssl command:
Optionally deploy the template certificate to a web server for inspection via
web browser. The below is taken from Nginx in a Ubuntu Virtual Machine
(VM). Naturally, alternatives would achieve similar results, such as Docker
with Apache or Nginx, Windows & IIS etc. Refer to Web Browser Testing with
Docker section. The certificate inpsected via Firefox browser:
If you import the root certificate to your trusted store or browser update your local DNS (or update local /etc/hosts file) to resolve template.middleaearth.test you will observe the certificate is trusted, denoted by the locked padlock symbol in your browser:
Furthermore, because the template script populates an IP address in the Subject Alternative Name (SAN) we have "trust" established when visiting the web URL via IP. Note, it's atypical to add an IP in the SAN for public certificates, however, for internal/private networks this is your discretion.
If the root certificate is not imported to the Web browser or added to the
Operating System trusted store, then an error similar to this will appear:
With minimal changes, custom leaf certificates can be generated with these scripts.
Edit the following in the env.sh file.
Host / Subdomain:
Edit Hostname (HOST_STRING) variable (aka subdomain)
Second Level Domain:
Edit Second Level Domain (SLD_STRING) variable.
Top Level Domain:
Edit Top Level Domain (TLD_STRING) variable.
IP Address - Subject Alternative Name:
Edit IP_SAN1 variable.
Optionally omit this variable to remove a IP entry in the SAN.
Common Name (CN) - Subject Common Name:
As a best practice, the Subject Altnernative DNS name defaults to the value
of $SUBJECT_CN_STRING
eg: hostname.secondleveldomain.topleveldomain or template.middleearth.test
---OPTIONAL VARIABLES---
edit ISSUER_NAME_CN for the Certificate Authority (CA) want to appear on certificates.
Certificate Authority (Issuer):
Edit the ISSUER_NAME_CN variable.
TTL:
Edit the TTL variable (aka expiry) to adjust the validity period of the
certificate in issue_cert_template.sh. This number is set in hours, per valid
by Vault software.
Presently the TTL is set for 9528 hours or 397 days. Set to 397 days as per default expiry for public leaf certificates governed by major browser manufacturers, Google, Apple, and Mozilla. More often than not, many public certificates have a one year maximum validity period. (sometimes less, in the case of automated certifcate renewals, such as Let's Encrypt). Naturally, in the case of development or private environment set the length to your desire. Note that it cannot exceed the TTL of the root CA, which is default 10 years. Refer to References section below for more information. Reference: https://support.apple.com/en-ca/HT211025
After the issue_cert_template.sh script has been run, you can test the
custom certificates in a web browser via docker container. Change to the
./docker folder and adjust the Dockerfile to your needs and execute the
test-certificate.sh script.
Additionally after the create_root_inter_certs.sh script has been run, the
root certificate is copied to the ./docker folder. This is not required for
the docker container, however, it's placed in this folder for convenience to
import to your trusted store or web browser.
To make a backup, or a snapshot of any certifcates (secrets) stored in Vault
run the following command:
./raft.sh save
This will save the state in the current folder under a folder named backup_xxxx. Where xxxx is a random identifier generated. The purpose of the unique identifier is to distinguish between multiple saved states.
Each backup folder will contain three files:
-the snapshot file itself, named snapshotxxxx
-the unseal key, saved in the file named unseal_keyxxxx
-the root token, saved in the file named root_tokenxxxx\
When a snapshot is taken, the matching unseal key and root token must be used to access the vault when a snapshot is restored. This is performed transparently by this script. Refer to below Restore section for more information.
To restore a backup, or a snap of any previously saved state run the following
command:
./raft.sh restore backup_xxxx
This will unseal the vault and login in the current terminal (user) as the root user with the associated backup unseal key and root token.
Any previously generated certificates or other stored data within the Vault will be restored.
To stop the Vault server process, run the following command:
./raft.sh stop
This will stop the current Vault service, remove the storage folder, and delete the associated unseal key and root token.
If there is data (secrets) to be kept with the working Vault session, run the command before stopping the sessions. Refer to Save Vault state section.
To clean up all certificates and backup folders, run the following command:
./raft.sh cleanup
In addition to executing ./raft.sh stop, cleanup will also remove the
root and intermediate certificates, all domain certificates created and the all
backup folders.
This command is destructive and is primarily used to sanitize the working folder.
A couple miscellaneous functions have been left in raft.sh as random functions to demonstrate basic secret storage and retrieval within Vault. These functions are irrelevant to PKI, arguably do not belong here, however they remain in this script as basic examples.
To enable a secrets engine named kv and store a value at kv/apikey with hardcoded example data webapp=AAABBB238472320238CCC, run the command:
./raft.sh putdata
To retrieve and display the above stored data run the following command:
./raft.sh getdata
If this repo / deployment is used as an ephemeral instance, revocation will not work without alternative configuration/additional servers. Web browsers or clients need to verify validity of certificates against a Certificate Revocation List (CRL) or via a Online Certificate Status Protocol (OCSP) server.
If this repo is only used for learning/development, revocation is moot. If this is to be used for a personal home network, then you can determine your risk tolerance and/or alternatives for maintaining certificate revocation and/ or expiry.
An alternative to verifying certificate validity against a CRL or OCSP would be to set short certificate expiry (aka Time-To-Live (TTL)). Doing so would require new certificates to be issued frequently, and preferably in an automated fashion. This would also require a Vault instance to deployed permanently...
For further information regarding certificate revocation, refer to the links in the Reference section.
The file vault_config.hcl is the configuration file for the Vault server. The configuration is fairly straight forward, for more information please refer to the HashiCorp documentation.
This is not meant for a production environment for several reasons.
-
This deployment is a single instance. This is not a reslient deployment.
-
Vault is configured with a single seal token. Typically, five tokens are created and a requires a minimum of three tokens to unseal.
-
The unseal token is exported/displayed in plain-text. A preferred method would be to export and encrypt the token with a public PGP key of the person(s) responsible for maintenance.
-
The root token is writen to disk. For a typical deployment, according to Hashicorp best practices, would be to destroy the root token after the root token has been utlized to setup Vault.
-
The secret certificate (key) for generated certificates are written to the local disk. The intenion here was to have all generated certificates readily available to inspect for learning purposes. Naturally storing secret certificates (keys) should be stored securely. Exposed secret certificates compromises secrecy/trust and defeats the purpose of a Public Key Infrastructure(PKI)!
The Bash scripts were written to adhere as close as possible to Google's Shell Style Guide
The reason the Vault script is name raft.sh is for convenience of execution.
Start to type ./r followed by tab to autocomplete ./raft.sh
followed by the subcommand to execute the desired command.
- Create a bash script to accept configuration parameters as command line arguments instead of editing the script directly.
HashiCorp Build Your Own Certificate Authority
HashiCorp Storage Backend
HashiCorp Vault Backup
HashiCorp Vault Restore
smallstep PKI article
Mozilla Blog - Public Certificate Expiry