Hokusai is a Docker + Kubernetes CLI for application developers.
Hokusai "dockerizes" applications and manages their lifecycle throughout development, testing, and release cycles.
Hokusai wraps calls to Kubectl, Docker, Docker-Compose and Git with a CLI, and defines a CI workflow.
Hokusai currently only supports Kubernetes deployments on AWS, configured to pull from ECS container repositories (ECR), although other providers may be added in the future.
At Artsy, as we began working with Kubernetes, while impressed with its design, capabilities, and flexibility, we were in need of tooling we could deliver to agile development teams that addressed the day-to-day tasks of application development, delivery, introspection and maintenance, while providing a clean and uncomplicated interface.
Transitioning teams to the Docker / Kubernetes ecosystem can be intimidating, and comes with a steep learning curve. We set out to create a Heroku-like CLI that would shepherd the application developer into the ecosystems of Docker and Kubernetes, and while introducing new tooling and concepts, outlining a clear practice for dependency management, local development, testing and CI, image repository structure, deployment and orchestration.
We recommend installing via Homebrew:
$ brew update
$ brew tap artsy/formulas
$ brew install hokusai
If you previously installed Hokusai via an alternate installation method, you may need to force the link step.
$ brew link --overwrite hokusai
If you previously installed Hokusai via Pip, you may want to first uninstall it:
$ pip uninstall hokusai
curl -sSL https://raw.githubusercontent.com/artsy/hokusai/main/get-hokusai.sh | sudo bash
Note: This method installs Hokusai to /usr/local/bin/hokusai.
Hokusai can be installed via Pip, on MacOS or Linux. If you do so, please first go through Pyenv, Python, and Virtualenv steps.
Python 3.7+ is required.
pip install hokusai
Note: If Pip fails at upgrading your system Python packages, try:
pip install hokusai --ignore-installed
We also maintain Hokusai Docker images for running Hokusai in Docker.
Release artifacts are available on Github.
Release artifacts are also available in AWS S3. You can use this convenience script or Curl to fetch them.
Hokusai currently supports Python 3.7+ only. The last version that supported Python 2.x was v0.5.18.
We assume that your org admin has already set up a Kubernetes cluster and an AWS infrastructure, and that your local environment has Git, Docker, and Docker-Compose installed. Perform the following steps to get Hokusai working:
-
Ensure your org admin has completed the steps mentioned in Administering Hokusai.
-
Configure your local environment with AWS credentials.
-
Optionally, enable Bash autocompletion:
eval "$(_HOKUSAI_COMPLETE=source hokusai)"
Once Hokusai is configured, you can start using it on a project. Please see Getting Started.
A full command reference can be found in Command Reference.
Hokusai can be used to simplify the process of spinning up a "review app" instance of your project, based on a feature branch or pull request.
Full details are in the Review App reference.
A descripton of all the files used by Hokusai.
To work on Hokusai itself, please set up:
We recommend using Pyenv to install the correct version of Python. For a tutorial of Pyenv, see this guide.
When installing on MacOS, please make sure to use brew-installed openssl and readline libraries, and xcode-installed zlib library. And make sure these libraries are correctly linked. Like so:
brew install openssl readline zlib
echo 'export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"' >> ~/.bash_profile
echo 'export LDFLAGS="-L/usr/local/opt/openssl@1.1/lib"' >> ~/.bash_profile
echo 'export CPPFLAGS="-I/usr/local/opt/openssl@1.1/include"' >> ~/.bash_profile
echo 'export PKG_CONFIG_PATH="/usr/local/opt/openssl@1.1/lib/pkgconfig"' >> ~/.bash_profile
Hokusai is currently tested on Python 3.10 so we recommend using that Python version.
If you use Pyenv to install Python, you should see an output similar to this:
$ pyenv install 3.10
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.10.13.tar.xz...
-> https://www.python.org/ftp/python/3.10.13/Python-3.10.13.tar.xz
Installing Python-3.10.13...
python-build: use tcl-tk from homebrew
python-build: use readline from homebrew
python-build: use ncurses from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.10.13 to /Users/jxu/.pyenv/versions/3.10.13
With the desired Python version installed, activate it globally:
pyenv global 3.10
Note: If you want to create a PyInstaller distribution, Python must be installed with development libraries. Use the environment variable PYTHON_CONFIGURE_OPTS="--enable-framework" on Darwin and PYTHON_CONFIGURE_OPTS="--enable-shared" on Linux when running pyenv install.
We recommend using a virtual environment to isolate Hokusai's dependencies from that of other projects on your local environment.
The Pyenv install comes with pyenv-virtualenv which can be used to create virtual environments.
Use Poetry to install Hokusai's dependencies as well as Hokusai itself in editable mode. Please see this guide for working with Poetry.
Install Poetry:
pip install --upgrade pip
pip install poetry
Install dependencies and Hokusai in editable mode:
poetry install
To update dependencies:
poetry lock
Minikube is used for integration tests.
brew install minikube
minikube start --kubernetes-version=<version of your Kubernetes clusters, example: v1.2.3>
To run unit tests:
make test
To run integration tests:
make integration-local
Only specific modules, TestClasses, or even methods:
python -m unittest test.unit.test_module.TestClass.test_method
Tip: Set DEBUG=1 environment variable to print boto logging
Merging a branch into main automatically creates a beta version useful for testing. A script that runs in CI automatically generates a Beta version number and writes it into VERSION file. The version number is based on the combination of the latest canonical version found in RELEASE_VERSION file and the latest Git commit in main branch. VERSION file in version control has just a dummy number (e.g. 999.999.999).
To install the beta:
$ brew uninstall hokusai
$ brew uninstall hokusai-beta
$ brew update
$ brew tap artsy/formulas
$ brew install hokusai-beta
curl -sSL https://raw.githubusercontent.com/artsy/hokusai/main/get-hokusai.sh | sudo bash -s beta
To create an official release, such as v1.2.3, perform the following:
-
Create a branch named
prepare-v1.2.3and make the following changes:- Bump canonical version in RELEASE_VERSION file.
- Upate CHANGELOG.
- Open a PR to merge into
main. Please see past PRs for example.
-
Open a PR to merge
mainintorelease. Please see past PRs for example.
A CI script will copy the version in RELEASE_VERSION file to VERSION file.
The project is named for the great Japanese artist Katsushika Hokusai (1760-1849).
This project is the work of engineers at Artsy, the world's leading and largest online art marketplace and platform for discovering art. One of our core Engineering Principles is being Open Source by Default which means we strive to share as many details of our work as possible.
You can learn more about this work from our blog and by following @ArtsyOpenSource or explore our public data by checking out our API. If you're interested in a career at Artsy, read through our job postings!