Who supervises the supervisor (of your application)?
DeployEx is a lightweight tool designed for managing deployments in Elixir and Gleam applications without relying on additional deployment tools like Docker or Kubernetes. Its primary goal is to utilize the release package for executing full deployments or hot-upgrades, depending on the package's content, while leveraging OTP distribution for monitoring and data extraction.
DeployEx acts as a central deployment runner, gathering crucial deployment data such as the current version and release package contents. The content of the release package enables it to run for a full deployment or a hot-upgrade. Meanwhile, on the development front, your CI/CD pipeline takes charge of crafting and updating packages for the target release. This integration ensures that DeployEx is always equipped with the latest packages, ready to facilitate deployments.
DeployEx is currently used by:
- Calori Web Server for Elixir applications and you can check it at homepage.
- Cochito Web Server for Gleam applications and you can check it at homepage.
Upon deployment, the following dashboard becomes available, offering access to logs for both DeployEx and monitored applications, along with an IEX and/or erl terminal."
- Monitors multiple Elixir/Gleam application instances and automatically restarts them if they crash for any reason.
- Includes a backoff delay restart mechanism to prevent excessive restarts.
- Performs full deployments based solely on the release files generated by:
mix release
for Elixir.gleam export
for Gleam.
- Supports hot code reloading for Elixir applications using the Jellyfish library.
- Supports the following cloud providers:
- Amazon Web Services (AWS)
- Google Cloud Provisioning (GCP)
- Provides rollback functionality if a monitored app version remains unstable for 10 minutes.
- Rolled-back monitored app versions are ghosted, preventing their redeployment.
- Ensures all instances remain connected to the OTP distribution, including DeployEx itself.
- Supports OTP distribution with mutual TLS (mTLS) for secure monitoring of apps and DeployEx.
- Provides the ability to run pre-commans prior deployments for Database migrations or any other eval command.
- Provides a friendly UI that only authenticated users can access.
- Allows setting a previously configured version in the UI, enabling DeployEx to enforce deployment of a specific version.
- Supports individual application restarts via the UI.
- Allows access to current log files (stdout and stderr) for both monitored apps and DeployEx.
- Provides access to the shell:
- IEx shell for monitored Elixir apps and DeployEx.
- Erlang shell for monitored Gleam apps.
- Provides installer script to be used with ubuntu hosts.
- Provides status information per instance:
- OTP connectivity
- Version history
- Last deployment status
- Number of crash restarts
- Number of forced restarts
Note
All examples and deployments in this project use NGINX as a reverse proxy and load balancer. However, DeployEx does not depend on NGINX; it is used here purely for convenience.
- 🚧 Add Gleam support.
- 🚧 Add telemetry support for DeployEx to capture metrics and telemetry via OTP distribution.
- 💤 Lazy deployments for Phoenix apps (Delay Endpoint start to allow fast switch for full deployments)
- 💤 Integrate CPU utilization monitoring from the OTP distribution.
- 💤 Continuous improvement in UI design.
- 💤 Health checks via OTP distribution
- 💤 Secrets from Environment variables to allow installation on servers with no cloud provider
- 💤 Orchestrate distributed databasses like Khepri
- 💤 Enhance installer to become an elixir app capable of hotupgrade DeployEx
You can kickstart the setup with the following commands, the default number of replicas is 3:
mix deps.get
iex --sname deployex --cookie cookie -S mix phx.server
[info] Initialising deployment server
[info] Running DeployexWeb.Endpoint with Bandit 1.5.7 at 127.0.0.1:5001 (http)
[info] Access DeployexWeb.Endpoint at http://localhost:5001
[watch] build finished, watching for changes...
Erlang/OTP 26 [erts-14.1.1] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]
Interactive Elixir (1.16.0) - press Ctrl+C to exit (type h() ENTER for help)
Rebuilding...
Done in 390ms.
[error] Invalid version map at: /tmp/myphoenixapp/versions/myphoenixapp/local/current.json reason: enoent
Now you can visit localhost:5001
from your browser and enter the credentials for the admin user, username: admin password: deployex. You should expect the following dashboard:
Note
The error message in the CLI is due to no monitored app is available to be deployed. If you want to proceed for a local test, follow the steps at Running DeployEx and Monitored app locally. Also, it is important to note that the distribution will be required so this is the reason to add -sname deployex
in the command
The DeployEx app expects a current.json
file to be available, which contains version, hash information and any pre-command. This file is mandatory for full deployment and hot upgrades.
Expected location in the release folder:
# production path
./{bucket}/versions/{monitored_app}/{env}/current.json
# local test path
/tmp/{monitored_app}/versions/{monitored_app}/{env}/current.json
Expected JSON format for current.json
:
{
"version": "1.0.0",
"pre_commands": [ "eval MyApp.Migrator.create", "eval MyApp.Migrator.migrate" ], # optional field
"hash": "local"
}
Once the file is captured, the deployment will start if no app is running or if the current app is running with a version that differs from the current.json
file.
Expected location in the release folder:
# production path
./{bucket}/dist/{monitored_app}/{monitored_app}-{version}.tar.gz
# local test path
/tmp/{monitored_app}/dist/{monitored_app}/{monitored_app}-{version}.tar.gz
DeployEx application typically requires several environment variables to be defined for proper operation. Ensure that you have the following environment variables set when running in production where the ones that have a default value available are not required:
ENV NAME | EXAMPLE | SOURCE | DEFAULT | DESCRIPTION |
---|---|---|---|---|
DEPLOYEX_SECRET_KEY_BASE | 42otsNl...Fpq3dIJ02 | aws secrets | -/- | secret key used for encryption |
DEPLOYEX_ERLANG_COOKIE | cookie | aws secrets | -/- | erlang cookie |
DEPLOYEX_ADMIN_HASHED_PASSWORD | $2b$1...5PAYTZjNQ42ASi | aws secrets | -/- | Hashed admin password for authentication |
DEPLOYEX_MONITORED_APP_NAME | myphoenixapp | system ENV | -/- | Monitored app name |
DEPLOYEX_MONITORED_APP_LANG | elixir | system ENV | -/- | Monitored app language |
DEPLOYEX_CLOUD_ENVIRONMENT | prod | system ENV | -/- | cloud env name |
AWS_REGION | us-east2 | system ENV | -/- | the aws region |
GOOGLE_APPLICATION_CREDENTIALS | /path/to/file.json | system ENV | -/- | the google application credentials path |
DEPLOYEX_PHX_HOST | example.com | system ENV | -/- | The hostname for your application |
DEPLOYEX_PHX_PORT | 5001 | system ENV | 5001 | The port on which the application will run |
DEPLOYEX_PHX_SERVER | true | system ENV | true | enable/disable server |
DEPLOYEX_RELEASE_ADAPTER | s3 or gcp-storage | system ENV | -/- | release adapter type |
DEPLOYEX_RELEASE_BUCKET | myphoenixapp-prod-distribution | system ENV | -/- | release distribution bucket name |
DEPLOYEX_SECRETS_ADAPTER | aws or gcp | system ENV | -/- | release adapter type |
DEPLOYEX_SECRETS_PATH | deployex-myphoenixapp-prod-secrets | system ENV | -/- | secret path to be retrieved from |
DEPLOYEX_MONITORED_APP_PORT | 4000 | system ENV | 4000 | the initial port for starting the monitored apps |
DEPLOYEX_MONITORED_REPLICAS | 2 | system ENV | 3 | Number of replicas to monitor |
DEPLOYEX_DEPLOY_TIMEOUT_ROLLBACK_MS | 600000 | system ENV | 600000 | The maximum time allowed for attempting a deployment before considering the version as non-deployable and rolling back |
DEPLOYEX_DEPLOY_SCHEDULE_INTERVAL_MS | 5000 | system ENV | 5000 | Periodic checking for new deployments |
For local testing, these variables are not expected or set to default values.
DeployEx offers a comprehensive set of Terraform examples for programmatically deploying in AWS and GCP, including detailed step-by-step setup instructions:
If you intend to install DeployEx directly on an Ubuntu server, you can utilize the installer script provided in the release package. For an example of monitored app, please see the setup for the Calori Web Server - AWS/Calori Web Server - GCP. The installer script requires a JSON configuration file, an example of which can be found here. This JSON file can also export environment variables specific to the monitored applications.
Currently, the release and installation process supports Ubuntu versions 20.04 and 22.04. However, you have the option to manually compile and install DeployEx on your target system.
Your application will likely require database commands, such as migrations. DeployEx handles these through pre-commands specified in current.json
under the pre_commands
field. These commands will be executed in the order they are listed, before the application starts. If a pre-command is needed and does not require changes to the application itself, using pre-commands in conjunction with hotupgrade is ideal to avoid unnecessary downtime.
DeployEx uses Secret Manager (AWS or GCP) to fetch its secrets via the config provider. The following environment variable configuration is expected for Secret Manager:
DEPLOYEX_SECRETS_ADAPTER=gcp
DEPLOYEX_SECRETS_PATH=deployex-myapp-prod-secrets
Within the secrets, the following key-value pairs are required:
ENV NAME | EXAMPLE | DESCRIPTION |
---|---|---|
DEPLOYEX_SECRET_KEY_BASE | 42otsNl...Fpq3dIJ02 | mix phx.gen.secret |
DEPLOYEX_ERLANG_COOKIE | my-cookie | erlang cookie |
DEPLOYEX_ADMIN_HASHED_PASSWORD | $2b$1...5PAYTZjNQ42ASi | Bcrypt.hash_pwd_salt("my-pass") |
For local testing, the root path used for distribution releases and versions is /tmp/{monitored_app}
. Follow these steps:
Create the required release folders:
export monitored_app_name=myphoenixapp
mkdir -p /tmp/${monitored_app_name}/dist/${monitored_app_name}
mkdir -p /tmp/${monitored_app_name}/versions/${monitored_app_name}/local/
It is important to note that for local deployments, DeployEx will use the path /tmp/deployex
for local storage. This means you can delete the entire folder to reset any local version, history, or configurations.
In this example, we create a brand new application using mix phx.new
and added the library Jellyfish for testing hotupgrades.
mix local.hex
mix archive.install hex phx_new
mix phx.new myphoenixapp --no-ecto
cd myphoenixapp
vi rel/env.sh.eex
# Add the following lines:
#!/bin/sh
# Set a default Erlang cookie value if not provided by ENV VAR.
# This default is temporary; update it using AWS secrets and config provider.
[ -z ${RELEASE_COOKIE} ] && export RELEASE_COOKIE="cookie"
export RELEASE_DISTRIBUTION=sname
[ -z ${RELEASE_NODE_SUFFIX} ] && export RELEASE_NODE_SUFFIX=""
export RELEASE_NODE=<%= @release.name %>${RELEASE_NODE_SUFFIX}
# save the file :wq
Add Jellyfish library ONLY if the application will need hotupgrades
def deps do
[
{:jellyfish, "~> 0.1.3"}
]
end
You also need to add the following lines in the mix project
def project do
[
...
compilers: Mix.compilers() ++ [:gen_appup, :appup],
releases: [
myphoenixapp: [
steps: [:assemble, &Jellyfish.Releases.Copy.relfile/1, :tar]
]
],
...
]
end
Open the config/prod.exs
and replace the static manifest for a live reload
#config :myphoenixapp, MyphoenixappWeb.Endpoint,
# cache_static_manifest: "priv/static/cache_manifest.json"
# Since the application is using the Hot upgrade, the manifest cannot be static
config :myphoenixapp, MyphoenixappWeb.Endpoint,
live_reload: [
patterns: [
~r"priv/static/.*(js|css|png|jpeg|jpg|gif|svg)$",
~r"priv/gettext/.*(po)$"
]
]
Then you can compile and generate a release
mix deps.get
MIX_ENV=prod mix assets.deploy
MIX_ENV=prod mix release
...
No appups, nothing to move to the release
* assembling myphoenixapp-0.1.0 on MIX_ENV=prod
* using config/runtime.exs to configure the release at runtime
* hot-upgrade copying release file to /Users/testeves/Workspace/Esl/myphoenixapp/_build/prod/rel/myphoenixapp/releases/myphoenixapp-0.1.0.rel
* building /Users/testeves/Workspace/Esl/myphoenixapp/_build/prod/myphoenixapp-0.1.0.tar.gz
Move the release file to the distributed folder and updated the version:
export app_name=myphoenixapp
cp _build/prod/${app_name}-0.1.0.tar.gz /tmp/${app_name}/dist/${app_name}
echo "{\"version\":\"0.1.0\",\"pre_commands\": [],\"hash\":\"local\"}" | jq > /tmp/${app_name}/versions/${app_name}/local/current.json
Move back to the DeployEx project and run the command line with the required ENV vars.
NOTE: All env vars that are available for DeployEx will also be available to the monitored_app
export SECRET_KEY_BASE=e4CXwPpjrAJp9NbRobS8dXmOHfn0EBpFdhZlPmZo1y3N/BzW9Z/k7iP7FjMk+chi
export PHX_SERVER=true
export DATABASE_URL=ecto://postgres:postgres@localhost:5432/myphoenixapp_prod # In case your monitored add is using ecto
iex --sname deployex --cookie cookie -S mix phx.server
...
[info] Update is needed at instance: 1 from: <no current set> to: 0.1.0.
[warning] HOT UPGRADE version NOT DETECTED, full deployment required, result: []
[info] Full deploy instance: 1 deploy_ref: 32656.
[info] Initialising monitor server for instance: 1
[info] Ensure running requested for instance: 1 version: 0.1.0
[info] # Identified executable: /tmp/deployex/varlib/service/myphoenixapp/1/current/bin/myphoenixapp
[info] # Starting application
[info] # Running instance: 1, monitoring pid = #PID<0.790.0>, OS process = 36891 deploy_ref: 32656.
[info] # Application instance: 1 is running
[info] # Moving to the next instance: 2
...
iex(deployex@hostname)1>
You should then visit the application and check it is running localhost:5001. Since you are not using mTLS, the dashboard should look like this:
Note that the OTP-Nodes are connected, but the mTLS is not supported. The mTLS can be enabled and it will be covered ahead. Leave this terminal running and open a new one to compile and release the monitored app.
In this scenario, the existing application will undergo termination, paving the way for the deployment of the new one. It's crucial to maintain the continuous operation of DeployEx throughout this process. Navigate to the myphoenixapp
project and increment the version in the mix.exs
file. Typically, during release execution, the CI/CD pipeline either generates the package from scratch or relies on the precompiled version, particularly for hot-upgrades. If you've incorporated the Jellyfish library and wish to exclusively create the full deployment package, for this test you must follow the steps:
- Remove any previously generated files and generate a new release
cp myphoenixapp
rm -rf _build/prod
MIX_ENV=prod mix assets.deploy
MIX_ENV=prod mix release
...
Generated myphoenixapp app
No appups, nothing to move to the release
Check your digested files at "priv/static"
No appups, nothing to move to the release
* assembling myphoenixapp-0.1.1 on MIX_ENV=prod
* using config/runtime.exs to configure the release at runtime
* hot-upgrade copying release file to /Users/testeves/Workspace/Esl/myphoenixapp/_build/prod/rel/myphoenixapp/releases/myphoenixapp-0.1.1.rel
* building /Users/testeves/Workspace/Esl/myphoenixapp/_build/prod/myphoenixapp-0.1.1.tar.gz
- Now, keep DeployEx running in another terminal and copy the release file to the distribution folder and proceed to update the version accordingly:
export app_name=myphoenixapp
cp _build/prod/${app_name}-0.1.1.tar.gz /tmp/${app_name}/dist/${app_name}
echo "{\"version\":\"0.1.1\",\"pre_commands\": [],\"hash\":\"local\"}" | jq > /tmp/${app_name}/versions/${app_name}/local/current.json
- You should then see the following messages in the DeployEx terminal while updating the app:
[info] Update is needed at instance: 1 from: 0.1.0 to: 0.1.1.
[warning] HOT UPGRADE version NOT DETECTED, full deployment required, result: []
[info] Full deploy instance: 1 deploy_ref: 37406.
[info] Requested instance: 1 to stop application pid: #PID<0.790.0>
[warning] Remaining beam app removed for instance: 1
[info] Initialising monitor server for instance: 1
[info] Ensure running requested for instance: 1 version: 0.1.1
[info] # Identified executable: /tmp/deployex/varlib/service/myphoenixapp/1/current/bin/myphoenixapp
[info] # Starting application
[info] # Running instance: 1, monitoring pid = #PID<0.843.0>, OS process = 37992 deploy_ref: 37406.
[info] # Application instance: 1 is running
[info] # Moving to the next instance: 2
...
For this scenario, the project must first be compiled to the current version and subsequently compiled for the version it's expected to update to. The current.json
file deployed includes the git hash representing the current application version. In this local testing phase, it suffices to compile for the previous version, such as 0.1.1
, and the subsequent version, like 0.1.2
, so the necessary files will be automatically populated.
- Since the application is already compiled for
0.1.1
, change themix.exs
to0.1.2
, apply any other changes if you want to test and execute the command:
MIX_ENV=prod mix assets.deploy
MIX_ENV=prod mix release
...
Generated myphoenixapp app
You can find your generated appups in rel/appups/myphoenixapp/ with the .appup extension
Check your digested files at "priv/static"
You can find your generated appups in rel/appups/myphoenixapp/ with the .appup extension
* assembling myphoenixapp-0.1.2 on MIX_ENV=prod
* using config/runtime.exs to configure the release at runtime
* hot-upgrade copying release file to /Users/testeves/Workspace/Esl/myphoenixapp/_build/prod/rel/myphoenixapp/releases/myphoenixapp-0.1.2.rel
* building /Users/testeves/Workspace/Esl/myphoenixapp/_build/prod/myphoenixapp-0.1.2.tar.gz
- Now, copy the release file to the distribution folder and proceed to update the version accordingly:
export app_name=myphoenixapp
cp _build/prod/${app_name}-0.1.2.tar.gz /tmp/${app_name}/dist/${app_name}
echo "{\"version\":\"0.1.2\",\"pre_commands\": [],\"hash\":\"local\"}" | jq > /tmp/${app_name}/versions/${app_name}/local/current.json
You can then check that DeployEx had executed a hot upgrade in the application:
[info] Update is needed at instance: 1 from: 0.1.1 to: 0.1.2.
[warning] HOT UPGRADE version DETECTED, from: 0.1.1 to: 0.1.2
[info] Hot upgrade instance: 1 deploy_ref: 37406.
[info] Unpacked successfully: ~c"0.1.2"
[info] Installed Release: ~c"0.1.2"
[info] Made release permanent: 0.1.2
[info] Release upgrade executed with success at instance: 1 from: 0.1.1 to: 0.1.2
[info] # Moving to the next instance: 2
...
you can check that the version and the deployment status has changed in the dashboard:
In order to improve security, mutual TLS (mTLS
for short) can be employed to encrypt communication during OTP distribution. To implement this, follow these steps:
- Generate the necessary certificates, DeployEx has a good examples of how to create self-signed tls certificates:
cd deployex
make tls-distribution-certs
- Copy the generated certificates to the
/tmp
folder:
cp ca.crt /tmp
cp deployex.crt /tmp
cp deployex.key /tmp
- Create the
inet_tls.conf
file with the appropriate paths, utilizing the command found inrel/env.sh.eex
:
export DEPLOYEX_OTP_TLS_CERT_PATH=/tmp
test -f /tmp/inet_tls.conf || (umask 277
cd /tmp
cat >inet_tls.conf <<EOF
[
{server, [
{certfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/deployex.crt"},
{keyfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/deployex.key"},
{cacertfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/ca.crt"},
{verify, verify_peer},
{secure_renegotiate, true}
]},
{client, [
{certfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/deployex.crt"},
{keyfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/deployex.key"},
{cacertfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/ca.crt"},
{verify, verify_peer},
{secure_renegotiate, true},
{server_name_indication, disable}
]}
].
EOF
)
- Ensure that
myphoenixapp
also utilizes the same options and certificate by updatingrel/env.sh.eex
:
cd myphoenixapp
vi rel/env.sh.eex
# Add the following line
#!/bin/sh
export ELIXIR_ERL_OPTIONS="-proto_dist inet_tls -ssl_dist_optfile /tmp/inet_tls.conf"
# save the file :q
- To enable
mTLS
for DeployEx, set the appropriate Erlang options before running the application in the terminal:
ELIXIR_ERL_OPTIONS="-proto_dist inet_tls -ssl_dist_optfile /tmp/inet_tls.conf -setcookie cookie" iex --sname deployex -S mix phx.server
After making these changes, create and publish a new version 0.1.3
for myphoenixapp
and run the DeployEx with the command from item 5. After the deployment, you should see the following dashboard:
[!ATTENTION] Ensure that the cookie is properly set
For local testing, the root path used for distribution releases and versions is /tmp/{monitored_app}
. Let's create the required release folders:
export monitored_app_name=mygleamapp
mkdir -p /tmp/${monitored_app_name}/dist/${monitored_app_name}
mkdir -p /tmp/${monitored_app_name}/versions/${monitored_app_name}/local/
Since Elixir is the default language for deployex, it will require set the respective values in the same terminal where deployex will run:
export DEPLOYEX_MONITORED_APP_NAME=mygleamapp
export DEPLOYEX_MONITORED_APP_LANG=gleam
It is important to note that for local deployments, DeployEx will use the path /tmp/deployex
for local storage. This means you can delete the entire folder to reset any local version, history, or configurations.
In this example, we create a brand new gleam app:
gleam new mygleamapp
cd mygleamapp
Add the following dependency (gleam_erlang) at gleam.toml
:
[dependencies]
gleam_stdlib = ">= 0.34.0 and < 2.0.0"
gleam_erlang = ">= 0.27.0 and < 1.0.0"
Modify the main function to sleep forever at src/mygleamapp.gleam
, otherwise the application will run and exit:
import gleam/io
import gleam/erlang/process
pub fn main() {
io.println("Hello from mygleamapp!")
process.sleep_forever()
}
Then you can compile and generate a release
gleam deps update
gleam export erlang-shipment
Pack the release and move it to the distributed folder and updated the version:
cd build
export app_name=mygleamapp
export release_path=erlang-shipment
tar -czvf ${release_path}/${app_name}-0.1.0.tar.gz ${release_path}
cp ${release_path}/${app_name}-0.1.0.tar.gz /tmp/${app_name}/dist/${app_name}
echo "{\"version\":\"0.1.0\",\"pre_commands\": [],\"hash\":\"local\"}" | jq > /tmp/${app_name}/versions/${app_name}/local/current.json
Note
Gleam doesn't have a release command (yet). For DeployEx to operate properly, we need a tarbal that contains the erlang-shipment with the respective version. There is an example in cochito
Move back to the DeployEx project and run the command line with the required ENV vars.
NOTE: All env vars that are available for DeployEx will also be available to the monitored_app
export DEPLOYEX_MONITORED_APP_NAME=mygleamapp
export DEPLOYEX_MONITORED_APP_LANG=gleam
export SECRET_KEY_BASE=e4CXwPpjrAJp9NbRobS8dXmOHfn0EBpFdhZlPmZo1y3N/BzW9Z/k7iP7FjMk+chi
export PHX_SERVER=true
iex --sname deployex --cookie cookie -S mix phx.server
...
[info] Update is needed at instance: 1 from: <no current set> to: 0.1.0
[warning] HOT UPGRADE version NOT DETECTED, full deployment required, result: []
[info] Full deploy instance: 1 deploy_ref: 9k416t
[info] Initialising monitor server for instance: 1
[info] Ensure running requested for instance: 1 version: 0.1.0
[info] # Identified executable: /tmp/deployex/varlib/service/mygleamapp/1/current/erlang-shipment
[info] # Starting application
[info] # Running instance: 1, monitoring pid = #PID<0.819.0>, OS process = 87157 deploy_ref: 9k416t
[info] # Application instance: 1 is running
[info] # Moving to the next instance: 2
...
iex(deployex@hostname)1>
You should then visit the application and check it is running localhost:5001. Since you are not using mTLS, the dashboard should look like this:
Note that the OTP-Nodes are connected, but the mTLS is not supported. The mTLS can be enabled and it will be covered ahead. Leave this terminal running and open a new one to compile and release the monitored app.
In this scenario, the existing application will undergo termination, paving the way for the deployment of the new one. It's crucial to maintain the continuous operation of DeployEx throughout this process. Navigate to the mygleamapp
project and increment the version in the gleam.toml
file.
- Remove any previously generated files and generate a new release
gleam export erlang-shipment
- Now, keep DeployEx running in another terminal and copy the release file to the distribution folder and proceed to update the version accordingly:
export app_name=mygleamapp
export release_path=erlang-shipment
cd build
tar -czvf ${release_path}/${app_name}-0.1.1.tar.gz ${release_path}
cp ${release_path}/${app_name}-0.1.1.tar.gz /tmp/${app_name}/dist/${app_name}
echo "{\"version\":\"0.1.1\",\"pre_commands\": [],\"hash\":\"local\"}" | jq > /tmp/${app_name}/versions/${app_name}/local/current.json
- You should then see the following messages in the DeployEx terminal while updating the app:
[info] Update is needed at instance: 1 from: 0.1.0 to: 0.1.1
[warning] HOT UPGRADE version NOT DETECTED, full deployment required, result: []
[info] Full deploy instance: 1 deploy_ref: xkmmz7
[info] Requested instance: 1 to stop application pid: #PID<0.819.0>
[info] Initialising monitor server for instance: 1
[info] Ensure running requested for instance: 1 version: 0.1.1
[info] # Identified executable: /tmp/deployex/varlib/service/mygleamapp/1/current/erlang-shipment
[info] # Starting application
[info] # Running instance: 1, monitoring pid = #PID<0.1455.0>, OS process = 88828 deploy_ref: xkmmz7
[info] # Application instance: 1 is running
[info] # Moving to the next instance: 2
...
In order to improve security, mutual TLS (mTLS
for short) can be employed to encrypt communication during OTP distribution. To implement this, follow these steps:
- Generate the necessary certificates, DeployEx has a good examples of how to create self-signed tls certificates:
cd deployex
make tls-distribution-certs
- Copy the generated certificates to the
/tmp
folder:
cp ca.crt /tmp
cp deployex.crt /tmp
cp deployex.key /tmp
- Create the
inet_tls.conf
file with the appropriate paths, utilizing the command found inrel/env.sh.eex
in deployex project:
export DEPLOYEX_OTP_TLS_CERT_PATH=/tmp
test -f /tmp/inet_tls.conf || (umask 277
cd /tmp
cat >inet_tls.conf <<EOF
[
{server, [
{certfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/deployex.crt"},
{keyfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/deployex.key"},
{cacertfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/ca.crt"},
{verify, verify_peer},
{secure_renegotiate, true}
]},
{client, [
{certfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/deployex.crt"},
{keyfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/deployex.key"},
{cacertfile, "${DEPLOYEX_OTP_TLS_CERT_PATH}/ca.crt"},
{verify, verify_peer},
{secure_renegotiate, true},
{server_name_indication, disable}
]}
].
EOF
)
- To enable
mTLS
for DeployEx, set the appropriate Erlang options before running the application in the terminal:
ELIXIR_ERL_OPTIONS="-proto_dist inet_tls -ssl_dist_optfile /tmp/inet_tls.conf -setcookie cookie" iex --sname deployex -S mix phx.server
After making these changes, create and publish a new version 0.1.2
for mygleamapp
and run the DeployEx with the command from item 5. After the deployment, you should see the following dashboard:
# production
tail -f /var/log/deployex/deployex-stdout.log
tail -f /var/log/deployex/deployex-stderr.log
# local test
# not available when running as dev env
export RELEASE_NODE_SUFFIX=""
export RELEASE_COOKIE=cookie
# production
/opt/deployex/bin/deployex remote
# local test
# not available when running as dev env
export instance=1
export monitored_app_name=myphoenixapp
# production
tail -f /var/log/${monitored_app_name}/${monitored_app_name}-${instance}-stdout.log
tail -f /var/log/${monitored_app_name}/${monitored_app_name}-${instance}-stderr.log
# local test
tail -f /tmp/${monitored_app_name}/${monitored_app_name}/${monitored_app_name}-${instance}-stdout.log
tail -f /tmp/${monitored_app_name}/${monitored_app_name}/${monitored_app_name}-${instance}-stderr.log
export instance=1
export monitored_app_name=myphoenixapp
export RELEASE_NODE_SUFFIX=-${instance}
export RELEASE_COOKIE=cookie
# production
/var/lib/deployex/service/${monitored_app_name}/${instance}/current/bin/${monitored_app_name} remote
# local test
/tmp/deployex/varlib/service/${monitored_app_name}/${instance}/current/bin/${monitored_app_name} remote
DeployEx operates by monitoring applications and versions using folders and files, treating the monitored app as a service:
# test environment
/tmp/deployex/varlib/service/${monitored_app}/${instance}/previous/${monitored_app}
/tmp/deployex/varlib/service/${monitored_app}/${instance}/new/${monitored_app}
/tmp/deployex/varlib/service/${monitored_app}/${instance}/current/${monitored_app}
# production environment
/var/lib/deployex/service/${monitored_app}/${instance}/previous/${monitored_app}
/var/lib/deployex/service/${monitored_app}/${instance}/new/${monitored_app}
/var/lib/deployex/service/${monitored_app}/${instance}/current/${monitored_app}
The deployment process involves several steps to ensure smooth transitions:
- Download and Unpack the New Version:
The new version of the application is downloaded and unpacked into the
new
service folder, ready for deployment. - Check if the release contain a hot-upgrade or full deployment: DeployEx will check the release file received and if it is a full deployment, goes to the step 3 .
- Stop the Current Application: The currently running application instance is stopped to prepare for the new deployment.
- Delete the Previous Service Folder:
The
previous
service folder, containing the previous version of the application, is deleted to make space for the new version. - Move the Current Service:
The
current
service folder, representing the current version of the application, is moved to theprevious
service folder. Simultaneously, thenew
service folder is moved to become the newcurrent
service folder. - Run pre_commands:
Before starting the application, DeployEx will attempt to run any pre_commands (if set) using the
current
service folder. - Start the Application:
Finally, the application is started using the version now residing in the
current
service folder, ensuring that the latest version is active and operational.
By following this process, DeployEx facilitates deployments, ensuring that applications are updated while minimizing downtime.
For this scenario, there will be no moving files/folders since the target is to keep the current service folder updated. The sequence is:
- Download and Unpack the New Version:
The new version of the application is downloaded and unpacked into the
new
service folder, ready for deployment. - Check if the release contain a hot-upgrade or full deployment: DeployEx will check the release file received and if it is a hot-upgrade, goes to the step 3 .
- Run pre_commands:
Before executing the hot-upgrade, DeployEx will attempt to run any pre_commands (if set) using the
new
service folder. - Execute the Hotupgrade checks and verification DeployEx will try to run the hotupgrade sequence and if succeeds, it makes the changes permanent. In any case of failure, it tries to execute a full deployment with the same release file.
☎️ Contact us: Feel free to contact me on Linkedin.
Copyright (c) 2024, Thiago Esteves.
DeployEx source code is licensed under the MIT License.