SHIP is a lightweight, shell script only, command line tool to manage application deployments on a kubernetes/marathon cluster using the power of user computer to do the builds. It was created with small startups limited on budget that can not afford paying for a CI service or havin one or their cluster.
On default Ship configuration, each assumes each app have an unique name that will represent:
- Folder name in user workspace (only for local deploys and deploys using ship.json file - see below);
- Git repository name. Ex.: bitbucket.org/company/app-name.git;
- Docker image name: Ex.: registry.company.com:5000/app-name:latest;
- Kubernetes controller and service names or Marathon app name:
- Marathon App: app-name
- K8s Service: app-name
- K8s RC: app-name-controller
So, when you do a deploy like ship deploy app-name -t node, ship will use the docker template found in $SHIP_TEMPLATES/node for node apps, replace some environment variables on the template, trigger a docker build, push the image to the registry, trigger de deployment on kubernetes/marathon and watch if it was successful, otherwise it will do a rollback.
Ship assumes that the Dockerfile template will do all the procedures needed by the application, like the checkout of the application inside the container and run yarn install and whatever is needed by you aplication, you are in control of the build.
All the procedures are made in the user machine, it was designed with small companies/startups in mind, when you can't afford having servers for a CI tool, or does not have the budget to pay for a commercial solution.
But nothing stops you from running ship on your Jenkins/CI server. :)
SHIP also supports local deployments to simulate production environment on developer machine. When creating a container locally, SHIP will mount the user $WORKSPACE/APP-NAME folder on the container LOCAL_DEPLOY_APP_DIR directory so you can continue the development and run the application inside the container. It will also mount your entire workspace inside /workspace so you can link projects while in development.
There are drawbacks in this method. Although you have a environment much closer to the production environment and can prevent many build problems, you have to be aware of a few things:
- App Compiled Dependencies:
Although you can develop in your machine and do all the programming as you normally do, when installing compiled dependencies (like some nodejs dependencies and golang packages), the dependencies must be compiled inside the container, you can't execute yarn/npm install on your machine: you must use 'ship :shell app-name' and execute inside the container.
-
Your workspace is used to:
- Look for WORKSPACE/app-name/ship.json when deploying using 'ship deploy app-name'
- Mount app folder insinde the contianer on local deploys
-
If you don't configure your workspace you can continue deploying by:
- Using 'ship deploy' inside a folder containing a ship.json file
- Using ship command line flags: 'ship deploy app-name -t node'
- Two folders are mounted in each local container to simulate persistent data in your cluster:
- /data (App only data)
- /shared (Shared by all apps)
- The common folders used by package management tools are mounted in each container and shared by them all:
- /root/.cache (yarn, bower)
- /root/.npm (npm)
- /root/.composer (npm)
- /root/.sbt (scala sbt projects)
Ship will replace by default this placeholders in the Dockerfiles:
| Placeholder | Description |
|---|---|
| APP\ | The app name |
| APP_BRANCH\ | The app git branch to use |
| APP_VERSION\ | The app version (the actual docker image tag) |
| APP_REPO\ | The complete repository URL |
| MAIN_SCRIPT\ | The main script to execute on nodejs apps |
This way you can build generic images to language specific apps.
You can define any variable placeholder inside a docker template and set a fixed value to be replaced by SHIP either in your config.json (usualy ~/.ship/config.json), which will be applied globally to all app that uses a specific template, or in app ship.json so it will be applied to that app only.
The app ship.json takes precedence over user config.json.
To make it work, insert the placeholders in your template Docker file and then configure the replacements in either user or app json file. Examples:
Template Dockerfile
FROM alpine:3.6
ENV REPLACED_VAR __PLACEHOLDER__
RUN sh -c "while :; do echo ${REPLACED_VAR}; echo '__PLACEHOLDER2__'; sleet 5; done"
Global Replacement (in all apps)
In your config.json (usualy ~/.ship/config.json) do as follow:
{
"ship": {},
"container_management": {},
"registry": {},
"cluster": {},
"dev": {},
"replacement": {
"TEMPLATE_NAME": {
"__PLACEHOLDER__": "REPLACEMENT_VALUE",
"__PLACEHOLDER2__": "REPLACEMENT_VALUE2"
}
}
}By APP
In your app ship.json do as follow:
{
"ship": {},
"remote": {},
"vcs": {},
"dependencies": {},
"replacement": {
"__PLACEHOLDER__": "REPLACEMENT_VALUE",
"__PLACEHOLDER2__": "REPLACEMENT_VALUE2"
}
}Each project/app can have a ship.json file in the following format:
{
"ship": {
"name": "company-site",
"type": "node",
"scope": "app",
"script": "app.js",
"url": "yourcompany.com,www.yourcompany.com",
"ports": ["3024:80"]
},
"remote": {
"resource_profile": "medium",
"expose": false,
"port": 10080,
"health": "/health",
"env": [{
"name": "SOME_ENV_VAR",
"value": "some_content"
}],
"labels": [{
"name": "enableHTTP",
"value": "1"
}],
"volumes": [{
"container": "/data",
"host": "/nfs/persistent"
}]
},
"vcs": {
"url": "git@github.org:yourcompany/company-site.git",
"branch": "dist"
},
"registry": {
"url": "registry.yourcompany.com:5000",
"username": "user",
"password": "pwd"
},
"replacement": {
"__PLACEHOLDER__": "REPLACEMENT_VALUE",
"__PLACEHOLDER2__": "REPLACEMENT_VALUE2"
},
"dependencies": [{
"name": "api-database",
"image": "api-database",
"ports": [],
"volumes": [],
"type": "node"
},
{
"name": "api-indicators",
"image": "api-indicators",
"ports": [],
"volumes": [],
"type": "node"
}
]
}Using a config file is easier because you don't need to pass many parameters when doing operations with ship.
Examples:
-
Deploy app-name without a ship.json file:
- Issue this command from anywhere: ship deploy app-name -t node
-
Deploy app-name using a ship.json file:
- Issue this command from app-name folder: ship deploy
- Issue this command from anywhere: ship deploy app-node
- name: The app name. Must be equal to the app folder name in your workspace and equal to the bitbucket git repo;
- type: The app type. Will be used to choose the right template for this app from TEMPLATES_DIR/TYPE/Dockerfile
- scope: The app scope. Is used to classify the application on the cluster, label it in Marathon or Kubernetes;
- url: The external url that this app responds to.
- script: The name of the main app script.
- ports: The ports to expose (Ex.: ["80:80", "1234:1234/udp"]).
- expose: Boolean. This should be exposed with NodePort so external reverse proxy can reach it?
- health: Health check URI to http curl ping (Ex.: "/health").
- port: Node (mesos slave) port to expose when using expose: true.
- resource_profile: One of the profiles found in the ship project
config.jsonfile - env: An array of objects containing the environment variable name and value to apply to the service if available on orchestration engine (this will also be used on local deploys)
- labels: An array of objects containing the label name and value to apply to the service if available on orchestration engine
- volumes: A lists of volumes to mount on the application containers (
hostpath andcontainerpath)
Tip: Avoid creating large resource_profiles, your application must bre prepared to scale in number of containers. If you are creating a very large resource profile there's a great change that you are doing something wrong. Even so, there are a few applications that may require a high minimum level of resources.
Dependencies are used for local deployments only.
- name: The app name. Must be equal to the app folder name in your workspace and equal to the bitbucket git repo;
- type: The app type. Will be used to choose the right template for this app (node,node6,html,php);
- ports: Ports array to expose in the dependency container; (Ex.: "volumes": ["80:80","443:443"])
- volumes: Volumes array to mount in the dependency container. (Ex.: "volumes": ["/test1:/app","/test2:/app2"])
When you put the dependencies in the ship.json file inside the project root folder you have two options:
"dependencies": [
{"name": "api-database", "type": "node"},
{"name": "app-database"}
]- Put only the dependency name
- In this case it will look in your workspace for a folder with this name to look for a ship.json file and use the correct param when launching this application container.
- Put the dependency name and type
- It will use this template type if it can't find the application ship.json file.
You can define any variable placeholder inside a docker template and set a fixed value to be replaced by SHIP in your config.json (usualy ~/.ship/config.json). They will be applied globally to all app that uses a specific template.
Don't forget to add the placeholders to your template Dockerfile.
In this example, SHIUP will look for "**NPM_TOKEN**" in the Dockerfile of the template "nodejs" and replace by "TOKEN_VALUE":
"replacement": {
"nodejs": {
"__NPM_TOKEN__": "TOKEN_VALUE"
}
}- Deploy a container locally pointing to the code on your machine
ship :deploy app-name # You have the env var WORKSPACE set and a ship.json file in the app folder
ship :deploy app-name --type=node # You have the env var WORKSPACE set and don't have a ship.json file in the app folder
ship :deploy app-name --type=node --workspace==/home/user/workspace/ # Will look for /home/user/workspace/app-name- Destroy all local containers created by Ship
ship :destroy
ship :destroy all- Destroy a single local container created by Ship
ship :destroy app-name- Deploy a container locally with a specific production version
ship :deploy app-name --version=999- Deploy api-indicators with ship.json file settings:
ship deploy app-name- Deploy app-name with node template:
ship deploy app-name --type=node- Deploy a new application with node template, classifying as an api:
ship deploy new-awesome-app --type=node --scope=api- Deploy app-name setting the default script to app/app.js:
ship deploy app-name -type=node --script=app/app.js
ship deploy app-name --type=node --script=http-server- Deploy app-name from test branch with the name app-name-beta
ship deploy app-name-beta --branch=test --repo=app-name --type=node- Rollback a deployment to the previous version (saved in the previous deploy attempt):
ship rollback app-name- Update to a specific existing image tag:
ship update app-name --version=32ship ACTION APP_NAME [OPTIONS]ACTIONS [REMOTE]:
| Action | Shorthand | Description |
|---|---|---|
| deploy | none | Deploy APP_NAME (Build, push to registry, trigger deployment) |
| build | none | Only build the app image (do NOT push to registry and do NOT trigger deployment) |
| release | none | Build the app image and push to registry (do NOT trigger deployment) |
| rollback | none | Rollback to the last saved version of APP_NAME (doesn't work if ship in container) |
| update | none | Update an app to an existing image tag |
| list | none | List all apps on cluster |
ACTIONS [LOCAL]:
| Action | Shorthand | Description |
|---|---|---|
| config | none | Configure ship when running directly in your computer (not in SHIP container) |
| local:deploy | :deploy | Deploy locally on your computer the app APP_NAME |
| local:destroy | :destroy | Remove all containers created by SHIP (no argument) or the container APP_NAME |
| local:list | :list | List all containers created by ship |
| local:shell | :shell | Connect to a local container shell (sh) |
| local:restart | :restart | Restarts a local container |
| local:logs | :logs | Retrieve local container logs |
| help | none | Display ship help |
| examples | none | Display examples of usage |
OPTIONS:
| Option | Description |
|---|---|
| --version | Deploy/update to selected version |
| --type | Template to use (will look for TEMPLATES_DIR/type/Dockerfile) |
| --scope | Scope of application to classify in cluster (Defaults to 'none'). Only needed for new applications that are not in k8s yet (OPTIONAL) |
| --script | Main script of the application. Defaults to app.js (OPTIONAL) |
| --health_uri | Health check URI for the application. Defaults to '/health' (OPTIONAL) |
| --port | Which host port to bind to when deploying locally, defaults to RANDOM (OPTIONAL) |
| --workspace | Define a custom workspace path and try to mount WORKSPACE/APP_NAME inside container. If not defined will use the env var WORKSPACE if set. Else will deploy production version. Example: ship local flow-engine -t php -w $WORKSPACE (OPTIONAL - Only used when deploying a local application with the 'local' option) |
| --app-dir | Where to mount your local workspace app dir inside container |
| --skip-mount | Does not mount your development code into the container LOCAL_DEPLOY_APP_DIR |
| --force | Force running update even if the remote version is the same as define in --version (OPTIONAL) |
| --all | Run action to all containers (OPTIONAL) |