Scaleway Image Toolbox

This repository contains the tools, documentations, examples and contents for building, debug and running images on Scaleway.

Distribs & Community

• The distribution images are located on the scaleway organization
• The apps and stacks are located on the scaleway-community organization

Getting start

You can look the docker-based hello-world image.

See Building images on Scaleway with Docker presentation on Slideshare.

Repository

This repository contains :

• Common scripts in ./skeleton-* directories used in the images (upstart, sysvinit, openrc, common helpers, etc)
• The Builder

Official images built with image-tools

Official images are available when creating a new server

Type	Name	State	Versions	Parent	Links
distrib	Ubuntu	released	12.04, 14.04, 14.10, 15.05	n/a	Source
distrib	Debian	released	wheezy	n/a	Source
distrib	Fedora	released	21	n/a	Source
distrib	Alpine Linux	released	3.1	n/a	Source
distrib	Arch Linux	released	n/a	n/a	Source
distrib	Opensuse	wip	n/a	n/a	Source
distrib	Slackware	wip	n/a	n/a	Source
distrib	Busybox	wip	n/a	n/a	Source
app	Docker	released	1.5	ubuntu	Source
app	Ghost	released	n/a	ubuntu	Source
app	Owncloud	released	n/a	ubuntu	Source
app	Pydio	released	6	ubuntu	Source
app	Wordpress	released	4	ubuntu	Source
app	Torrents	released	1	ubuntu	Source
app	OpenVPN	wip	n/a	ubuntu	Source
app	TimeMachine	wip	n/a	ubuntu	Source
app	SeedBox	planned	n/a	ubuntu	Source
app	Mesos	planned	n/a	ubuntu	Source
app	Proxy	planned	n/a	n/a	n/a
app	LEMP	released	n/a	ubuntu	Source
app	Node.js	released	n/a	ubuntu	Source
app	Python	released	n/a	ubuntu	Source
app	Discourse	wip	n/a	ubuntu	Source
app	Gitlab	wip	n/a	ubuntu	Source
app	Java	released	n/a	ubuntu	Source
app	ELK	released	n/a	ubuntu	Source
service	Try-it	released	n/a	docker	Source
service	Rescue	released	n/a	ubuntu	Source
community	moul' dev	private	n/a	ubuntu	Source
community	moul' bench	private	n/a	ubuntu	Source
community	mxs' 3.2 perf	private	n/a	ubuntu	Source
community	mxs' 3.17 perf	private	n/a	ubuntu	Source
community	Camlistore	private	n/a	ubuntu	Source
community	Serendipity	released	n/a	ubuntu	Source
community	Tor	private	n/a	debian	Source

• ImageLayers for distrib images

Builder

We use the Docker's building system to build, debug and even run the generated images.

We added some small hacks to let the image be fully runnable on a C1 server without Docker.

The advantages are :

• Lots of available base images and examples on the Docker's official registry
• Easy inheritance between images (app-timemachine image inherits from app-openvpn image which inherits from ubuntu image)
• Easy debug with docker run ...
• A well-known build format file (Dockerfile)
• Docker's amazing builder advantages (speed, cache, tagging system)

Install

The minimal command to install an image on an attached volume :

# write the image to /dev/nbd1
make install

Commands

# Clone the hello world docker-based app on an armhf server with Docker
git clone https://github.com/scaleway/image-helloworld.git

# Run the image in Docker
make shell

# push the rootfs.tar on s3 (requires `s3cmd`)
make publish_on_s3 S3_URL=s3://my-bucket/my-subdir/

# push the image on docker registry
make release DOCKER_NAMESPACE=myusername

# remove build directories
make clean

# remove build directories and docker images
make fclean

Debug commands

# push the rootfs.tar.gz on s3 (requires `s3cmd`)
make publish_on_s3.tar.gz S3_URL=s3://my-bucket/my-subdir/

# push the rootfs.sqsh on s3 (requires `s3cmd`)
make publish_on_s3.sqsh S3_URL=s3://my-bucket/my-subdir/

# build the image in a rootfs directory
make build

# build a tarball of the image
make rootfs.tar

# build a squashfs of the image
make rootfs.sqsh

Install builder's .mk files

# From a shell
wget -qO - https://raw.githubusercontent.com/scaleway/image-tools/master/builder/install.sh | bash
# or
wget -qO - http://j.mp/scw-builder | bash

Or from a Makefile (example)

## Image tools  (https://github.com/scaleway/image-tools)
all:    docker-rules.mk
docker-rules.mk:
    wget -qO - http://j.mp/scw-builder | bash
-include docker-rules.mk

Unit test a running instance

At runtime, you can proceed to unit tests by calling

# using curl
SCRIPT=$(mktemp); curl -s -o ${SCRIPT} https://raw.githubusercontent.com/scaleway/image-tools/master/builder/unit.bash && bash ${SCRIPT}
# using wget
SCRIPT=$(mktemp); wget -qO ${SCRIPT} https://raw.githubusercontent.com/scaleway/image-tools/master/builder/unit.bash && bash ${SCRIPT}

Image check list

List of features, scripts and modifications to check for proper scaleway image creation.

• Add sysctl entry vm.min_free_kbytes=65536
• Configure NTP to use internal server
• Configure SSH to only accept login through public keys and deny environment customization to avoid errors due to users locale
• Configure default locale to en_US.UTF-8
• Configure network scripts to use DHCP and enable them Although not, strictly speaking, needed since kernel already has IP address and gateway this allows DHCP hooks to be called for setting hostname, etc
• Install custom DHCP hook for hostname to set entry in /etc/hosts for software using getent_r to get hostname
• Install scripts to fetch SSH keys
• Install scripts to fetch kernel modules
• Install scripts to connect and/or mount NBD volumes
• Install scripts to manage NBD root volume
• Disable all physical TTY initialization
• Enable STTY @ 9600 bps

Before making the image public, do not forget to check it boots, stops and restarts from the OS without any error (most notably kernel) since a failure could lead to deadlocked instances.

How to download the common scripts on a target image

Those scripts are mainly used in the base image (distrib) but can sometimes be useful in the app images (inherited from distrib).o

An example of usage in the Ubuntu image

wget -qO - http://j.mp/scw-skeleton | bash -e

# Using upstart flavor
wget -qO - http://j.mp/scw-skeleton | FLAVORS=upstart bash -e
# Using sysvinit, docker-based and common flavors
wget -qO - http://j.mp/scw-skeleton | FLAVORS=sysvinit,docker-based,common bash -e

# Specific GIT branch
wget -qO - http://j.mp/scw-skeleton | FLAVORS=upstart BRANCH=feature-xxx bash -e

# Use curl
curl -L -q http://j.mp/scw-skeleton | DL=curl bash -e

# Alternative URL
wget -qO - https://raw.githubusercontent.com/scaleway/image-tools/master/install.sh | ... bash -e

A running instance can be updated by calling the same commands. It is planned to to create packages (.deb) for distributions.

Licensing

© 2014-2015 Scaleway - MIT License. A project by