Customize CIAB image creation, for automation base image 7.6

[nivgsk]

Motivation

We need to support Skai's MemSQL automation image use case:

1. Start a cluster-in-a-box memsql docker image.
2. Run several cucumbers to populate it with schemas, tables and data.
3. Commit the docker state and upload the image to an artifactory.
4. Download the image in different automation jobs, start it and run tests that rely on the committed data.

Q: Aren't the default SingleStore scripts sufficient for this use case?
A: No, because they don't allow the retention of data after having populated the image with data and committed it.
Additionally, with the image built by the default scripts, it's impossible to start the image twice (i.e. start the committed image), due to the usage of the sdb-toolbox tool in the startup script.

Implementation

The main changes in the PR are:

• Makefile: Explicitly specify our desired memsql version.
• Dockerfile-ciab:
  • Specify the base image and other tool versions.
  • Add permissions to run installations on the image.
  • Skip the VOLUME command, presumably to allow data retention on the image once we use it in our Jenkins automations.
• Startup:
  • Unregister host, as it prevents the committed image to be started again as a new docker in automation jobs.
  • Add custom memsql variable values to reduce the final committed image size.

Instructions - build base image

The customized scripts, all in all, serve for us to build a clean CIAB 7.6 image, ready to be populated with data.
We can do it locally, or with a jenkins job (that would do the same things we'd do locally):

1. cd to repo base directory.
2. Run commands:
   • make - create the base image(s) for the Dockerfiles in the repo
   • docker build -t kenshoo-docker.jfrog.io/ks-db-memsql-76-cluster-in-a-box-base -f Dockerfile-ciab . - build a cluster-in-a-box image on top of the default image, tagged for our artifactory
   • docker push kenshoo-docker.jfrog.io/ks-db-memsql-76-cluster-in-a-box-base - push it to the artifactory

Then, to create a populated MemSQL docker image for the automation jobs, run the automation-tests-db-upload on Jenkins-Search.

Recommendations - discovery for next version upgrade

When it's time to upgrade from 7.6 to some new version, one of the steps will be to create a new base image for Skai's automation environment.

I suggest trying it like this:

1. Check what changes have been made on SingleStore's deployment-docker scripts repository since the version we forked.
   2 methods for doing that:
   • Via Github: master...memsql:deployment-docker:master
   • Locally:
     1. Checkout this repo.
     2. git remote add -f memsql_src git@github.com:memsql/deployment-docker.git
     3. git remote update
     4. git diff master remotes/memsql_src/master
     5. git remote rm memsql_src
        If they seem necessary, apply them to the repo on your local machine.
2. Try to apply the same changes that this PR contains (see Implementation section above) to the current version of the official repo. Adjust version numbers accordingly.
3. If the repo has changed so much that this PR's changes are no longer applicable, know that the main changes were skipping the 'VOLUME' command, and running sdb-toolbox-config unregister-host. All the others were workarounds to issues we ran into.
4. Good luck!