Reduce Heimdall app container size to 600MB (#64)

* Fix LDAP in CI tools

* Update README.md to fix formatting and authors

* gitlab CI for swarm

* Remove Internal Configs

* Patch bug where wrong Dockerfile is getting internal config inserted

* Shrink heimdall docker image to 600MB

.dockerignore

@@ -1,5 +1,4 @@
 .git
 .idea
 .DS_Store
-Gemfile.lock
 dockerfiles

.gitignore

@@ -14,6 +14,7 @@
 config/secrets.yml
 config/ldap.yml
 /public/uploads
+inspec-tools/
 
 rails_best_practices_output.html
 
@@ -49,7 +50,6 @@ build-iPhoneSimulator/
 
 # for a library or gem, you might want to ignore these files since the code is
 # intended to run in multiple environments; otherwise, check them in:
-Gemfile.lock
 .ruby-version
 .ruby-gemset
 

Gemfile

@@ -62,6 +62,7 @@ gem 'docsplit'
 gem 'pdftotext'
 gem 'roo'
 
+
 group :development, :test do
   # Call 'byebug' anywhere in the code to stop execution and get a debugger console
   gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]

README.md

@@ -50,26 +50,32 @@ Heimdall to perform most actions** You may view ldap.example.yml for how
 authentication of people's internal email addresses works with a LDAP server
 which allows anonymous access.
 
-##### Automated Build Steps
-1. Run the following commands from base folder (where it is located):
-	1. `./gen-secrets.sh ` (Generate Random keys to be stored in a named Docker volume **Do not run if you've ever run it before**)
-   2. `./docker_build.sh` (may need to first run `chmod +x docker_build.sh` to give the file executable rights)
+
+#### Automated Build Steps
+1. Run the following commands from a terminal:
+	1. `git clone https://github.com/aaronlippold/heimdall.git && cd heimdall` # download heimdall and change to it's directory
+	2. `./gen-secrets.sh ` # (Generate Random keys to be stored in a named Docker volume **Do not run if you've ever run it before**)
+  3. `./docker_build.sh` # (may need to first run `chmod +x docker_build.sh` to give the file executable rights)
 2. Jump to [Running Docker Container](#running-docker-container)
 
 ##### Manual Build Steps
 1. Install Docker
-2. Navigate to the base folder where `docker-compose.yml` is located
-3. Run the following command in a terminal window from the heimdall source directory:
+2. Clone this repository
+	* `git clone https://github.com/aaronlippold/heimdall.git`
+3. Navigate to the base folder where `docker-compose.yml` is located
+4. Run the following command in a terminal window from the heimdall source directory:
+   * `git clone https://github.com/aaronlippold/inspec-tools.git`  
+5. Run the following command in a terminal window from the heimdall source directory:
    * `docker-compose build`  
-4. Generate keys for secrets.yml. Use secrets.example.yml for a template.
+6. Generate keys for secrets.yml. Use secrets.example.yml for a template.
 	_Internally we generate it with the shell script `./gen-secrets.sh` Which
 	creates a named volume which is symlinked to config/secrets.yml at runtime.
 	If you are deploying this container to a docker swarm please use docker
 	secrets as it is far more secure than a named volume._
-5. Run one of the following commands in a terminal window from the heimdall source directory:
+7. Run one of the following commands in a terminal window from the heimdall source directory:
 	* `docker-compose run web rake db:reset` **This destroys and rebuilds the db**
 	* `docker-compose run web rake db:migrate` **This updates the db**
-6. Jump to [Running Docker Container](#running-docker-container)
+8. Jump to [Running Docker Container](#running-docker-container)
 
 
 ##### Running Docker Container

docker_build.sh

@@ -4,6 +4,11 @@ set -xe
 
 # build db and web if needed
 docker-compose build
+
+# Attempt db setup
+docker-compose run web bundle exec rake db:create >/dev/null 2>/dev/null
+docker-compose run web bundle exec rake db:setup >/dev/null 2>/dev/null 
+docker-compose run web bundle exec rake db:seed
 # update db state
-docker-compose run web rake db:migrate
+docker-compose run web bundle exec rake db:migrate
 

dockerfiles/heimdall/Dockerfile

@@ -1,29 +1,54 @@
-FROM ruby:2.4.4
-RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs
+# Use alpine 3.5 as it is the most recent alpine with an old enough imagemagick
+FROM alpine:3.5
+
+# sed is build only should be removed at TODO-remove-at-release
+RUN apk --no-cache update && apk --no-cache  --update add ruby ruby-irb ruby-json ruby-rake \
+    ruby-bigdecimal ruby-io-console libstdc++ tzdata nodejs sed libressl \
+	 libxml2 'imagemagick<7.0.0.0'
 
 ENV RAILS_ROOT /var/www/heimdall
 
 RUN mkdir -p $RAILS_ROOT/tmp/pids
 
 WORKDIR $RAILS_ROOT
 
-ADD Gemfile Gemfile
+# TODO-remove-at-release
+COPY inspec-tools/ inspec-tools/
+
+COPY Gemfile Gemfile
+
+# Edit inspec_tools to use a locally downloaded gem
+RUN sed -i"" -e 's#.inspec_tools.*:git.*$#"inspec_tools", :path => "./inspec-tools"#g' Gemfile
+
+# Ensure we never install docs
+RUN echo "gem: --no-rdoc --no-ri" >> ~/.gemrc
 
-RUN gem install bundler && bundle install --jobs 20 --retry 5
+# Install and remove build dependencies. Single line implies single layer so
+# the final image does not have any of the build dependencies. TODO use
+# --deployment once Gemfile.lock has been commited
+RUN apk --no-cache --update add --virtual build-dependencies build-base ruby-dev  \
+    postgresql-dev libc-dev linux-headers git libxml2-dev 'imagemagick-dev<7.0.0.0' pkgconf && \
+    gem install bundler --no-rdoc --no-ri && \
+    bundle install --retry 5 --no-cache --jobs 20 --without development test && \
+    apk del build-dependencies
 
 # Deploy production server to container
 ARG RAILS_ENV=production
 ARG RAILS_RELATIVE_URL_ROOT=/heimdall
 
-ADD . .
-
+COPY . .
 RUN mv config/mongoid.yml config/mongoid.yml.orig
 RUN mv config/mongoid.yml.docker config/mongoid.yml
 
+# Edit inspec_tools to use a locally downloaded gem
+RUN sed -i"" -e 's#.inspec_tools.*:git.*$#"inspec_tools", :path => "./inspec-tools"#g' Gemfile
+
 # precompile is only necessary for production builds
 RUN cp config/secrets.example.yml config/secrets.yml
-RUN bash -c "RAILS_ENV=$RAILS_ENV RAILS_RELATIVE_URL_ROOT=$RAILS_RELATIVE_URL_ROOT SECRET_KEY_BASE=$(openssl rand -hex 64) bundle exec rake assets:precompile"
+RUN sh -c "RAILS_ENV=$RAILS_ENV RAILS_RELATIVE_URL_ROOT=$RAILS_RELATIVE_URL_ROOT SECRET_KEY_BASE=$(openssl rand -hex 64) bundle exec rake assets:precompile"
 RUN rm config/secrets.yml
+
+# Setup broken symlink, which fixed at run time by a volume mount
 RUN mkdir -p /srv/secrets/
 RUN touch /srv/secrets/secrets.yml
 RUN ln -s /srv/secrets/secrets.yml config/secrets.yml