Add docs on how to build your own pre-provisioned base box

[oxyc]

Issue Type

• Documentation Report

Summary

My laptop just ran out of space (71 GiB ~/VirtualBox VMs) so I thought it might be a good idea to build a pre-provisioned box and leverage linked clones. As Drupal VM is so awesome for teams, might this be something worth adding to the docs?

I've never used packer or looked into building boxes, but it looks easy enough and quite useful. I know other co-workers have complained about the VM disk usage..

[geerlingguy]

"But 512 K is enough for everybody, right?" /s

In all seriousness, though... basically something like "how to build and host your own derivative box" would be a decent addition. Or if not that, at least a blog post, akin to one I did a while back about Packer + Ansible: Building VM images with Ansible and Packer.

Or more specifically, how to optimize a project for your team even further than something like Beetbox.

[oxyc]

Thanks for the link! Currently I'm just testing it out by building our boilerplate project together with a post-provision task that runs:

#!/bin/bash

rm -f /home/vagrant/.ssh/known_hosts
apt-get clean
dd if=/dev/zero of=/EMPTY bs=1M
rm -f /EMPTY
cat /dev/null > ~/.bash_history

config.yml

build_composer_project: false
install_site: false
vagrant_synced_folders: []
nginx_hosts: []
apache_vhosts: []
mysql_databases: []
mysql_users: []
postgresql_databases: []
postgresql_users: []
post_provision_scripts:
  - "{{ config_dir }}/box-cleanup.sh"

And then

vagrant package --output mybox.box

Everyone at work uses VirtualBox so I guess as a really simple solution, this should work.

[oxyc]

The simple solution works. To make it maintainable I guess I do need packer though.

[oxyc]

For packer I guess the easiest is to fork one of your box repositories and configuring it to run Drupal VM's playbook with the instructions from Deploying Drupal VM to a production environment.

@geerlingguy feel free to close this if you think it's better suited as a blog post or in general too far from the common use case.

[geerlingguy]

Not necessarily... it might still be suited for a blog post, but we can explore the topic in this issue a bit more before closing it out.

For months, I've been wanting to do basically what you've suggested:

fork one of your box repositories and configuring it to run Drupal VM's playbook with the instructions from Deploying Drupal VM to a production environment

But make it into a digestible and as-simple-as-possible process so it's really easy for people to do it on their own.

It would also alleviate some of my bandwidth, for which I'm now up to about $25/month just to host my boxes 😳

[thom8]

@oxyc take a look @ the minimize script in the boxcutter project - https://github.com/boxcutter/ubuntu/blob/master/script/minimize.sh

This massively reduced the size of the beetbox base image -- https://atlas.hashicorp.com/beet/boxes/box (was > 800MB)

@geerlingguy maybe this could be added to your build?

[thom8]

Also just to explain where beetbox heading RE: "preprovisioned profiles" which could provide a bunch of preprovisioned base boxes eg. PHP7, Acquia, Platform.sh etc.

We've added the concept of a profile which extends the default config so maintenance is reduced.

This is set by BEET_PROFILE /beet_profile so our docker container uses the docker profile.
The idea going forward is to include more profiles and add more boxes, but first I'd like to get a full replica docker build which will help for CI and eventually plan to make it a docker first project which also works with Vagrant so each base box would also have a corresponding container.

Maybe even a Centos / RHEL box/container...

It's not much overhead to manage as each box is automatically built on a new tag & thanks to Hashicorp we've been sponsored a Vagrant Enterprise account.

These profiles could also potentially be set inside composer.json --

beetboxvm/beetbox#364 (comment)
beetboxvm/beetbox#371

[oxyc]

@thom8 nice thanks for the resources!

We've added the concept of a profile which extends the default config so maintenance is reduced.

Drupal VM's DRUPALVM_ENV variable could be used in a similar way. We'd just need to set it to docker and include a docker.config.yml file.

[geerlingguy]

@thom8 - The Ansible packer roles for Debian/RHEL do clean up all the packages they installed for configuration (or at least they should), and the minimal base box images are already as close to barebones as I can get them. I think I gave up doing things like removing all man pages and such, just because my base boxes are meant to emulate the actual minimal OS installs, and they still include things like man pages.

[oxyc]

A bit related. But you can now use beet/box for faster provisions. It's essentially a pre-provisioned Drupal VM box. acquia/blt#1986

[thom8]

@oxyc this has actually always worked, the difference described in that issue is that all of BLT's default config is now part of the preprovisioning.

Even, before the changes it still worked as it would just provision anything that wasn't already included.

[oxyc]

I just didn't think of doing that before you mentioned it in the issue queue. Once linked clones are working again I'll give it a try and see how it compares.

[thom8]

All good, the reverse is also true beetbox using geerlingguy/ubuntu1604 as the base box.

vagrant_box: geerlingguy/ubuntu1604
vagrant_box_version: "~> 1.1.0"

even the default ubuntu xenial64 should work >>

vagrant_box: ubuntu/xenial64
vagrant_box_version: ">= 0"

[oxyc]

One annoyance with beet/box on Drupal VM with vagrant-vbguest is that the pre-installed VirtualBox Guest Additions are older than the ones I have on the host. With @geerlingguy boxes the box almost always have a newer version than I do. Thus when I update the box, the guest additions iso is updated on my host and then used on all new VMs until I run vagrant box update again. Edit: If I always update my boxes at the same time as VirtualBox they will be in sync for all new VM's until the next update.

With beet/box the vagrant up takes quite some time because it needs to update the VM's guest additions rather than the iso on the host.

For reference beet/box v0.7.4 (latest) has Guest Additions v5.0.40 while geerlingguy/ubuntu1604 v1.1.4 (5 months old) has Guest Additions v5.1.22

[oxyc]

@thom8 is that a bug or?

[thom8]

@oxyc it's just using the default version on atlas -- https://github.com/beetboxvm/beetbox/blob/master/provisioning/ubuntu-16.04.json#L56

I don't use vagrant-vbguest so don't run into this issue and haven't had any issues with the old versions.

Looks like packer now supports guest_additions_url -- https://www.packer.io/docs/builders/virtualbox-iso.html#guest_additions_url but would have to manually keep this updated..

[oxyc]

Alright, it's a bit annoying with the plugin installed. Checked some more and it's just tied to the installed version of VirtualBox.

Guess it's just worked well for me with geerlingguy/ubuntu1604 because anytime I update VirtualBox I'm usually on an update mission bumping all the things and geerlingguy/ubuntu1604 almost always has the latest version.

[thom8]

happy to add the latest version to the Atlas build -- beetboxvm/beetbox#446

[thom8]

@oxyc looks like it's working with vagrant-vbguest on the latest release beet/dev, will be rolled into the next release 0.8.0 later in the week.

==> beetbox: Machine booted and ready!
[beetbox] GuestAdditions 5.1.28 running --- OK.
==> beetbox: Checking for guest additions in VM...

[thom8]

@oxyc pushed 0.8.0 yesterday, mainly a clean up release and everything looks good.

Also publishing link to build --> https://github.com/beetboxvm/beetbox/releases/tag/0.8.0

Which should help with any debugging, wish I could make the Atlas build public... but happy to give you access if you're interested

[geerlingguy]

Also related:

• Build and push Drupal VM base box to Vagrant Cloud, and use it as default #1774 (see, specifically, the Packer build for Drupal VM here: https://github.com/geerlingguy/packer-drupal-vm
• Can we increase the filesystem on the VM #1844

[geerlingguy]

I'm going to close this issue and move further work into #1774 — basically to document how the Drupal VM base box is built, and how you can customize that further.