Skip to content

Latest commit

 

History

History
154 lines (101 loc) · 5.78 KB

vagrant.md

File metadata and controls

154 lines (101 loc) · 5.78 KB
Raw

A local Cluster with Vagrant

This document shows you how to configure a complete cluster using a virtual machine. This setup replicates a real cluster in your development machine as closely as possible. After you complete the steps outlined here, you will be ready to create and run your first Aurora job.

The following sections describe these steps in detail:

  1. Overview
  2. Install VirtualBox and Vagrant
  3. Clone the Aurora repository
  4. Start the local cluster
  5. Log onto the VM
  6. Run your first job
  7. Rebuild components
  8. Shut down or delete your local cluster
  9. Troubleshooting

Overview

The Aurora distribution includes a set of scripts that enable you to create a local cluster in your development machine. These scripts use Vagrant and VirtualBox to run and configure a virtual machine. Once the virtual machine is running, the scripts install and initialize Aurora and any required components to create the local cluster.

Install VirtualBox and Vagrant

First, download and install VirtualBox on your development machine.

Then download and install Vagrant. To verify that the installation was successful, open a terminal window and type the vagrant command. You should see a list of common commands for this tool.

Clone the Aurora repository

To obtain the Aurora source distribution, clone its Git repository using the following command:

 git clone git://git.apache.org/aurora.git

Start the local cluster

Now change into the aurora/ directory, which contains the Aurora source code and other scripts and tools:

 cd aurora/

To start the local cluster, type the following command:

 vagrant up

This command uses the configuration scripts in the Aurora distribution to:

  • Download a Linux system image.
  • Start a virtual machine (VM) and configure it.
  • Install the required build tools on the VM.
  • Install Aurora's requirements (like Mesos and Zookeeper) on the VM.
  • Build and install Aurora from source on the VM.
  • Start Aurora's services on the VM.

This process takes several minutes to complete.

You may notice a warning that guest additions in the VM don't match your version of VirtualBox. This should generally be harmless, but you may wish to install a vagrant plugin to take care of mismatches like this for you:

 vagrant plugin install vagrant-vbguest

With this plugin installed, whenever you vagrant up the plugin will upgrade the guest additions for you when a version mis-match is detected. You can read more about the plugin here.

To verify that Aurora is running on the cluster, visit the following URLs:

Log onto the VM

To SSH into the VM, run the following command in your development machine:

 vagrant ssh

To verify that Aurora is installed in the VM, type the aurora command. You should see a list of arguments and possible commands.

The /vagrant directory on the VM is mapped to the aurora/ local directory from which you started the cluster. You can edit files inside this directory in your development machine and access them from the VM under /vagrant.

A pre-installed clusters.json file refers to your local cluster as devcluster, which you will use in client commands.

Run your first job

Now that your cluster is up and running, you are ready to define and run your first job in Aurora. For more information, see the Aurora Tutorial.

Rebuild components

If you are changing Aurora code and would like to rebuild a component, you can use the aurorabuild command on the VM to build and restart a component. This is considerably faster than destroying and rebuilding your VM.

aurorabuild accepts a list of components to build and update. To get a list of supported components, invoke the aurorabuild command with no arguments:

 vagrant ssh -c 'aurorabuild client'

Shut down or delete your local cluster

To shut down your local cluster, run the vagrant halt command in your development machine. To start it again, run the vagrant up command.

Once you are finished with your local cluster, or if you would otherwise like to start from scratch, you can use the command vagrant destroy to turn off and delete the virtual file system.

Troubleshooting

Most of the Vagrant related problems can be fixed by the following steps:

  • Destroying the vagrant environment with vagrant destroy
  • Killing any orphaned VMs (see AURORA-499) with virtualbox UI or VBoxManage command line tool
  • Cleaning the repository of build artifacts and other intermediate output with git clean -fdx
  • Bringing up the vagrant environment with vagrant up

If that still doesn't solve your problem, make sure to inspect the log files:

  • Scheduler: /var/log/upstart/aurora-scheduler.log
  • Observer: /var/log/upstart/aurora-thermos-observer.log
  • Mesos Master: /var/log/mesos/mesos-master.INFO (also see .WARNING and .ERROR)
  • Mesos Agent: /var/log/mesos/mesos-slave.INFO (also see .WARNING and .ERROR)