-
Notifications
You must be signed in to change notification settings - Fork 6
Home
Welcome to the VirtualMesos wiki!
VirtualMesos is ultimately intended as a patch to the Mesos project to allow isolation containers based on virtual machines. This patch works with the KVM virtual machine manager. I've placed the relevant code in a branch called mesos-vm. The code is based upon the mesos lxc isolation module that is located in src/slave/lxc_isolation_module.cpp/.hpp. My code based on the mesos master branch dated
Wed Nov 23 12:02:07 2011 -0800
,
commit 059aabb2ec5bd7b20ed08ab9c439531a352ba3ec
. Most of the implementation is contained in vm_isolation_module.cpp and vm_isolation_module.hpp. I use the libvirt library, to manage the virtual machine container in which the jobs are executed.
The code has been tested on Ubuntu 11.04 and 11.10 and depends on
libpython2.6
libvirt0
The virtual machine invocation depends upon a few configuration assumptions: ssh public keys installed on the container. I assume that the container is setup to allow password less secure access. Directory structure on the container matches the servant machine. For example, in invoking a spark executor, assume that the paths match the setup on the container host.
In the $MESOS_HOME/conf/mesos.conf file add the line
isolation=vm
to use the virtual machine isolation. The Mesos slave is invoked with the
isolation
parameter set to
vm
. For example
sudo bin/mesos-slave -m mesos://master@mesos-host:5050 -w 9839 --isolation=vm
The vm_isolation_module class forks a process that in turn launches a virtual machine. A routine find_addr.pl located in the bin directory is responsible for figuring out the IP address of the launched virtual machine. This is probably not portable since it is explicitly looking for entry in the virbr0 network.
A script vmLauncherTemplate.sh located in bin assists the the vmLauncher method to setup the environment for launching tasks inside of the virtual machine. The vmLauncher method uses vmLauncherTemplate.sh to create a tasks specific shell vmLauncherTemplate-<task_id>.sh, which is copied to the running guest and used to run the executor inside the VM. This communicates with the slave on the host.