MARS (Multiversion Asynchronous Replicated Storage) Installation Tutorial
This is an effort to document MARS' installation process using Ubuntu.
You can find information about the installation process in at least 3 places:
This document merges all this information plus additional research that was required to make MARS work using Ubuntu 14.04. Future documentation will also include details on how to perform the installation process on Ubuntu 16.04.
- Read the manual. Seriously.
- MARS only runs on a 64-bit compatible OS. I highly recommend doing this tutorial using Ubuntu 14.04 the first time because that will increase your chances of success. After getting MARS up and running you can then try a newer OS version or even a different OS when you understand the basics.
- You will need a host to recompile the kernel, this tutorial assumes your host already has all the utilities required to perform this task. Moreover, the kernel will be built using the old fashioned way method.
- You will need an extra partition to host the data that will be replicated by MARS. This means at the very minimum, you will need the default partition to host your OS and an additional partition to host your MARS data.
1. Determine which kernel version you are going to use and install it
MARS requires patching the kernel sources prior to compilation, these patches are available for several versions of the kernel but not all of them. Ubuntu 14.04 in particular comes by default with a kernel version (3.13) that is not listed among this list which is why it is highly recommended to patch the OS with a supported version prior to performing any other action.
To install a new kernel version in Ubuntu, you need to execute the following command:
sudo apt-get install linux-generic-lts-[kernel first name]
An updated list of Ubuntu Kernel names is available here. Just look up the kernel version you want to use and replace [kernel first name] with the actual name you want. For instance, say you want to install kernel "xenial xerus" (4.4), then the command you will have to execute would be:
sudo apt-get install linux-generic-lts-xenial
Then make sure you reboot your server to ensure your new kernel is now being used.
2. Download your kernel's source code for recompilation
You are going to need a working folder, this guide will assume your working directory is going to be /home/myuser from now on.
Create a new folder under /home/myuser named "kernel" and from that folder run the following command:
apt-get source linux-image-$(uname -r)
This command will download the source code for the kernel that's currently running on your host (thus the importance of performing a reboot prior to executing this step).
If everything goes well, you will end up with a structure similar to this:
/home/myuser/kernel/linux-lts[kernel version]
3. Patch your kernel with MARS' pre-patch files
Clone MARS' repo into your /home/myuser/kernel/linux-lts[kernel version]/block folder (make sure you perform the clone from the right folder, you don't want to clone your files on the wrong folder)
git clone https://github.com/schoebel/mars
And now you need to patch your kernel source with the right files. This action needs to be performed from the /home/myuser/kernel/linux-lts[kernel version] folder
patch -p1 < /home/myuser/kernel/linux-[kernel version]/block/mars/pre-patches/vanilla-[kernel version]/[patch name].patch
Just make sure you apply all the patches listed there.
4. Build your new kernel packages
First, you are going to enable MARS prior to performing the compilation, just execute the following command:
make oldconfig
Just make sure that when you get this prompt "storage system MARS (EXPERIMENTAL) (MARS) [N/m/y/?] (NEW)" you type m. You can simply press ENTER on each prompt to keep the default values.
Now you are ready to actually compile your new kernel and generate the .deb files you need to patch your existing kernel:
make deb-pkg
At the end of this process and if everything goes well, you will see some new .deb files under /home/myuser/kernel.
5. Copy your new kernel into all your MARS hosts
Create a new folder into all of your MARS hosts
mkdir -p /home/myuser/kernel
Copy the following files into your newly created folders on all of your MARS hosts
/home/myuser/kernel/linux-headers-[kernel version]_amd64.deb
/home/myuser/kernel/linux-image-[kernel version]_amd64
/home/myuser/kernel/linux-[kernel version]/block/mars/userspace/marsadm
1. Log into all of your MARS host servers and install your new kernel on each of them
dpkg -i /home/myuser/kernel/*.deb
Make sure you reboot all of your MARS hosts after completing the kernel update.
2. Install the marsadm utility
cp /home/myuser/kernel/marsadm /usr/local/bin && chmod +x /usr/local/bin/marsadm
3. Enable SSH root login
You will have to select one of your MARS hosts as your "Node A" (the first node on your cluster). Your other MARS hosts will require remote SSH access using their local root account.
The first step is to enable SSH root login on your Node A, you need to manually open the SSHD configuration file /etc/ssh/sshd_config within your Node A and change these lines
FROM:
PermitRootLogin prohibit-password
PermitRootLogin no
TO:
PermitRootLogin yes
PermitRootLogin yes
Then reload the configuration file to apply your changes
service ssh reload
4. SSH Login Without Password
Make sure you execute these commands using your root account. You will have to repeat these instructions in all of your MARS hosts except from your Node A. Execute the following command and press ENTER on each prompt
ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user/.ssh/id_rsa.
Your public key has been saved in /home/user/.ssh/id_rsa.pub.
The key fingerprint is:
b2:ad:a0:80:85:ad:6c:16:bd:1c:e7:63:4f:a0:00:15 user@host
The key's randomart image is:
+--[ RSA 2048]----+
| E. |
| . |
|. |
|.o. |
|.ooo o. S |
|oo+ * .+ |
|++ +.+... |
|o. ...+. |
| . .. |
+-----------------+
Then copy your newly created key into your Node A
ssh-copy-id root@[Node A]
Finally log into your Node A from your current MARS host
ssh root@[Node A]
This section assumes you have a /dev/sdb disk available to create the new block device. More information on how to create block devices can be found here. If you want to learn more on how to create partition then you will find more information here.
1. Create a new MBR partition
Start fdisk using the following command:
fdisk /dev/sdb
In fdisk, to create a new partition, type the following command:
n
- When prompted to specify the Partition type, type p to create a primary partition or e to create an extended one. There may be up to four primary partitions. If you want to create more than four partitions, make the last partition extended, and it will be a container for other logical partitions.
- When prompted for the Number, in most cases, type 3 because a typical Linux virtual machine has two partitions by default.
- When prompted for the Start cylinder, type a starting cylinder number or press Return to use the first cylinder available.
- When prompted for the Last cylinder, press Return to allocate all the available space or specify the size of a new partition in cylinders if you do not want to use all the available space.
Write your changes:
w
2. Create a new physical volume
pvcreate /dev/sdb1
3. Create a new volume group
vgcreate vol_grp_myspace /dev/sdb1
4. Create the logical volume
lvcreate --name myspace --extents 100%FREE vol_grp_myspace
5. Format the new volume
mkfs.ext4 /dev/vol_grp_myspace/myspace
1. Create a new folder for the transaction logs
MARS uses the folder /mars to store its transaction logs, it is a good idea to mount a different partition (dedicated disk) into this folder, but in this tutorial you will simply create a new folder in every MARS host
mkdir /mars
2. Create a new MARS cluster
Execute the following command in the Node A only
marsadm create-cluster
3. Join the rest of the cluster nodes to your newly created MARS cluster
marsadm join-cluster [Node A]
4. Load the MARS kernel module
Execute the following command on all of your MARS cluster nodes
modprobe mars
5. Check if the cluster is healthy
If the cluster is healthy, you will see communication going back and forth using port 7777
netstat --tcp | grep 7777
1. Create a new primary resource
In order to create a new cluster, execute the following command ONE TIME ONLY on your Node A
marsadm create-resource mydata /dev/vol_grp_myspace/myspace
As a result, a directory /mars/resource-mydata/ will be created on Node A, containing some symlinks. Node A will automatically start in the primary role for this resource. Therefore, a new pseudo-device /dev/mars/mydata will also appear after a few seconds. Note that the initial contents of /dev/mars/mydata will be exactly the same as in your pre-existing disk /dev/vol_grp_myspace/myspace.
2. Create a new folder to mount the volume
mkdir /mydata
3. Mount the new volume
mount /dev/mars/mydata /mydata
At this point you can access the contents of your disk using the mountpoint /mydata.
1. Join a primary resource
marsadm join-resource mydata /dev/vol_grp_myspace/myspace
2. Check all of your nodes to ensure everything is OK
marsadm view mydata