-
Notifications
You must be signed in to change notification settings - Fork 0
/
INSTALL
111 lines (80 loc) · 3.98 KB
/
INSTALL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
You need the kernel sources for at least 2.6.32. Some elder versions
may work, but we have not tested it.
Currently, MARS Light builds only with 64bit kernels.
Go to your linux source tree, and apply one of the pre-patches for MARS.
You can find them in the subdirectory pre-patches/ of the MARS sources.
These pre-patches are almost trivial, they contain only a few
EXPORT_SYMBOL() statements. Porting to other kernel versions should
be trivial.
One of the pre-patches has been ported to an openvz kernel
even by one of our sysadmins, who usually does no C programming.
So this should be no major hurdle.
Because of the need for some small pre-patches, we recommend to build
MARS inside the kernel source tree (in-tree).
If you are an experience sysadmin and want to build MARS as an external
kernel module, you may try the Debian based scripts provided by our
sysadmins. Be warned that you have to check yourself for source
compatibility, the right compiler flags, etc; very nasty errors may
appear if you fail to do so!
Therefore, try the in-tree method first.
For the in-tree build, you need to also apply one of the patches
pre-patches/vanilla-*/*necessary-for-in-tree-build.patch
or similar.
Go to ${your_kernel_source}/block/ and clone the MARS git repository there.
Then build your kernel as usual. You need to use 64bit. In addition, you
need to enable the following options before MARS shows up at all:
PROC_SYSCTL
HIGH_RES_TIMERS
Don't enable CONFIG_DEBUG_SG (this will currently not work until it is fixed).
In Kconfig, you will find lots of additional options for MARS. Most
of them should be left at their default. It suffices just to switch
on MARS as a whole, and let it build as a single kernel module.
If no Kconfig options for MARS show up in the in-tree build, you have
probably forgotten to apply the appropriate
pre-patches/vanilla-*/*necessary-for-in-tree-build.patch
or similar.
Finally, copy userspace/marsadm to some appropriate location in
your $PATH.
Do the following at both your primary and secondary node:
After booting your pre-patched kernel, don't modprobe mars. Before
that, create an empty filesystem with at least 100GB (ext4 is highly
recommended; there seem to remain some recursion deadlock problems
with xfs which will be hopefully fixed in the next time)
and mount it to /mars/ .
Additionally, you need an empty block device having exactly the same
size at both nodes. In the following, they are called /dev/vg-x/myspace .
On the primary:
marsadm create-cluster
On the secondary:
marsadm join-cluster ${hostname_of_primary}
Only after that, do on both nodes:
modprobe mars
On the primary:
marsadm create-resource myspace /dev/vg-x/myspace
Wait a few seconds until the state information about the new resource has
spread over the whole cluster.
On the secondary:
marsadm join-resource myspace /dev/vg-x/myspace
Shortly after that, the initial full sync should start automatically.
On the primary, a device /dev/mars/myspace should appear, having exactly
the same size as /dev/vg-x/myspace .
Now you can use /dev/mars/myspace on the primary for creating a filesystem,
mounting, or exporting via iSCSI, etc.
From time to time, you should execute the following commands on one of
your nodes:
marsadm log-rotate all
sleep 10
marsadm log-delete-all all
... in order to prevent your /mars/ filesystem from running full.
hint: use cron jobs for automation.
Most marsadm commands are very similar to drbdadm. Details can be
found in docu/mars-user-manual.pdf. The sourcecode of marsadm is a very
simple and stupid perl script, which intentionally does not use any
perl module and no OO. The source code will tell you almost anything
about the symlinks present in /mars/.
If you are curious about how MARS replicates its state information
over the network, just do the following on both nodes:
watch ls -l /mars/resource-myspace/
Alternatively / additionally, you may try Joerg's script mars-status.pl
which will deliver colorful state reports from the practical viewpoint
of an experienced sysadmin.