Skip to content

Commit

Permalink
Merge pull request #87 from github/merge-upstream-ob
Browse files Browse the repository at this point in the history
Merge upstream ob
  • Loading branch information
Shlomi Noach committed May 18, 2016
2 parents 35038a2 + ed53215 commit 4376b4c
Show file tree
Hide file tree
Showing 12 changed files with 86 additions and 78 deletions.
9 changes: 7 additions & 2 deletions Vagrantfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ BOX = ENV['VAGRANT_BOX'].nil? || ENV['VAGRANT_BOX'].empty? ? 'nrel/CentOS-6.6-x8
VAGRANTFILE_API_VERSION = "2"

system("
if [ #{ARGV[0]} = 'up' ]; then
if [[ #{ARGV[0]} = 'up' ]] && [[ ! -e 'vagrant/vagrant-ssh-key' ]]; then
ssh-keygen -t rsa -b 768 -N '' -q -f vagrant/vagrant-ssh-key
fi
")
Expand All @@ -21,14 +21,19 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
vb.customize ['modifyvm', :id, '--nictype2', 'virtio']
end

config.vm.synced_folder '.', '/orchestrator', type: 'rsync'
config.vm.synced_folder '.', '/orchestrator', type: 'rsync',
rsync__auto: true,
rsync__exclude: ".git/"

(0..4).each do |n|
name = (n > 0 ? ("db" + n.to_s) : "admin")
config.vm.define name do |db|
db.vm.hostname = name
db.vm.network "private_network", ip: "192.168.57.20" + n.to_s
db.vm.provision "shell", path: "vagrant/base-build.sh"
if name == "admin"
db.vm.network "forwarded_port", guest:3000, host:3000
end
end
end
end
33 changes: 17 additions & 16 deletions doc/Orchestrator-Manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -1142,7 +1142,7 @@ The following assumes you have [Executed as web/API service](#executing-as-webap
Open your browser and direct it at `http://your.host:3000`. If all went well, you should see
the following welcome page:

![Orcehstrator screenshot](images/orchestrator-about.png)
![Orchestrator screenshot](images/orchestrator-about.png)

If this is your first time using _orchstrator_, then you should begin by teaching it.
_orchestrator_ needs to know what replication topologies you have. The web interface
Expand All @@ -1169,7 +1169,7 @@ time _orchestrator_ will automaticaly find them.
_Orchestrator_ resolves the `CNAME` of every input it gets, either from the user or from the replication
topology itself. This is for avoiding ambiguities or implicit duplicates.

![Orcehstrator screenshot](images/orchestrator-discover.png)
![Orchestrator screenshot](images/orchestrator-discover.png)

Once _orchestrator_ is familiar with a topology, you can view and manipulate it via the `cluster` page.
Click the `clusters` drop down on navigation bar to see available clusters.
Expand All @@ -1182,7 +1182,7 @@ tree infographic, based on a D3 widget. Sub trees are collapsible.
Each node in the tree presents a single MySQL instance, listing its fully qualified name, its version,
binary log format and replication lag.

![Orcehstrator screenshot](images/orchestrator-simple.png)
![Orchestrator screenshot](images/orchestrator-simple.png)

Note that each server has a _settings_ icon to the right. Clicking this icon opens a modal with some
extra info on that server as well as operations to be performed.
Expand All @@ -1191,7 +1191,7 @@ The modal allows you to begin/terminate maintenance mode on an instance; perform
(by default instances are polled once per minute - this is configurable); stop/start replication; forget
the instance (may be rediscovered a minute later if still connected to the topology).

![Orcehstrator screenshot](images/orchestrator-instance-modal.png)
![Orchestrator screenshot](images/orchestrator-instance-modal.png)

The topology can be refactored: slaves can be moved around via _drag and drop_. Start dragging an instance:
all possible _droppable_ targets are immediately colored green. You may turn your instance to be the slave of
Expand All @@ -1208,37 +1208,37 @@ the operation if it finds a deeper block, such as the target not having binary l

Begin dragging: possible targets colored green

![Orcehstrator screenshot](images/orchestrator-simple-drag.png)
![Orchestrator screenshot](images/orchestrator-simple-drag.png)

Move over your target and drop:

![Orcehstrator screenshot](images/orchestrator-simple-drag-hover.png)
![Orchestrator screenshot](images/orchestrator-simple-drag-hover.png)

Topology refactored:

![Orcehstrator screenshot](images/orchestrator-simple-dropped.png)
![Orchestrator screenshot](images/orchestrator-simple-dropped.png)

Dragging a master over its slave makes for a co-masters (master-master) topology:

![Orcehstrator screenshot](images/orchestator-cm-simple-drag-master.png)
![Orchestrator screenshot](images/orchestator-cm-simple-drag-master.png)

A co-master topology:

![Orcehstrator screenshot](images/orchestator-cm-co-masters.png)
![Orchestrator screenshot](images/orchestator-cm-co-masters.png)

_Orchestrator_ visually indicates replication & accessibility related problems: slave lag, replication not working,
instance not accessed for long time, instance access failure, instance under maintenance.

![Orcehstrator screenshot](images/orchestrator-simple-with-problems.png)
![Orchestrator screenshot](images/orchestrator-simple-with-problems.png)

_Problems_ drop down is available on all pages, and indicates all currently known issues across all topologies:

![Orcehstrator screenshot](images/orchestrator-problems.png)
![Orchestrator screenshot](images/orchestrator-problems.png)

The `Audit` page presents with all actions taken via _orchestrator_: slave move, detection, maintenance etc.
(`START SLAVE` and `STOP SLAVE` are currently not audited).

![Orcehstrator screenshot](images/orchestrator-audit-small.png)
![Orchestrator screenshot](images/orchestrator-audit-small.png)

`Queries` -> `Long queries` page list last met long running queries over the entire topology. these would be
queries running over `60` seconds, non-replication, non-event-scheduler.
Expand Down Expand Up @@ -1292,6 +1292,7 @@ _Orchestrator_ picks best course of action.
is the master of the topology.
* `/api/clusters`: list names of known topologies.
* `/api/clusters-info`: list known clusters (topologies) and basic info
* `/api/cluster-pool-instances/:clusterName`: get pool information
* `/api/search/:searchString`: list instances matching search string
* `/api/problems`: list instances who have known problems (e.g. not replicating, lagging etc.)
* `/api/long-queries`: list of long running queries on all topologies (queries running for over 60 seconds, excluding replication and event-scheduler queries)
Expand Down Expand Up @@ -2168,13 +2169,13 @@ To move slaves via Pseudo-GTID mechanism, click the **Classic mode** green butto
(depicted by a win-glass icon) is eligible for dragging. At this point such a slave can be dropped on an accessible sibling
or ancestor (including its very own master/parent).

![Orcehstrator screenshot](images/orchestrator-pseudo-gtid-dead-relay-master.png)
![Orchestrator screenshot](images/orchestrator-pseudo-gtid-dead-relay-master.png)

![Orcehstrator screenshot](images/orchestrator-pseudo-gtid-dead-relay-master-begin-drag.png)
![Orchestrator screenshot](images/orchestrator-pseudo-gtid-dead-relay-master-begin-drag.png)

![Orcehstrator screenshot](images/orchestrator-pseudo-gtid-dead-relay-master-drop.png)
![Orchestrator screenshot](images/orchestrator-pseudo-gtid-dead-relay-master-drop.png)

![Orcehstrator screenshot](images/orchestrator-pseudo-gtid-dead-relay-master-refactored-1.png)
![Orchestrator screenshot](images/orchestrator-pseudo-gtid-dead-relay-master-refactored-1.png)


## Topology recovery
Expand Down
12 changes: 12 additions & 0 deletions vagrant/admin-build.sh
100644 → 100755
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/bin/bash

# Install orchestrator
rpm -i /tmp/orchestrator-release/orchestrator*.rpm
/sbin/chkconfig orchestrator on
cp /usr/local/orchestrator/orchestrator-sample.conf.json /etc/orchestrator.conf.json
/sbin/service orchestrator start

echo '* * * * * root /usr/bin/orchestrator -c discover -i db1' > /etc/cron.d/orchestrator-discovery

# Discover instances
/usr/bin/orchestrator -c discover -i localhost
44 changes: 35 additions & 9 deletions vagrant/base-build.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,45 @@

if [[ -e /etc/redhat-release ]]; then
# Percona's Yum Repository
sudo yum -y install http://www.percona.com/downloads/percona-release/redhat/0.1-3/percona-release-0.1-3.noarch.rpm
yum -d 0 -y install http://www.percona.com/downloads/percona-release/redhat/0.1-3/percona-release-0.1-3.noarch.rpm epel-release

# All the project dependencies to build plus some utilities
# No reason not to install this stuff in all the places :)
sudo yum -y install Percona-Server-server-56 Percona-Server-shared-56 Percona-Server-client-56 Percona-Server-shared-compat
sudo yum -y install percona-toolkit percona-xtrabackup
yum -d 0 -y install Percona-Server-server-56 Percona-Server-shared-56 Percona-Server-client-56 Percona-Server-shared-compat percona-toolkit percona-xtrabackup ruby-devel gcc rpm-build git vim-enhanced golang
# Pin to 1.4 due to 1.5 no longer working on EL6
gem install fpm --version 1.4

# Build orchestrator and orchestrator agent
mkdir -p /home/vagrant/go/{bin,pkg,src} /tmp/orchestrator-release
mkdir -p /home/vagrant/go/src/github.com/outbrain/orchestrator
mount --bind /orchestrator /home/vagrant/go/src/github.com/outbrain/orchestrator

# Build Orchestrator
export GOPATH=/home/vagrant/go
export GO15VENDOREXPERIMENT=1
cd ${GOPATH}/src/github.com/outbrain/orchestrator
/usr/bin/go get ./...
${GOPATH}/src/github.com/outbrain/orchestrator/build.sh
chown -R vagrant.vagrant /home/vagrant /tmp/orchestrator-release

# Setup mysql
/sbin/chkconfig mysql on

if [[ -e "/orchestrator/vagrant/${HOSTNAME}-my.cnf" ]]; then
rm -f /etc/my.cnf
cp /orchestrator/vagrant/${HOSTNAME}-my.cnf /etc/my.cnf
fi

# All the project dependencies to build
sudo yum -y install ruby-devel gcc rpm-build git
gem install fpm
/sbin/service mysql start
cat <<-EOF | mysql -u root
CREATE DATABASE IF NOT EXISTS orchestrator;
GRANT ALL PRIVILEGES ON orchestrator.* TO 'orc_client_user'@'%' IDENTIFIED BY 'orc_client_password';
GRANT SUPER, PROCESS, REPLICATION SLAVE, RELOAD ON *.* TO 'orc_client_user'@'%';
GRANT ALL PRIVILEGES ON orchestrator.* TO 'orc_client_user'@'localhost' IDENTIFIED BY 'orc_client_password';
GRANT SUPER, PROCESS, REPLICATION SLAVE, RELOAD ON *.* TO 'orc_client_user'@'localhost';
GRANT ALL PRIVILEGES ON orchestrator.* TO 'orc_server_user'@'localhost' IDENTIFIED BY 'orc_server_password';
EOF

# Go (via EPEL)
sudo yum -y install epel-release
sudo yum -y install golang
elif [[ -e /etc/debian_version ]]; then
sudo echo exit 101 > /usr/sbin/policy-rc.d
sudo chmod +x /usr/sbin/policy-rc.d
Expand Down
14 changes: 2 additions & 12 deletions vagrant/db1-build.sh
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,18 +1,8 @@
if [[ -e /etc/redhat-release ]]; then
sudo rm -rf /etc/my.cnf
sudo cp /orchestrator/vagrant/db1-my.cnf /etc/my.cnf
sudo service mysql start
elif [[ -e /etc/debian_version ]]; then
if [[ -e /etc/debian_version ]]; then
sudo cp /orchestrator/vagrant/db1-my.cnf /etc/mysql/my.cnf
sudo /etc/init.d/mysql restart

sudo cat <<-EOF >> /root/.my.cnf
[client]
user = root
password = vagrant
EOF
fi

/usr/bin/mysql -uroot -ss -e 'GRANT REPLICATION SLAVE ON *.* TO "repl"@"192.168.57.%" IDENTIFIED BY "vagrant_repl"'
/usr/bin/mysql -uroot -ss -e 'CHANGE MASTER TO MASTER_HOST="192.168.57.202", MASTER_USER="repl", MASTER_PASSWORD="vagrant_repl"'
/usr/bin/mysql -uroot -ss -e 'CHANGE MASTER TO MASTER_HOST="192.168.57.202", MASTER_USER="repl", MASTER_PASSWORD="vagrant_repl", MASTER_CONNECT_RETRY=10, MASTER_RETRY_COUNT=36'
/usr/bin/mysql -uroot -ss -e 'START SLAVE'
1 change: 1 addition & 0 deletions vagrant/db1-my.cnf
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ symbolic-links=0
server_id=1
log_bin
log-slave-updates
report-host=db1

[mysqld_safe]
log-error=/var/log/mysqld.log
Expand Down
16 changes: 3 additions & 13 deletions vagrant/db2-build.sh
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,18 +1,8 @@
if [[ -e /etc/redhat-release ]]; then
sudo rm -rf /etc/my.cnf
sudo cp /orchestrator/vagrant/db2-my.cnf /etc/my.cnf
sudo service mysql start
elif [[ -e /etc/debian_version ]]; then
sudo cp /orchestrator/vagrant/db2-my.cnf /etc/mysql/my.cnf
if [[ -e /etc/debian_version ]]; then
sudo cp /orchestrator/vagrant/db1-my.cnf /etc/mysql/my.cnf
sudo /etc/init.d/mysql restart

sudo cat <<-EOF >> /root/.my.cnf
[client]
user = root
password = vagrant
EOF
fi

/usr/bin/mysql -uroot -ss -e 'GRANT REPLICATION SLAVE ON *.* TO "repl"@"192.168.57.%" IDENTIFIED BY "vagrant_repl"'
/usr/bin/mysql -uroot -ss -e 'CHANGE MASTER TO MASTER_HOST="192.168.57.201", MASTER_USER="repl", MASTER_PASSWORD="vagrant_repl"'
/usr/bin/mysql -uroot -ss -e 'CHANGE MASTER TO MASTER_HOST="192.168.57.201", MASTER_USER="repl", MASTER_PASSWORD="vagrant_repl", MASTER_CONNECT_RETRY=10, MASTER_RETRY_COUNT=36'
/usr/bin/mysql -uroot -ss -e 'START SLAVE'
1 change: 1 addition & 0 deletions vagrant/db2-my.cnf
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ symbolic-links=0
server_id=2
log_bin
log-slave-updates
report-host=db2

[mysqld_safe]
log-error=/var/log/mysqld.log
Expand Down
16 changes: 3 additions & 13 deletions vagrant/db3-build.sh
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,18 +1,8 @@
if [[ -e /etc/redhat-release ]]; then
sudo rm -rf /etc/my.cnf
sudo cp /orchestrator/vagrant/db3-my.cnf /etc/my.cnf
sudo service mysql start
elif [[ -e /etc/debian_version ]]; then
sudo cp /orchestrator/vagrant/db3-my.cnf /etc/mysql/my.cnf
if [[ -e /etc/debian_version ]]; then
sudo cp /orchestrator/vagrant/db1-my.cnf /etc/mysql/my.cnf
sudo /etc/init.d/mysql restart

sudo cat <<-EOF >> /root/.my.cnf
[client]
user = root
password = vagrant
EOF
fi

/usr/bin/mysql -uroot -ss -e 'GRANT REPLICATION SLAVE ON *.* TO "repl"@"192.168.57.%" IDENTIFIED BY "vagrant_repl"'
/usr/bin/mysql -uroot -ss -e 'CHANGE MASTER TO MASTER_HOST="192.168.57.201", MASTER_USER="repl", MASTER_PASSWORD="vagrant_repl"'
/usr/bin/mysql -uroot -ss -e 'CHANGE MASTER TO MASTER_HOST="192.168.57.201", MASTER_USER="repl", MASTER_PASSWORD="vagrant_repl", MASTER_CONNECT_RETRY=10, MASTER_RETRY_COUNT=36'
/usr/bin/mysql -uroot -ss -e 'START SLAVE'
1 change: 1 addition & 0 deletions vagrant/db3-my.cnf
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ symbolic-links=0
server_id=3
log_bin
log-slave-updates
report-host=db3

[mysqld_safe]
log-error=/var/log/mysqld.log
Expand Down
16 changes: 3 additions & 13 deletions vagrant/db4-build.sh
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,18 +1,8 @@
if [[ -e /etc/redhat-release ]]; then
sudo rm -rf /etc/my.cnf
sudo cp /orchestrator/vagrant/db4-my.cnf /etc/my.cnf
sudo service mysql start
elif [[ -e /etc/debian_version ]]; then
sudo cp /orchestrator/vagrant/db4-my.cnf /etc/mysql/my.cnf
if [[ -e /etc/debian_version ]]; then
sudo cp /orchestrator/vagrant/db1-my.cnf /etc/mysql/my.cnf
sudo /etc/init.d/mysql restart

sudo cat <<-EOF >> /root/.my.cnf
[client]
user = root
password = vagrant
EOF
fi

/usr/bin/mysql -uroot -ss -e 'GRANT REPLICATION SLAVE ON *.* TO "repl"@"192.168.57.%" IDENTIFIED BY "vagrant_repl"'
/usr/bin/mysql -uroot -ss -e 'CHANGE MASTER TO MASTER_HOST="192.168.57.202", MASTER_USER="repl", MASTER_PASSWORD="vagrant_repl"'
/usr/bin/mysql -uroot -ss -e 'CHANGE MASTER TO MASTER_HOST="192.168.57.202", MASTER_USER="repl", MASTER_PASSWORD="vagrant_repl", MASTER_CONNECT_RETRY=10, MASTER_RETRY_COUNT=36'
/usr/bin/mysql -uroot -ss -e 'START SLAVE'
1 change: 1 addition & 0 deletions vagrant/db4-my.cnf
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ symbolic-links=0
server_id=4
log_bin
log-slave-updates
report-host=db4

[mysqld_safe]
log-error=/var/log/mysqld.log
Expand Down

0 comments on commit 4376b4c

Please sign in to comment.