Skip to content
This repository has been archived by the owner on Aug 23, 2023. It is now read-only.

Add tool for cross-cluster split-metric migrations #529

Merged
merged 2 commits into from
Mar 20, 2017

Conversation

replay
Copy link
Contributor

@replay replay commented Feb 13, 2017

fixes #528

This tool can be used like shown here:

mst@ubuntu:~/go/src/github.com/raintank/metrictank/cmd/mt-split-metrics-by-ttl$ ./cross_cluster_tool.py --destination-address 192.168.56.101 --temp-dir /tmp/met/ 840 8640 
nodetool snapshot -t migration-1 --table metrics metrictank &&
mkdir -p /tmp/met &&
ln -s /var/lib/cassandra/data/metrictank/metrics/snapshots/migration-1 /tmp/met/metrictank/metrics_512 &&
ln -s /var/lib/cassandra/data/metrictank/metrics/snapshots/migration-1 /tmp/met/metrictank/metrics_8192 &&
sstableloader -d 192.168.56.101 /tmp/met/metrictank/metrics_512 &&
sstableloader -d 192.168.56.101 /tmp/met/metrictank/metrics_8192
mst@ubuntu:~/go/src/github.com/raintank/metrictank/cmd/mt-split-metrics-by-ttl$ ./cross_cluster_tool.py --destination-address 192.168.56.101 --temp-dir /tmp/met/ 840 8640 > migrate.sh
mst@ubuntu:~/go/src/github.com/raintank/metrictank/cmd/mt-split-metrics-by-ttl$ . migrate.sh

.format(
data_dir=data_dir,
src_kspace=args.src_kspace,
src_table=args.src_table,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cassandra stores tables in directories with a unique hash suffixed to the table name.
So you will need to look in '{data_dir}/{src_kspace}/' for a director that starts with "{src_table}-"

eg, the directory is something like

/var/lib/cassandra/data/raintank/metric-5f3434601d1e11e6886b212ea10d84c9/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, i missed that. This should fix it: ab6d214

@Dieterbe
Copy link
Contributor

what's the status of this? needs reviewing? can be merged? should it even still be merged (as in, is it still a relevant use case)

@replay
Copy link
Contributor Author

replay commented Mar 16, 2017

I think we've actually used that for one of the migrations but then forgot to merge the PR. @woodsaj is that ok to merge?

@replay
Copy link
Contributor Author

replay commented Mar 20, 2017

Considering that this only adds a file and hence can't break anything I'm thinking i should just merge it

@woodsaj
Copy link
Member

woodsaj commented Mar 20, 2017

LGTM

@Dieterbe Dieterbe merged commit a7681c5 into master Mar 20, 2017
@Dieterbe Dieterbe deleted the cross_cluster_migration_script branch January 2, 2018 16:01
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Need a cross cluster migration tool for split tables migration
3 participants