Pruning the ElasticSearch database

You can use ElasticSearch Curator to prune the ElasticSearch database.

Example usage:

1. Installation

apt-get -y install python-pip
pip install elasticsearch-curator

1. Delete all indexes, that are older than 3 days and prefix starts with cif.observables

/usr/local/bin/curator --host localhost --master-only delete indices --prefix cif.observables --older-than 3 --time-unit days --timestring '%Y.%m.%d'

1. Using --dry-run will show you what would be deleted

/usr/local/bin/curator --dry-run --host localhost --master-only delete indices --prefix cif.observables --older-than 3 --time-unit days --timestring '%Y.%m.%d'
2015-06-24 07:51:46,243 INFO      Job starting: delete indices
2015-06-24 07:51:46,316 INFO      Pruning Kibana-related indices to prevent accidental deletion.
2015-06-24 07:51:46,317 INFO      DRY RUN MODE.  No changes will be made.
2015-06-24 07:51:46,322 INFO      DRY RUN: delete: cif.observables-2015.05.17
2015-06-24 07:51:46,327 INFO      DRY RUN: delete: cif.observables-2015.05.18
2015-06-24 07:51:46,332 INFO      DRY RUN: delete: cif.observables-2015.05.26
2015-06-24 07:51:46,337 INFO      DRY RUN: delete: cif.observables-2015.05.27
2015-06-24 07:51:46,341 INFO      DRY RUN: delete: cif.observables-2015.05.28
<SNIP>
2015-06-24 07:51:46,397 INFO      DRY RUN: delete: cif.observables-2015.06.17
2015-06-24 07:51:46,402 INFO      DRY RUN: delete: cif.observables-2015.06.18
2015-06-24 07:51:46,406 INFO      DRY RUN: delete: cif.observables-2015.06.19
2015-06-24 07:51:46,411 INFO      DRY RUN: delete: cif.observables-2015.06.20

Source: cif-users group few questions

Newer versions of curator don't take command line arguments. You can suss out the various config file entries that will duplicate what's here, or you can do:

pip install elasticsearch-curator==3.5.1

for the most recent version of curator that doesn't require the config file.