Data reimported on _river index recreation when Mongo collection is empty

[golspe]

What's the proper way to delete all data from Mongo and have that reflected in ES?

Steps taken:

1. Load data into MongoDB
2. Configure river plugin using these options
   "options": {
   "secondary_read_preference": true,
   "import_all_collections": true,
   "skip_initial_import": true
3. See that data is imported into ES
4. Delete all data from MongoDB
5. Delete river index and ES index
6. Recreate river index

After step #6, the data is re-imported to ES but there's no data in Mongo (db.collection.count() returns 0 on both Primary and Secondary nodes of the replica set)

[richardwilly98]

If you want to delete all data just drop the collection / database in MongoDB and delete the river and the index in ES then recreate the index.

[golspe]

I have tried that it when I recreate the index, the data reappears in ES. I'm assuming it's reimporting the data from the oplog but I'm not sure why. Currently running:

MongoDB 2.4.3
Elasticsearch 0.90.7
River MongoDB 1.7.3

Do I need to downgrade my ES instead and River plugin to match the version of MongoDB?

[talha-asad]

I think you should use the latest version of this plugin, i.e 2.0.0
Also i think, what you are looking for is this option: "drop_collection"

[richardwilly98]

The initial import (using the collection data and not oplog.rs) has been implemented since 1.7.1 [1].
So you should not have such issue.

Before 1.7.1 the initial import was using oplog.rs.

[1] -#47

[benmccann]

Sounds like this has been addressed in newer versions of the river