Refactor reindexing of harvested datasets

[landreev]

Among the indexing improvements in 6.3, we now have added logic that prevents deleting solr documents when existing and already indexed datasets are updated. Unfortunately, we still cannot take advantage of this improvement when it comes to reindexing (re-)harvested datasets - since the harvesting framework relies on completely deleting an existing harvested dataset, then re-creating it from scratch. So we still end up going to the trouble of deleting all the existing solr documents associated with it, then rebuilding them, even if it was a minor metadata update (documents, plural - because harvested datasets can have files).
A most obvious way to solve this is to add some straightforward mods to the destroy dataset framework, and make it spare existing solr documents during a re-harvesting workflow. An alternative would be to modify the harvesting framework itself, and figure out how to avoid having to destroy the dataset and still avoid creating multiple versions... but that may be more difficult (?).

By nature of harvesting, it often involves having to modify a large number of datasets in quick succession, so this can still be a serious performance premium in production. It would be great to address this before we restart serious-scale harvesting at HDV.