boltdb shipper index list cache improvements

[sandeepsukhani]

What this PR does / why we need it:
To reduce the number of list calls we make to the object store when using boltdb-shipper, we have a cache in place. The cache stays valid for 1 min and is refreshed when a list call is made and it finds that cache needs to be rebuilt. The list call could take a couple of seconds to finish in a large cluster which can add up to the query latency when downloading index at query time.

This PR adds a parameter to list calls to skip the list cache and directly load the objects list from the object store. We will skip the cache only while doing query time download of index which is not too common, we see just up to 8k query time download operations in a day in our largest Loki cluster.

The other change this PR does is to detect the staleness of the list cache. Before explaining the problem, let us keep few things in mind:

• While running compaction, it would first upload the newly created(compacted) file and then remove the source files. So it is possible that a list call can see the newly compacted file with the old source files while the compactor is still removing them.
• When we see a 404 error while downloading the index, we just ignore it assuming the compactor would have removed it during compaction.

Now the problem is if compaction happened just after we cached the list of objects, the sync operation on the recently compacted table would try to download old source files that were compacted away but we would just ignore missing files as mentioned above. This means we won't have downloaded either of source index files and the new compacted file, still not know too us due to stale cache. This PR adds a check in sync code to see if we skipped downloading all the files, if so then we would retry the sync after force refreshing the index list cache.

Checklist

• Tests updated

[cyriltovena]

LGTM