Improve index entries deduping and concurrency

[cyriltovena]

This PR improves deduping of entries by using different deduper depending if we are running with or without concurrency, but also make sure the concurrency is bounded even for a lot of index queries ( above 100k) maximun 10 goroutine are spawned per query.

Benchmark result:

❯ benchstat before.txt after.txt                                                                                                                                  
name                    old time/op    new time/op    delta
_MultiQueries/50-16        102µs ± 3%      45µs ± 2%  -55.39%  (p=0.008 n=5+5)
_MultiQueries/100-16       203µs ± 3%      90µs ± 5%  -55.57%  (p=0.008 n=5+5)
_MultiQueries/1000-16     4.16ms ± 1%    2.44ms ± 1%  -41.35%  (p=0.008 n=5+5)
_MultiQueries/10000-16    43.5ms ± 3%    24.5ms ± 1%  -43.59%  (p=0.008 n=5+5)
_MultiQueries/50000-16     354ms ± 0%     127ms ± 2%  -64.21%  (p=0.008 n=5+5)

name                    old alloc/op   new alloc/op   delta
_MultiQueries/50-16       91.3kB ± 0%    27.8kB ± 0%  -69.52%  (p=0.008 n=5+5)
_MultiQueries/100-16       182kB ± 0%      56kB ± 0%  -69.48%  (p=0.008 n=5+5)
_MultiQueries/1000-16     1.85MB ± 0%    0.76MB ± 0%  -58.87%  (p=0.008 n=5+5)
_MultiQueries/10000-16    18.3MB ± 0%     7.5MB ± 0%  -58.96%  (p=0.008 n=5+5)
_MultiQueries/50000-16    90.6MB ± 0%    37.2MB ± 0%  -58.94%  (p=0.008 n=5+5)

name                    old allocs/op  new allocs/op  delta
_MultiQueries/50-16        1.12k ± 0%     0.51k ± 0%  -54.74%  (p=0.008 n=5+5)
_MultiQueries/100-16       2.22k ± 0%     1.01k ± 0%  -54.73%  (p=0.008 n=5+5)
_MultiQueries/1000-16      22.1k ± 0%     14.0k ± 0%  -36.47%  (p=0.008 n=5+5)
_MultiQueries/10000-16      221k ± 0%      140k ± 0%  -36.53%  (p=0.016 n=5+4)
_MultiQueries/50000-16     1.10M ± 0%     0.70M ± 0%  -36.51%  (p=0.008 n=5+5)