Support RocksDB auto-tune rate limiter for the background IO

[wanghenshui]

fix #1417

[PragmaTwice]

To make the CI pass, you can run ./x.py format in your local before pushing your code.

[wanghenshui]

@xiaobiaozhao please benchmark this in compaction with heavy write scenario

[mapleFU]

This LGTM. Let wait some benchmarking for it

[xiaobiaozhao]

# The maximum allowed aggregated write rate of flush and compaction (in MB/s).
# If the rate exceeds max-io-mb, io will slow down.
# 0 is no limit
# Default: 500
max-io-mb 500

I think you can reuse this configuration 'max-io-mb'. If it is 0 use auto-tune.

[wanghenshui]

# The maximum allowed aggregated write rate of flush and compaction (in MB/s).
# If the rate exceeds max-io-mb, io will slow down.
# 0 is no limit
# Default: 500
max-io-mb 500

I think you can reuse this configuration 'max-io-mb'. If it is 0 use auto-tune.

auto tune donesn't means auto detect write bandwidth, write bandwidth limit is always useful

[git-hulk]

@xiaobiaozhao Did you test this PR?

[xiaobiaozhao]

@xiaobiaozhao Did you test this PR?
I will test it this weekend.

[xiaobiaozhao]

Press data into kvrocks script

docker run -d --rm redislabs/memtier_benchmark:latest -t 4 -c 20 -s 172.27.255.244 -p 6666 --distinct-client-seed --key-minimum=1000000000 --key-maximum=10000000000 --random-data --key-prefix="" --data-size=50 -n 100000000 --pipeline=100 --command "mset key data key data key data key data key data key data key data key data key data key data"

write script

docker run --rm redislabs/memtier_benchmark:latest -t 4 -c 20 -s 172.27.255.244 -p 6666 --distinct-client-seed --key-minimum=1000000000 --key-maximum=10000000000 --key-prefix="" -n 1000 --command "set key data"

[mapleFU]

Thanks! Do you have idea that why the performance decline in normal p100(And does all performance optimization and decline comes from unstable test result)?

[xiaobiaozhao]

Thanks! Do you have idea that why the performance decline in normal p100(And does all performance optimization and decline comes from unstable test result)?

I think this is bias of a test, p99 p9999 should be more convincing

[git-hulk]

This option may bring better performance benefits when the background compaction is heavy, not sure if the benchmark data was under this scenario. But anyway, it's good to merge since it also brings slight performance improvement.

[git-hulk]

Thanks all, merging...