[indexer] optimize es bulk operate

[ssyuan]

No description provided.

[codecov]

Codecov Report

Merging #2543 (a4ac732) into master (d6606a0) will decrease coverage by 0.14%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #2543      +/-   ##
==========================================
- Coverage   33.27%   33.14%   -0.13%     
==========================================
  Files         498      498              
  Lines       43474    43546      +72     
  Branches    19675    19729      +54     
==========================================
- Hits        14462    14428      -34     
- Misses      14229    14378     +149     
+ Partials    14783    14740      -43     

Flag	Coverage Δ
unittests	33.14% <0.00%> (-0.13%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
cmd/indexer/src/es_sinker.rs	0.00% <0.00%> (ø)
sync/src/block_connector/test_write_block_chain.rs	17.65% <0.00%> (-7.56%)	⬇️
network-rpc/core/src/lib.rs	23.53% <0.00%> (-5.88%)	⬇️
account/src/account_manager.rs	32.54% <0.00%> (-4.51%)	⬇️
account/api/src/types.rs	57.58% <0.00%> (-3.71%)	⬇️
account/service/src/service.rs	17.80% <0.00%> (-2.92%)	⬇️
rpc/api/src/account/mod.rs	22.73% <0.00%> (-2.27%)	⬇️
account/src/account.rs	47.92% <0.00%> (-2.08%)	⬇️
executor/src/stdlib_test/sip_flag_test.rs	17.31% <0.00%> (-1.92%)	⬇️
sync/src/tasks/mock.rs	44.76% <0.00%> (-1.65%)	⬇️
... and 32 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d6606a0...a4ac732. Read the comment docs.

[github-actions]

Benchmark for dd15f6b

Click to view benchmark

Test	PR Benchmark	Master Benchmark	%
accumulator_append	2.3±0.32ms	2.2±0.10ms	+4.55%
block_apply/block_apply_10	428.6±5.44ms	425.7±9.54ms	+0.68%
block_apply/block_apply_1000	43.3±0.35s	42.2±0.31s	+2.61%
get_with_proof/db_store	39.2±0.46µs	38.6±0.37µs	+1.55%
get_with_proof/mem_store	31.4±0.24µs	31.9±0.23µs	-1.57%
put_and_commit/db_store/1	1231.2±47.49µs	1338.9±90.59µs	-8.04%
put_and_commit/db_store/10	2.3±0.34ms	2.4±0.34ms	-4.17%
put_and_commit/db_store/100	9.2±0.58ms	9.4±0.58ms	-2.13%
put_and_commit/db_store/5	1930.2±134.21µs	1934.1±153.30µs	-0.20%
put_and_commit/db_store/50	5.5±0.57ms	5.6±0.54ms	-1.79%
put_and_commit/mem_store/1	62.9±5.64µs	63.0±5.55µs	-0.16%
put_and_commit/mem_store/10	582.1±46.97µs	585.1±46.43µs	-0.51%
put_and_commit/mem_store/100	5.7±0.81ms	5.7±0.80ms	0.00%
put_and_commit/mem_store/5	295.4±24.03µs	294.5±24.50µs	+0.31%
put_and_commit/mem_store/50	2.9±0.18ms	2.9±0.18ms	0.00%
query_block/query_block_in(10)_times(100)	2.9±0.05ms	2.9±0.06ms	0.00%
query_block/query_block_in(10)_times(1000)	29.1±0.73ms	28.9±0.47ms	+0.69%
query_block/query_block_in(10)_times(10000)	293.2±3.99ms	292.8±4.73ms	+0.14%
query_block/query_block_in(1000)_times(100)	911.4±6.09µs	906.8±6.88µs	+0.51%
query_block/query_block_in(1000)_times(1000)	9.2±0.13ms	9.1±0.04ms	+1.10%
query_block/query_block_in(1000)_times(10000)	91.1±0.60ms	91.3±0.42ms	-0.22%
storage_transaction	47.2±3.46ms	48.0±3.95ms	-1.67%
vm/transaction_execution/1	331.4±0.96ms	333.5±1.04ms	-0.63%
vm/transaction_execution/10	112.3±2.78ms	112.2±0.37ms	+0.09%
vm/transaction_execution/20	105.1±1.06ms	105.3±0.30ms	-0.19%
vm/transaction_execution/5	133.3±0.37ms	134.2±0.36ms	-0.67%
vm/transaction_execution/50	120.6±0.91ms	120.9±0.36ms	-0.25%

[github-actions]

Benchmark for 31012b3

Click to view benchmark

Test	PR Benchmark	Master Benchmark	%
accumulator_append	2.4±0.24ms	2.5±0.22ms	-4.00%
block_apply/block_apply_10	432.4±8.22ms	430.9±11.12ms	+0.35%
block_apply/block_apply_1000	44.2±0.41s	44.5±0.25s	-0.67%
get_with_proof/db_store	38.0±0.41µs	38.2±0.52µs	-0.52%
get_with_proof/mem_store	31.7±0.24µs	31.4±0.22µs	+0.96%
put_and_commit/db_store/1	1388.3±75.15µs	1420.9±60.77µs	-2.29%
put_and_commit/db_store/10	2.5±0.32ms	2.5±0.36ms	0.00%
put_and_commit/db_store/100	9.3±0.59ms	9.5±0.59ms	-2.11%
put_and_commit/db_store/5	1971.9±172.84µs	3.3±9.04ms	-40.25%
put_and_commit/db_store/50	5.6±0.55ms	5.7±0.66ms	-1.75%
put_and_commit/mem_store/1	62.6±5.58µs	62.8±5.53µs	-0.32%
put_and_commit/mem_store/10	582.8±45.69µs	585.6±47.67µs	-0.48%
put_and_commit/mem_store/100	5.7±0.81ms	5.7±0.82ms	0.00%
put_and_commit/mem_store/5	296.1±24.30µs	294.8±24.12µs	+0.44%
put_and_commit/mem_store/50	2.9±0.18ms	2.9±0.19ms	0.00%
query_block/query_block_in(10)_times(100)	3.0±0.05ms	2.9±0.10ms	+3.45%
query_block/query_block_in(10)_times(1000)	29.6±0.85ms	29.4±0.42ms	+0.68%
query_block/query_block_in(10)_times(10000)	292.9±5.03ms	293.5±4.71ms	-0.20%
query_block/query_block_in(1000)_times(100)	918.1±4.33µs	917.8±5.22µs	+0.03%
query_block/query_block_in(1000)_times(1000)	9.2±0.04ms	9.2±0.07ms	0.00%
query_block/query_block_in(1000)_times(10000)	91.9±0.53ms	92.2±0.38ms	-0.33%
storage_transaction	51.2±3.73ms	55.6±5.76ms	-7.91%
vm/transaction_execution/1	334.0±1.32ms	332.3±0.98ms	+0.51%
vm/transaction_execution/10	111.7±0.24ms	111.6±0.71ms	+0.09%
vm/transaction_execution/20	105.3±0.55ms	105.6±1.29ms	-0.28%
vm/transaction_execution/5	133.8±0.59ms	134.2±1.90ms	-0.30%
vm/transaction_execution/50	120.6±0.29ms	120.4±0.35ms	+0.17%

[github-actions]

Benchmark for 5fc2b3f

Click to view benchmark

Test	PR Benchmark	Master Benchmark	%
accumulator_append	2.8±0.08ms	2.8±0.36ms	0.00%
block_apply/block_apply_10	503.1±4.66ms	511.8±10.02ms	-1.70%
block_apply/block_apply_1000	53.0±0.91s	52.7±1.24s	+0.57%
get_with_proof/db_store	38.0±0.40µs	38.4±0.37µs	-1.04%
get_with_proof/mem_store	31.8±0.23µs	31.9±0.32µs	-0.31%
put_and_commit/db_store/1	1805.8±118.38µs	1729.0±83.30µs	+4.44%
put_and_commit/db_store/10	2.9±0.35ms	2.9±0.37ms	0.00%
put_and_commit/db_store/100	10.0±0.97ms	9.9±0.87ms	+1.01%
put_and_commit/db_store/5	2.3±0.15ms	2.6±0.19ms	-11.54%
put_and_commit/db_store/50	6.2±0.70ms	6.2±0.74ms	0.00%
put_and_commit/mem_store/1	63.0±5.54µs	63.6±5.52µs	-0.94%
put_and_commit/mem_store/10	591.9±46.25µs	587.7±46.29µs	+0.71%
put_and_commit/mem_store/100	5.8±0.81ms	5.8±0.81ms	0.00%
put_and_commit/mem_store/5	297.0±23.80µs	297.0±24.36µs	0.00%
put_and_commit/mem_store/50	2.9±0.18ms	2.9±0.19ms	0.00%
query_block/query_block_in(10)_times(100)	2.9±0.08ms	2.9±0.06ms	0.00%
query_block/query_block_in(10)_times(1000)	29.6±0.86ms	29.7±0.60ms	-0.34%
query_block/query_block_in(10)_times(10000)	296.0±4.60ms	293.2±4.57ms	+0.95%
query_block/query_block_in(1000)_times(100)	932.5±5.62µs	920.6±4.75µs	+1.29%
query_block/query_block_in(1000)_times(1000)	9.2±0.02ms	9.2±0.06ms	0.00%
query_block/query_block_in(1000)_times(10000)	93.1±0.76ms	92.2±0.40ms	+0.98%
storage_transaction	70.5±3.42ms	68.2±3.21ms	+3.37%
vm/transaction_execution/1	332.0±0.65ms	331.8±2.00ms	+0.06%
vm/transaction_execution/10	111.2±0.22ms	111.4±0.67ms	-0.18%
vm/transaction_execution/20	104.9±0.36ms	104.7±0.30ms	+0.19%
vm/transaction_execution/5	133.6±0.55ms	133.2±0.43ms	+0.30%
vm/transaction_execution/50	121.1±1.48ms	120.3±0.40ms	+0.67%

[github-actions]

Benchmark for cf68540

Click to view benchmark

Test	PR Benchmark	Master Benchmark	%
accumulator_append	2.6±0.42ms	2.3±0.11ms	+13.04%
block_apply/block_apply_10	486.8±15.71ms	444.6±11.04ms	+9.49%
block_apply/block_apply_1000	47.6±1.10s	45.8±1.31s	+3.93%
get_with_proof/db_store	38.6±0.39µs	39.2±0.37µs	-1.53%
get_with_proof/mem_store	32.1±0.31µs	32.8±0.37µs	-2.13%
put_and_commit/db_store/1	1461.6±140.58µs	1664.5±127.10µs	-12.19%
put_and_commit/db_store/10	2.4±0.42ms	2.4±0.44ms	0.00%
put_and_commit/db_store/100	9.7±1.30ms	9.6±1.29ms	+1.04%
put_and_commit/db_store/5	2.0±0.19ms	1982.0±175.28µs	+0.91%
put_and_commit/db_store/50	5.9±0.81ms	5.8±0.83ms	+1.72%
put_and_commit/mem_store/1	62.9±5.56µs	62.6±5.60µs	+0.48%
put_and_commit/mem_store/10	582.7±46.27µs	582.8±49.82µs	-0.02%
put_and_commit/mem_store/100	5.7±0.81ms	5.7±0.81ms	0.00%
put_and_commit/mem_store/5	295.1±24.10µs	295.2±24.23µs	-0.03%
put_and_commit/mem_store/50	2.9±0.19ms	2.9±0.18ms	0.00%
query_block/query_block_in(10)_times(100)	2.9±0.05ms	3.0±0.05ms	-3.33%
query_block/query_block_in(10)_times(1000)	29.1±0.53ms	29.5±0.84ms	-1.36%
query_block/query_block_in(10)_times(10000)	292.3±3.96ms	295.3±5.14ms	-1.02%
query_block/query_block_in(1000)_times(100)	917.8±5.12µs	925.1±6.96µs	-0.79%
query_block/query_block_in(1000)_times(1000)	9.2±0.05ms	9.2±0.07ms	0.00%
query_block/query_block_in(1000)_times(10000)	91.4±0.59ms	93.6±0.51ms	-2.35%
storage_transaction	56.5±4.54ms	53.0±4.42ms	+6.60%
vm/transaction_execution/1	331.8±1.57ms	332.0±0.86ms	-0.06%
vm/transaction_execution/10	111.4±0.22ms	111.5±0.51ms	-0.09%
vm/transaction_execution/20	104.9±0.26ms	105.3±1.48ms	-0.38%
vm/transaction_execution/5	133.6±0.25ms	133.8±0.65ms	-0.15%
vm/transaction_execution/50	120.6±0.39ms	120.7±0.60ms	-0.08%