From 6158cc2ce716d7dc5a9a4d431e17be34823864f0 Mon Sep 17 00:00:00 2001 From: Jakob Meier Date: Wed, 21 Sep 2022 18:52:12 +0200 Subject: [PATCH] feat: enable receipt prefetching by default Prefetch receipt meta data (account and access keys) ahead of time. This recent performance optimization has been disabled by default. In lab settings, performance improvement is confirmed. Using the estimator to measure the time it takes to process empty receipts, on a DB with 2 million accounts, on a local SSD, with enabled shard caches. The result is as follows. - sender = receiver: 737us -> 386 us - sender != receiver: 1014us -> 644us - overhead per block: 6.9us -> 7.4us Note that this is with 100 empty receipts in the same block, with all different accounts. In real traffic it usually does not happen that so many different accounts are accessed in the same block. But it is allowed and we must be able process this case in reasonable time. So even if it might not help in the average case, it makes sense to activate this feature to speed up the worst-case. Currently we use 8 IO threads per shard. Repeated experiments with more threads showed no difference. Decreasing it to 4 threads is about equal to 8 threads. Going lower is significantly worse. Thus, overall, 8 threads seems reasonable here. Canary nodes in testnet and mainnet with the feature enabled show that the feature also works as expected on real traffic. The memory impact is minimal, usually less than 40MB of reserved capacity, which is less than 8MB actual memory because 8 threads reserve 4MB each ahead of actually fetching the data. --- core/store/src/config.rs | 2 +- runtime/runtime-params-estimator/src/testbed.rs | 12 +++++++++++- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/core/store/src/config.rs b/core/store/src/config.rs index 6c810c9c4a3..82d0418aceb 100644 --- a/core/store/src/config.rs +++ b/core/store/src/config.rs @@ -162,7 +162,7 @@ impl Default for StoreConfig { block_size: bytesize::ByteSize::kib(16), trie_cache_capacities: vec![(ShardUId { version: 1, shard_id: 3 }, 45_000_000)], - enable_receipt_prefetching: false, + enable_receipt_prefetching: true, sweat_prefetch_receivers: vec![ "token.sweat".to_owned(), "vfinal.token.sweat.testnet".to_owned(), diff --git a/runtime/runtime-params-estimator/src/testbed.rs b/runtime/runtime-params-estimator/src/testbed.rs index f9134056b90..df595465f48 100644 --- a/runtime/runtime-params-estimator/src/testbed.rs +++ b/runtime/runtime-params-estimator/src/testbed.rs @@ -31,7 +31,17 @@ impl RuntimeTestbed { StateDump::from_dir(dump_dir, workdir.path(), in_memory_db); // Ensure decent RocksDB SST file layout. store.compact().expect("compaction failed"); - let tries = ShardTries::test(store, 1); + + // Create ShardTries with relevant settings adjusted for estimator. + let shard_uids = [ShardUId { shard_id: 0, version: 0 }]; + let mut trie_config = near_store::TrieConfig::default(); + trie_config.enable_receipt_prefetching = true; + let tries = ShardTries::new( + store.clone(), + trie_config, + &shard_uids, + near_store::flat_state::FlatStateFactory::new(store.clone()), + ); assert!(roots.len() <= 1, "Parameter estimation works with one shard only."); assert!(!roots.is_empty(), "No state roots found.");