Skip to content

Commit

Permalink
feat: enable receipt prefetching by default
Browse files Browse the repository at this point in the history
Prefetch receipt meta data (account and access keys) ahead of time.
This recent performance optimization has been disabled by default.

In lab settings, performance improvement is confirmed. Using the
estimator to measure the time it takes to process empty receipts,
on a DB with 2 million accounts, on a local SSD, with enabled shard
caches. The result is as follows.

- sender = receiver:  737us -> 386 us
- sender != receiver: 1014us -> 644us
- overhead per block: 6.9us -> 7.4us

Note that this is with 100 empty receipts in the same block, with all
different accounts. In real traffic it usually does not happen that so
many different accounts are accessed in the same block. But it is
allowed and we must be able process this case in reasonable time.
So even if it might not help in the average case, it makes sense to
activate this feature to speed up the worst-case.

Currently we use 8 IO threads per shard. Repeated experiments with
more threads showed no difference.
Decreasing it to 4 threads is about equal to 8 threads. Going lower is
significantly worse. Thus, overall, 8 threads seems reasonable here.

Canary nodes in testnet and mainnet with the feature enabled show that
the feature also works as expected on real traffic. The memory impact is
minimal, usually less than 40MB of reserved capacity, which is less than
8MB actual memory because 8 threads reserve 4MB each ahead of actually
fetching the data.
  • Loading branch information
jakmeier committed Sep 23, 2022
1 parent 1e03e0a commit 6158cc2
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 2 deletions.
2 changes: 1 addition & 1 deletion core/store/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ impl Default for StoreConfig {
block_size: bytesize::ByteSize::kib(16),

trie_cache_capacities: vec![(ShardUId { version: 1, shard_id: 3 }, 45_000_000)],
enable_receipt_prefetching: false,
enable_receipt_prefetching: true,
sweat_prefetch_receivers: vec![
"token.sweat".to_owned(),
"vfinal.token.sweat.testnet".to_owned(),
Expand Down
12 changes: 11 additions & 1 deletion runtime/runtime-params-estimator/src/testbed.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,17 @@ impl RuntimeTestbed {
StateDump::from_dir(dump_dir, workdir.path(), in_memory_db);
// Ensure decent RocksDB SST file layout.
store.compact().expect("compaction failed");
let tries = ShardTries::test(store, 1);

// Create ShardTries with relevant settings adjusted for estimator.
let shard_uids = [ShardUId { shard_id: 0, version: 0 }];
let mut trie_config = near_store::TrieConfig::default();
trie_config.enable_receipt_prefetching = true;
let tries = ShardTries::new(
store.clone(),
trie_config,
&shard_uids,
near_store::flat_state::FlatStateFactory::new(store.clone()),
);

assert!(roots.len() <= 1, "Parameter estimation works with one shard only.");
assert!(!roots.is_empty(), "No state roots found.");
Expand Down

0 comments on commit 6158cc2

Please sign in to comment.