Known problems in the 1.37 release

[posvyatokum]

As we transition to the new release process, we will no longer wait for features before the release cut (unless they are vitally important for the network).
This issue now tracks known release problems.

List of observed issues in 1.37 release

Preview Give feedback

1. [1.37.0] Stack overflow #10749
   resharding-release +1
   
2. [1.37.0] OOM #10750
   resharding-release +1
   
3. [1.37.0] Slow sync #10751
   resharding-release +1
   
4. [1.37.0] Nodes not finding peers #10753
   resharding-release +1
   
5. [1.37.0] Node crashes on init #10755
   resharding-release +1
   
6. [1.37.0] Node crashes with index out of bounds #10767
   resharding-release +1

This a dependancy tracker issue for the next release. Please, add any features and fixes you want to be included in the release.

[DEPRECATED] Previous issue description

List of required features for mainnet release

Preview Give feedback

1. Split storage/big archival nodes in mocknet
2. Merge Shard_id refactoring in master
3. Proper resharding restart during all stages
4. State snapshot management on split storage nodes

List of required features

Preview Give feedback

1. feat: Change default shard trie cache size to 500MB #10373
   A-storage +1
2. config: adjust shard cache sizes #10409
   A-storage +1
3. Change TESTNET_MIN_BLOCK_PRODUCTION_DELAY (not a blocker for cut, can be cherry-picked)
4. configure shard cache size in the new shard layout for children of shard 3 zulip
5. Verify if we see validator kickouts we've seen when testing for 1.36 & if reproducible still revert near-vm: use a recycling pool of shared code memories instead of a in-memory cache of loaded artifacts #9244 or pick up store: increase the size of the CachedContractCode column cache #10402 and verify that it helps All indications point to this not being a problem

List of optional features

Preview Give feedback

1. Split storage/big archival nodes in mocknet (not a blocker for cut, but it is useful to test archival resharding not in vacuum)
2. Fix the epoch_manager: Unexpected error in is_last_block_in_finished_epoch warning popping up on every block
3. Remove spammy info logs: client: BlockResponse and INFO garbage_collection: GC

[posvyatokum]

@gmilescu @akhi3030 @walnut-the-cat Please add anything you want to be included in 1.37
@wacban @shreyan-gupta Please add any resharding-specific issues for 1.37.

[wacban]

I added two tasks to the wishlist about some warnings and info messages being spammy, this can be observed on a mainnet canary node.

[andrei-near]

@wacban, @gmilescu can we include in 1.37 the Cold storage copy loop optimizations as required? We will need to backport this to 1.36 as well