Add cap to arena count

[asdacap]

• Add a cap to netty arena count to prevent high LOH allocation on high core count machine.
• Note that there seems to be a bug where previous dotnetty config was not getting applied sometimes. If that was the case, the dotnetty default config is 16MB chunk size, with (logical cpu count * 2) arena count.
• Graph is goerli sync, 1st has 8 arena, 2nd has 64 arena, 3rd has 4 arena. Chunk size would be 16MB, 2MB, 32MB.

• Note, all three run uses the same 128MB budget, but during old blocks, all allocate more than 1 chunk per arena. For 8 and 4 arena, LOH for bytes array reduced significantly. GC stats is less smooth likely because GC count is lower. No significant difference between 8 and 4 arena. Note, during bodies sync new chunks are created in all case, however, with 8 and 4 arena, it is not released quickly after sync, doubling the memory allocated to it, it could be that it will get release after some times.

Types of changes

What types of changes does your code introduce?

• Bugfix (a non-breaking change that fixes an issue)
• Optimization

Testing

Requires testing

• Yes
• No

If yes, did you write tests?

• Yes
• No: Changes global state. Can't unit test this. Need to restart process to change config.

Notes on testing

• Have not tested yet.

[asdacap]

@tanishqjasoria can you check if this explain the allocation in zero netty handler?

[tanishqjasoria]

@tanishqjasoria can you check if this explain the allocation in zero netty handler?

i am not sure how to verify this without syncing the node and getting memory dumps. I was thinking of trying this out - https://www.jetbrains.com/dotmemory/unit/

[LukaszRozmej]

Can you provide metrics on how the allocations look now?

Love the idea, needs testing.

[asdacap]

Collecting...

[asdacap]

Added graph.