Make timeouts dynamic

[evan-forbes]

We expected mocha to have inconsistent block times due to its long proposal timeout, but we should not expect as many consecutive block producers as we're seeing. We need to investigate the cause of both, along with collecting more data to set better default timeouts.

[evan-forbes]

I did some hacking over the weekend and made the timeout commit relative, meaning that it accounted for the time already used in that round. It seems to work as expected, and we end up with more consistent block times. This might be something that we want to pursue after further analysis.

[evan-forbes]

changing this issue to only investigate and fix the inconsistent block times, then splitting off the repeating block producers into a different issue

[cmwaters]

What's remaining of this issue?

[evan-forbes]

What's remaining of this issue?

try out the latest change on mocha imo

[evan-forbes]

to update this, I think we have a good understanding of what is causing the variable block times, and we have decided that we are not pursing a temporary solution. Instead, we will either use protocol enforced block times or separate the block from the square and let comet come to consensus as quickly as possible

[evan-forbes]

we're reprioritizing this issue, we should have the ability to change the timeouts so that we can spend more time gossiping while still having consistent block times.

[evan-forbes]

blocked by #1333

[cmwaters]

I've worked on this prototype (#1342) to illustrate using a model based approach to having dynamic timeouts which I think would be the simplest as most effective solution to stable block times

[liamsi]

resharing this here as it also seems related: https://github.com/cometbft/cometbft/blob/4241776d5c2a73f19d63b7627a11af643a3fe6a8/docs/references/architecture/adr-115-predictable-block-times.md