Auto-updater improvements

[andresilva]

• A random delay should be added before starting an auto-update. This is necessary to make sure that all existing nodes don't trigger an auto-update and restart at the same time.
• When there is a failure in downloading the binary (hash mismatch, eg) the time between next attempt should increase exponentially.

[andresilva]

#7348 (comment)

[rphmeier]

wouldn't this random delay be encapsulated by the differences in time to receive the block that triggers the update?

[tomusdrw]

That's not enough. For Kovan block time is 3seconds, and if every authority runs with auto-updated we are basically killing the entire network.
The second issue is potential database migrations that might happen right after upgrade (they can take up to 30 minutes). See the linked comment for proposed solution.

[lght]

For Kovan block time is 3seconds, and if every authority runs with auto-updated we are basically killing the entire network.

Maybe if we have default threshold based on time and block height.

For example, set a time minimum randomly selected from a range (1-12hrs?), and compare to a minimum block height delta (500-1K blocks?), using whichever results in longer delay? Potentially, could also set different defaults for setups that would require different recovery times (e.g.archive vs light nodes).

Different default thresholds would help with archive nodes taking 48hrs to sync (#7438 proposed solution)[https://github.com//pull/7348#issuecomment-353059939]. Their default time delay could be a randomized value in 48-96hr range.