Receiving alarms must take precedence over receiving broadcasted messages

[masih]

The GPBFT algorithm assumes that when an alarm is triggered it will get delivered at the designated time before a broadcasted message that so happens to arrive at the same time.

This assumption does not hold in the current simulations, where all messages including alarms use the same queue for delivery prioritised by deliverAt timestamp regardless of whether they are an alarm or broadcasted messages.

• Respect the priority of alarms in simulation.
• Document this requirement at gpbft.Clock.

[anorth]

This spawned from #242, but that says the protocol...

assumes that when it sends a message, it is going to receive that message before it receives anything else

I.e. messages first, in the particular case of a send to self. So on the surface this strikes me as not the right fix.

It's also not documented in the code why the alarm-before-message assumption exists, and e.g. what would have to change for it to no longer be required (currently, it's just asserted with no justification). This should be documented somewhere in gpbft or the host API.

(The "why" for messages first is: so a send to self is received as if synchronously)

[anorth]

@Kubuxu noted in #318 that for message-over-alarm precedence:

This is hard or almost impossible to guarantee in an environment where signing is async and can fail. I will open an issue about it.

Elsewhere:

There is no guarantee that anyone will sign the message we requested broadcast for, and even if there is someone that will sign it, it would require us to block everything until the do.

Alternatively, we can try to go back to a pattern of internally-delivering sends-to-self at a post-validation step. This will be difficult without knowing the participant's own ID (which was the goal #103 that resulted in async self-sends in the first place).

[masih]

(The "why" for messages first is: so a send to self is received as if synchronously)

One can say the same about alarm. right?

This is my interpretation of the FIP, in that alarm triggering system is not considered a network call. I also acknowledge that the FIP can be interpreted in a different way and it does not dictate a strict ordering of messages vs alerts.

Thank you for clarifying the need for ordering of messages to self; I think both messages to self and alarm should be synchronous.

[anorth]

(The "why" for messages first is: so a send to self is received as if synchronously)

One can say the same about alarm. right?

I don't think there's any need for synchronous alarms. If code setting an alarm wants synchronous triggering, it should just call the trigger directly. Alarms are specifically to invoke asynchrony. The code (but not the FIP) uses "alarm at now" to invoke asynchrony with no delay just to unify the execution path into some bits of code, like starting a protocol instance.

It would be much simpler if messages-to-self could also use synchronous invocation by direct method call (and they used to). But that became impossible when we removed knowledge of the participant ID from the protocol code.