IPC: Reconfiguration and dynamic membership

[adlrocha]

Fendermint should pick up the membership information from the IPC agent (that itself picks it up from the subnet actor in the parent). This information should be use to trigger a reconfiguration when the membership set has changed.

The voting power of each validator should be also determined by the collateral that they have provided for the subnet in the parent (in the same call, the agent also provides this information to Fendermint).

[aakoshh]

Fendermint should pick up the membership information from the IPC agent

I think the membership should be relayed to Fendermint as part of top-down checkpoints or transactions, and be "picked up" from the ledger itself. I don't see a need for Fendermint to reach out to the agent in this scenario, if that's what the wording meant.

1. IPC agents following the parent subnet see a top-down checkpoint or a top-down transaction.
2. Some agents (for rewards that ensure there is a few of them, not all of them) relay this as a transaction onto the subnet.
3. The subnet Fendermint instances reach an agreement on when the top-down checkpoint is final (for this they consult the parent chain, which can happen by direct observation or through an agent, but I suggest a library approach should be possible here).
4. One of the subnet Fendermint instances include the top-down checkpoint for execution in a block, the others vote for it.
5. The subnet ledger executes the membership changing transaction, but all it does is add it to a queue in the ledger, so we can have a fixed length epoch with fixed validator set in the subnet ledger.
6. At the end of that epoch, the queued membership altering transactions are executed and the power table changes are returned to Tendermint in end_block.

[aakoshh]

Since Fendermint instances are expected to directly observe their parent, it's also okay if instead of 2-3 relayers we expect validators to add the next top-down transaction to their proposals automatically using prepare_proposal and not a signed transaction relayed by a third party. In that case yes, Fendermint can "pick it up" from the agent, although I would still recommend that the "parent view" is just a library running inside Fendermint itself, rather than a separate agent process, so we can cut down on yet another RPC interface.

[aakoshh]

The advantage of this "parent view" to be a separate process would be that it would be possible to stand up an environment where a Fendermint application doesn't have to be recompiled according to what kind of parent it has, it can be just pointed at a process which knows. That is, an instance of the Agent / parent view that follows Lotus can be launched, or one that follows a Tendermint parent, but the child doesn't need to have the connecting libraries compiled into its binary.

I'm okay with this because this is how I imagined relayers would work too, although I wouldn't necessarily cram all of these functions into the agent, but rather a special purpose container whose job is to implement this particular RPC interface. Even now the agent is only prepared to speak to Lotus. At the very least it should have its own RPC client that doesn't assume more about its functions than the operations necessary to decide when something is final, and to pick up top-down transactions in general.

[adlrocha]

Update to the design

We almost have the mechanics of moving the membership information top-down through the ParentView (see consensus-shipyard/fendermint#204, consensus-shipyard/fendermint#205, consensus-shipyard/fendermint#206, consensus-shipyard/fendermint#231), now we just need to make the membership set updates effective for Fendermint.

• We will have to change the gateway contract so that when a validator joins in the parent, the validators provide their public key and Cosmos endpoint, as it will be needed to update the membership information in Tendermint. We may be able to re-purpose the validator-addr and change it to some bytes that accept any validator-info that accepts any serialized data structure and can be processed by the underlying peer implementation (so we don't need to include an infinite number of arguments there according to the underlying peer).
• When all the requirements to start validating blocks in the subnet are fulfilled, the gateway should track the genesis_epoch for the subnet. This will be used to get the initial membership to deterministically construct the genesis file for the subnet. When a peer wants to create the genesis for a subnet, it will populate the genesis using the membership available at the genesis epoch. Alternatively (and it may be simpler), we could explicitly track the initialMembership directly in the contract.
• We should provide a genesis subnet-genesis command in Fendermint that picks up the information from the parent through the ipc-agent and populates the genesis.
• When Fendermint starts, it picks up the initial membership from genesis.
• From there on, as part of the proof of finality from the parent, the validator will update to Cosmos the membership information in end_block by passing the new set of validators (if there are changes) and the validators' corresponding public keys as described here.
• We should bear in mind that the new membership only takes effect in Tendermint after 3 rounds of consensus.

Finally, in order for validators to be able to interact with each other they will have to add the following to their Tendermint configuration:

• All of them will have to be configured with pex = true and seed_mode = true in their configuration, to support peer discovery.
• They should also introduce the validators in the current membership to their seed list so they have a list of peers to bootstrap from. We should add the modification of the configuration as part of some initialization command (like we did with subnet genesis).

References

• Run a validator in Fendermint: https://docs.cometbft.com/v0.37/core/validators

[adlrocha]

Being tackled in #305 and its corresponding PR consensus-shipyard/fendermint#272. @aakoshh, feel free to reopen if this is not the case.