Method in the Gateway to return the current validator set

[aakoshh]

At the end of the checkpointing period (typically called epoch), Fendermint needs to return to CometBFT the changes in the power table. For this it would help if the Gateway actor could return the current power table, so that we can calculate the delta.

That's because CometBFT expects updates, so if we currently have {A: 1, B: 2} and we want to transition to {B: 3, C: 2} then we need to give CometBFT an update of {A: 0, B: 3, C:2}, that is, not only do we say something about the validators in the next set, we also have to remove the previous ones if they are not in the next set.

[dnkolegov]

This is being addressed in consensus-shipyard/ipc-solidity-actors#200

[aakoshh]

@dnkolegov posited taking a page from the book of the Gravity Bridge and only storing hashes of the validator sets. IIUC the way it works there is that in the Solidity contract only a hash of the validator set is stored, and when a checkpoint is sent the whole validator set is part of it, checked against the hash, and then the power threshold can be tested on the inputs, rather than against a stored configuration.

The situation is somewhat different than ours, because Tendermint merely informs the Solidity contract about what's going on, while in our case the validator set is produced on the parent, assigned an ID, sent to the child, who then uses this ID to refer back to it.

Nevertheless it's true that currently we store entire validator sets and we don't have to:

• @cryptoAtwill passes in the whole validator set even if it did not change since the last finality (but the configuration ID is not sent, which is is a problem).
• The contract stores the validator set, which we will need when the contract needs to check whether signature submissions should be accepted in Method in Checkpoint actor to add new validator signature #205
• Retrieving past validator set configurations is also required after a restart, when the node has to decide if it has to re-submit a signature in Upon restart list incomplete checkpoints and re-broadcast signatures if necessary #298
• But, we can always ask CometBFT itself through the RPC about who the validators were in the past, and we can restore the power table and pass it to the contract for signature validation.

So the Gateway doesn't have to remember the history of the validator sets. It's enough if a checkpoint remembers the block height and the configuration number; we can use that to retrieve the validator set from CometBFT to make decisions or pass it to the contract if needed, and use the configuration ID to communicate to the parent who will sign next.

The parent has to remember the validator set, because after changes it has to be able to propose a new full set.

The child has to keep the full next validator set until the end of the checkpoint period, which is when it's activated, which means it has to be passed to CometBFT in full.

So the proposed changes are:

1. Keep a full record of a single next Membership record.
2. Keep a value for the current configuration number which does not change until the contract is instructed.
3. Upon instruction to adopt the next validator set, return the one stored and overwrite the current configuration number.
4. Fendermint will turn to CometBFT instead of calling the method mentioned in this ticket.

[adlrocha]

Overall, I agree with the changes, I like this approach a lot. Just a double-check:

Keep a full record of a single next Membership record.

All of this applies to the child subnet, right?

[aakoshh]

All of this applies to the child subnet, right?

Yes, this is to implement the bottom-up checkpoint period in the child subnet itself.

For example say we produce a checkpoint in the child subnet every 1000 blocks, and our current configuration number is 1. We may receive a new configuration from the parent at block 275, 589, 865, (confirguration 2, 3 and 4, respectively) but all we do with them is stick them into the next slot, replacing the previous one,while leaving the configuration number at 1. When we reach block 1000, we tell the contract to do the membership transition, so it overwrites the configuration number 1 with 4 and returning the membership we stored in block 865.

Then, we tell CometBFT what the changes are by comparing the validator set at block 1000 to the one in the return value to produce a changeset, and we construct a checkpoint record signed by the validators at block 1000, signing over their power to configuration number 4. We send this checkpoint to the parent.

[dnkolegov]

The parent has to remember the validator set, because after changes it has to be able to propose a new full set.

Does the parent (SubnetActor) need to remember the history on validator sets?

[aakoshh]

@dnkolegov I reckon it has to remember the following:

• The last configuration which it has seen adopted by the subnet, ie. the one it expects the next checkpoint to be signed by
• All the newer configurations which it has proposed so far to the subnet, but doesn't know which one the subnet will adopt

Once it sees in a checkpont that a configuration number has advanced, it can move forward to the the one indicated as next, and forget everything before that new high water mark.

In the above example the parent would remember 1,2,3,4. If the next checkpoint says it will adopt 3 (not 4), then it can forget 1 and 2.

This is for its own needs for checkpoint validation.

Then there is the question of what @cryptoAtwill is doing in consensus-shipyard/fendermint#236 :

1. Look at what is the last finalized block height of the parent subnet recorded by the child subnet
2. Look at what the current finalized block height of the parent subnet is as observed by the IPC Agent
3. Query the cross messages at heights between these to
4. Query the validator set at the final height (I think, although it might query them all the way - yes he does even though only the last one will be used).

Logically the child will not have adopted a configuration ID that hasn't been finalized, and therefore it will not have reported it adopted in a checkpoint, which means the parent will not have forgotten about it either, so all should be well.

We just have to remember that the Agent cannot query at any block height or whatever it is doing.