Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define libp2p protocol for light client sync #2802

Merged
merged 2 commits into from
Jul 26, 2022
Merged

Define libp2p protocol for light client sync #2802

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
56363cd
Define libp2p protocol for light client sync
etan-status May 3, 2022
52a741f
Improve grammar
etan-status Jul 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Features are researched and developed in parallel, and then consolidated into se
| Seq. | Code Name | Fork Epoch | Specs |
| - | - | - | - |
| 0 | **Phase0** |`0` | <ul><li>Core</li><ul><li>[The beacon chain](specs/phase0/beacon-chain.md)</li><li>[Deposit contract](specs/phase0/deposit-contract.md)</li><li>[Beacon chain fork choice](specs/phase0/fork-choice.md)</li></ul><li>Additions</li><ul><li>[Honest validator guide](specs/phase0/validator.md)</li><li>[P2P networking](specs/phase0/p2p-interface.md)</li><li>[Weak subjectivity](specs/phase0/weak-subjectivity.md)</li></ul></ul> |
| 1 | **Altair** | `74240` | <ul><li>Core</li><ul><li>[Beacon chain changes](specs/altair/beacon-chain.md)</li><li>[Altair fork](specs/altair/fork.md)</li></ul><li>Additions</li><ul><li>[Light client sync protocol](specs/altair/light-client/sync-protocol.md) ([full node](specs/altair/light-client/full-node.md), [light client](specs/altair/light-client/light-client.md))</li><li>[Honest validator guide changes](specs/altair/validator.md)</li><li>[P2P networking](specs/altair/p2p-interface.md)</li></ul></ul> |
| 1 | **Altair** | `74240` | <ul><li>Core</li><ul><li>[Beacon chain changes](specs/altair/beacon-chain.md)</li><li>[Altair fork](specs/altair/fork.md)</li></ul><li>Additions</li><ul><li>[Light client sync protocol](specs/altair/light-client/sync-protocol.md) ([full node](specs/altair/light-client/full-node.md), [light client](specs/altair/light-client/light-client.md), [networking](specs/altair/light-client/p2p-interface.md))</li><li>[Honest validator guide changes](specs/altair/validator.md)</li><li>[P2P networking](specs/altair/p2p-interface.md)</li></ul></ul> |
| 2 | **Bellatrix** <br/> (["The Merge"](https://ethereum.org/en/upgrades/merge/)) | TBD | <ul><li>Core</li><ul><li>[Beacon Chain changes](specs/bellatrix/beacon-chain.md)</li><li>[Bellatrix fork](specs/bellatrix/fork.md)</li><li>[Fork choice changes](specs/bellatrix/fork-choice.md)</li></ul><li>Additions</li><ul><li>[Honest validator guide changes](specs/bellatrix/validator.md)</li><li>[P2P networking](specs/bellatrix/p2p-interface.md)</li></ul></ul> |

### In-development Specifications
Expand Down
1 change: 1 addition & 0 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -935,6 +935,7 @@ def finalize_options(self):
self.md_doc_paths += """
specs/altair/light-client/full-node.md
specs/altair/light-client/light-client.md
specs/altair/light-client/p2p-interface.md
specs/altair/light-client/sync-protocol.md
specs/altair/beacon-chain.md
specs/altair/bls.md
Expand Down
257 changes: 257 additions & 0 deletions specs/altair/light-client/p2p-interface.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,257 @@
# Altair Light Client -- Networking

**Notice**: This document is a work-in-progress for researchers and implementers.

## Table of contents

<!-- TOC -->
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

- [Networking](#networking)
- [Configuration](#configuration)
- [The gossip domain: gossipsub](#the-gossip-domain-gossipsub)
- [Topics and messages](#topics-and-messages)
- [Global topics](#global-topics)
- [`light_client_finality_update`](#light_client_finality_update)
- [`light_client_optimistic_update`](#light_client_optimistic_update)
- [The Req/Resp domain](#the-reqresp-domain)
- [Messages](#messages)
- [GetLightClientBootstrap](#getlightclientbootstrap)
- [LightClientUpdatesByRange](#lightclientupdatesbyrange)
- [GetLightClientFinalityUpdate](#getlightclientfinalityupdate)
- [GetLightClientOptimisticUpdate](#getlightclientoptimisticupdate)
- [Light clients](#light-clients)
- [Validator assignments](#validator-assignments)
- [Beacon chain responsibilities](#beacon-chain-responsibilities)
- [Sync committee](#sync-committee)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- /TOC -->

## Networking

This section extends the [networking specification for Altair](../p2p-interface.md) with additional messages, topics and data to the Req-Resp and Gossip domains.

### Configuration

| Name | Value | Description |
| - | - | - |
| `MAX_REQUEST_LIGHT_CLIENT_UPDATES` | `2**7` (= 128) | Maximum number of `LightClientUpdate` instances in a single request |

### The gossip domain: gossipsub

Gossip meshes are added to allow light clients stay in sync with the network.
etan-status marked this conversation as resolved.
Show resolved Hide resolved

#### Topics and messages

New global topics are added to provide light clients with the latest updates.

| name | Message Type |
| - | - |
| `light_client_finality_update` | `LightClientFinalityUpdate` |
| `light_client_optimistic_update` | `LightClientOptimisticUpdate` |

##### Global topics

###### `light_client_finality_update`

This topic is used to propagate the latest `LightClientFinalityUpdate` to light clients, allowing them to keep track of the latest `finalized_header`.

The following validations MUST pass before forwarding the `finality_update` on the network.
- _[IGNORE]_ No other `finality_update` with a lower or equal `finalized_header.slot` was already forwarded on the network
- _[IGNORE]_ The `finality_update` is received after the block at `signature_slot` was given enough time to propagate through the network -- i.e. validate that one-third of `finality_update.signature_slot` has transpired (`SECONDS_PER_SLOT / INTERVALS_PER_SLOT` seconds after the start of the slot, with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance)

For full nodes, the following validations MUST additionally pass before forwarding the `finality_update` on the network.
- _[IGNORE]_ The received `finality_update` matches the locally computed one exactly (as defined in [`create_light_client_finality_update`](./full-node.md#create_light_client_finality_update))

For light clients, the following validations MUST additionally pass before forwarding the `finality_update` on the network.
- _[REJECT]_ The `finality_update` is valid -- i.e. validate that `process_light_client_finality_update` does not indicate errors
- _[IGNORE]_ The `finality_update` advances the `finalized_header` of the local `LightClientStore` -- i.e. validate that processing `finality_update` increases `store.finalized_header.slot`

Light clients SHOULD call `process_light_client_finality_update` even if the message is ignored.

###### `light_client_optimistic_update`

This topic is used to propagate the latest `LightClientOptimisticUpdate` to light clients, allowing them to keep track of the latest `optimistic_header`.

The following validations MUST pass before forwarding the `optimistic_update` on the network.
- _[IGNORE]_ No other `optimistic_update` with a lower or equal `attested_header.slot` was already forwarded on the network
- _[IGNORE]_ The `optimistic_update` is received after the block at `signature_slot` was given enough time to propagate through the network -- i.e. validate that one-third of `optimistic_update.signature_slot` has transpired (`SECONDS_PER_SLOT / INTERVALS_PER_SLOT` seconds after the start of the slot, with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance)

For full nodes, the following validations MUST additionally pass before forwarding the `optimistic_update` on the network.
- _[IGNORE]_ The received `optimistic_update` matches the locally computed one exactly (as defined in [`create_light_client_optimistic_update`](./full-node.md#create_light_client_optimistic_update))

For light clients, the following validations MUST additionally pass before forwarding the `optimistic_update` on the network.
- _[REJECT]_ The `optimistic_update` is valid -- i.e. validate that `process_light_client_optimistic_update` does not indicate errors
- _[IGNORE]_ The `optimistic_update` either matches corresponding fields of the most recently forwarded `LightClientFinalityUpdate` (if any), or it advances the `optimistic_header` of the local `LightClientStore` -- i.e. validate that processing `optimistic_update` increases `store.optimistic_header.slot`

Light clients SHOULD call `process_light_client_optimistic_update` even if the message is ignored.

### The Req/Resp domain

#### Messages

##### GetLightClientBootstrap

**Protocol ID:** `/eth2/beacon_chain/req/light_client_bootstrap/1/`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, should it be beacon_chain/ or light_client/?
I see that it's a "beacon chain" light client, but for implementers & service providers, maybe there some benefits of distinguishing them?

@etan-status what do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong opinion on this.

beacon_chain fits for these reasons:

  • What is being downloaded are partial beacon blocks / beacon states
  • The other light client related messages, i.e., sync contributions, are also rooted in beacon_chain
  • The light client still needs to deal with the status, ping, metadata, goodbye messages, so it cannot solely use light_client, at least on libp2p

light_client fits for these reasons:

  • The topics are not necessary for collecting validator rewards
  • Easier to reason starting point to direct a client implementation to

Other impacted API should also be considered, e.g., REST, should it have it under the light client umbrella, or the beacon chain one? What about portal?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with either way too! Let's keep it to beacon_chain now unless someone has a strong objection one day.


Request Content:

```
(
Root
)
```

Response Content:

```
(
LightClientBootstrap
)
```

Requests the `LightClientBootstrap` structure corresponding to a given post-Altair beacon block root.

The request MUST be encoded as an SSZ-field.

Peers SHOULD provide results as defined in [`create_light_client_bootstrap`](./full-node.md#create_light_client_bootstrap). To fulfill a request, the requested block's post state needs to be known.

When a `LightClientBootstrap` instance cannot be produced for a given block root, peers SHOULD respond with error code `3: ResourceUnavailable`.

A `ForkDigest`-context based on `compute_fork_version(compute_epoch_at_slot(bootstrap.header.slot))` is used to select the fork namespace of the Response type.

Per `context = compute_fork_digest(fork_version, genesis_validators_root)`:

[0]: # (eth2spec: skip)

| `fork_version` | Response SSZ type |
| ------------------------------- | ------------------------------------ |
| `GENESIS_FORK_VERSION` | n/a |
| `ALTAIR_FORK_VERSION` and later | `altair.LightClientBootstrap` |

##### LightClientUpdatesByRange

**Protocol ID:** `/eth2/beacon_chain/req/light_client_updates_by_range/1/`

Request Content:
```
(
start_period: uint64
count: uint64
)
```

Response Content:
```
(
List[LightClientUpdate, MAX_REQUEST_LIGHT_CLIENT_UPDATES]
)
```

Requests the `LightClientUpdate` instances in the sync committee period range `[start_period, start_period + count)`, leading up to the current head sync committee period as selected by fork choice.

The request MUST be encoded as an SSZ-container.

The response MUST consist of zero or more `response_chunk`. Each _successful_ `response_chunk` MUST contain a single `LightClientUpdate` payload.

Peers SHOULD provide results as defined in [`create_light_client_update`](./full-node.md#create_light_client_update). They MUST respond with at least the earliest known result within the requested range, and MUST send results in consecutive order (by period). The response MUST NOT contain more than `min(MAX_REQUEST_LIGHT_CLIENT_UPDATES, count)` results.

For each `response_chunk`, a `ForkDigest`-context based on `compute_fork_version(compute_epoch_at_slot(update.attested_header.slot))` is used to select the fork namespace of the Response type. Note that this `fork_version` may be different from the one used to verify the `update.sync_aggregate`, which is based on `update.signature_slot`.

Per `context = compute_fork_digest(fork_version, genesis_validators_root)`:

[0]: # (eth2spec: skip)

| `fork_version` | Response chunk SSZ type |
| ------------------------------- | ------------------------------------ |
| `GENESIS_FORK_VERSION` | n/a |
| `ALTAIR_FORK_VERSION` and later | `altair.LightClientUpdate` |

##### GetLightClientFinalityUpdate

**Protocol ID:** `/eth2/beacon_chain/req/light_client_finality_update/1/`

No Request Content.

Response Content:

```
(
LightClientFinalityUpdate
)
```

Requests the latest `LightClientFinalityUpdate` known by a peer.

Peers SHOULD provide results as defined in [`create_light_client_finality_update`](./full-node.md#create_light_client_finality_update).

When no `LightClientFinalityUpdate` is available, peers SHOULD respond with error code `3: ResourceUnavailable`.

A `ForkDigest`-context based on `compute_fork_version(compute_epoch_at_slot(finality_update.attested_header.slot))` is used to select the fork namespace of the Response type. Note that this `fork_version` may be different from the one used to verify the `finality_update.sync_aggregate`, which is based on `finality_update.signature_slot`.

Per `context = compute_fork_digest(fork_version, genesis_validators_root)`:

[0]: # (eth2spec: skip)

| `fork_version` | Response SSZ type |
| ------------------------------- | ------------------------------------ |
| `GENESIS_FORK_VERSION` | n/a |
| `ALTAIR_FORK_VERSION` and later | `altair.LightClientFinalityUpdate` |

##### GetLightClientOptimisticUpdate

**Protocol ID:** `/eth2/beacon_chain/req/light_client_optimistic_update/1/`

No Request Content.

Response Content:

```
(
LightClientOptimisticUpdate
)
```

Requests the latest `LightClientOptimisticUpdate` known by a peer.

Peers SHOULD provide results as defined in [`create_light_client_optimistic_update`](./full-node.md#create_light_client_optimistic_update).

When no `LightClientOptimisticUpdate` is available, peers SHOULD respond with error code `3: ResourceUnavailable`.

A `ForkDigest`-context based on `compute_fork_version(compute_epoch_at_slot(optimistic_update.attested_header.slot))` is used to select the fork namespace of the Response type. Note that this `fork_version` may be different from the one used to verify the `optimistic_update.sync_aggregate`, which is based on `optimistic_update.signature_slot`.

Per `context = compute_fork_digest(fork_version, genesis_validators_root)`:

[0]: # (eth2spec: skip)

| `fork_version` | Response SSZ type |
| ------------------------------- | ------------------------------------ |
| `GENESIS_FORK_VERSION` | n/a |
| `ALTAIR_FORK_VERSION` and later | `altair.LightClientOptimisticUpdate` |

## Light clients

Light clients using libp2p to stay in sync with the network SHOULD subscribe to the [`light_client_finality_update`](#light_client_finality_update) and [`light_client_optimistic_update`](#light_client_optimistic_update) pubsub topics and validate all received messages while the [light client sync process](./light-client.md#light-client-sync-process) supports processing `LightClientFinalityUpdate` and `LightClientOptimistic` structures.

Light clients MAY also collect historic light client data and make it available to other peers. If they do, they SHOULD advertise supported message endpoints in [the Req/Resp domain](#the-reqresp-domain), and MAY also update the contents of their [`Status`](../../phase0/p2p-interface.md#status) message to reflect the locally available light client data.

If only limited light client data is locally available, the light client SHOULD use data based on `genesis_block` and `GENESIS_SLOT` in its `Status` message. Hybrid peers that also implement full node functionality MUST only incorporate data based on their full node sync progress into their `Status` message.

## Validator assignments

This section extends the [honest validator specification](../validator.md) with additional responsibilities to enable light clients to sync with the network.

### Beacon chain responsibilities

All full nodes SHOULD subscribe to and provide stability on the [`light_client_finality_update`](#light_client_finality_update) and [`light_client_optimistic_update`](#light_client_optimistic_update) pubsub topics by validating all received messages.

### Sync committee

Whenever fork choice selects a new head block with a sync aggregate participation `>= MIN_SYNC_COMMITTEE_PARTICIPANTS` and a post-Altair parent block, full nodes with at least one validator assigned to the current sync committee at the block's `slot` SHOULD broadcast derived light client data as follows:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: There may be some room to optimize it, based on how many full nodes enable the light client protocol.

e.g., if most nodes are altruistic and enable light client protocol, maybe we only need to assign the sync committee aggregators to publish the updates.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it may work with just aggregators, but it's too early to tell. Also have to consider that the topic also extends beyond just full nodes, with all light clients also being subscribed to it, so I erred towards having a few more broadcasts for now. Originally, I implemented it with just the proposer sending the update, but that has the obvious downside of the slot not appearing at all if the proposer does not implement the light client sync protocol – in general, it still worked surprisingly well though.

The messages involved here are tiny (BeaconBlockHeader + SyncAggregate essentially), and the additional broadcast lines up with the general concept to reward the sync committee for enabling light clients to sync. Furthermore, the sync committee already sends messages at 1/3 into the block, so sending the second message around that time also groups up those tasks nicely.


- If `finalized_header.slot` increased, a `LightClientFinalityUpdate` SHOULD be broadcasted to the pubsub topic `light_client_finality_update` if no matching message has not yet been forwarded as part of gossip validation.
- If `attested_header.slot` increased, a `LightClientOptimisticUpdate` SHOULD be broadcasted to the pubsub topic `light_client_optimistic_update` if no matching message has not yet been forwarded as part of gossip validation.

These messages SHOULD be broadcasted after one-third of `slot` has transpired (`SECONDS_PER_SLOT / INTERVALS_PER_SLOT` seconds after the start of the slot). To ensure that the corresponding block was given enough time to propagate through the network, they SHOULD NOT be sent earlier. Note that this is different from how other messages are handled, e.g., attestations, which may be sent early.
1 change: 1 addition & 0 deletions specs/altair/light-client/sync-protocol.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ uses sync committees introduced in [this beacon chain extension](./beacon-chain.
Additional documents describe how the light client sync protocol can be used:
- [Full node](./full-node.md)
- [Light client](./light-client.md)
- [Networking](./p2p-interface.md)

## Constants

Expand Down