Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Peer backup storage (feature 40/41/42/43) #881

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Peer backup storage (feature 40/41/42/43) #881

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions 01-messaging.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ All data fields are unsigned big-endian unless otherwise specified.
* [The `error` and `warning` Messages](#the-error-and-warning-messages)
* [Control Messages](#control-messages)
* [The `ping` and `pong` Messages](#the-ping-and-pong-messages)
* [Node Backup Storage](#node-backup-storage)
* [Appendix A: BigSize Test Vectors](#appendix-a-bigsize-test-vectors)
* [Appendix B: Type-Length-Value Test Vectors](#appendix-b-type-length-value-test-vectors)
* [Appendix C: Message Extension](#appendix-c-message-extension)
Expand Down Expand Up @@ -264,13 +265,19 @@ The `features` field MUST be padded to bytes with 0s.
1. type: 1 (`networks`)
2. data:
* [`...*chain_hash`:`chains`]
1. type: 2 (`node_backup`)
2. data:
* [`...*byte`:`backup_data`]
1. type: 3 (`remote_addr`)
2. data:
* [`...*byte`:`data`]

The optional `networks` indicates the chains the node is interested in.
The optional `remote_addr` can be used to circumvent NAT issues.

The `node_backup` contains the last backup our peer sent us, as described in
the [Node Backup Storage](#node-backup-storage) section.

#### Requirements

The sending node:
Expand Down Expand Up @@ -473,6 +480,55 @@ every message maximally).
Finally, the usage of periodic `ping` messages serves to promote frequent key
rotations as specified within [BOLT #8](08-transport.md).

## Node Backup Storage

Nodes that advertise the `provide_peer_backup_storage` feature offer storing
arbitrary data for their peers. The data stored must not exceed 32000 bytes,
which lets it fit in existing lightning messages (e.g. an `init` message).

Nodes that advertise the `want_peer_backup_storage` feature want to have their
data stored by their peers that support `provide_peer_backup_storage`.

Nodes can verify that their `provide_peer_backup_storage` peers correctly store
their backup data at each reconnection, by comparing the contents of the
retrieved backups with the last one they sent. However, nodes should not expect
their peers to always have their latest backup available (note that there is no
acknowledgement to ensure a backup has been correctly stored).

There are two types of backups:
- `node_backup`, described in this section
- `channel_backup`, described in [Bolt #2](02-peer-protocol.md#channel-backup-storage)

Nodes ask their peers to store data using the `update_node_backup` message:

1. type: 20 (`update_node_backup`)
2. data:
* [`u16`: `length`]
* [`length*byte`:`node_backup`]

A node with `want_peer_backup_storage` activated:
- if its peer doesn't support `provide_peer_backup_storage`:
- MUST NOT send `update_node_backup`
- otherwise:
- MAY send `update_node_backup` whenever necessary
- MUST limit its `node_backup` to 32000 bytes
- when it receives `init` with an outdated or missing `node_backup`:
- MAY send a warning

A node with `provide_peer_backup_storage` activated:
- when it receives `update_node_backup`:
- if it doesn't have any channel with the sender:
- MAY ignore the message
- if the `node_backup` exceeds 32000 bytes:
- SHOULD send a warning
- MUST NOT store this backup data
- otherwise:
- SHOULD store this backup data
t-bast marked this conversation as resolved.
Show resolved Hide resolved
- when it sends `init`:
- MUST include the last `node_backup` it stored for that peer
- when all channels with that peer are closed:
- SHOULD wait at least 2016 blocks before deleting the `node_backup`

## Appendix A: BigSize Test Vectors

The following test vectors can be used to assert the correctness of a BigSize
Expand Down
53 changes: 53 additions & 0 deletions 02-peer-protocol.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ operation, and closing.
* [Completing the Transition to the Updated State: `revoke_and_ack`](#completing-the-transition-to-the-updated-state-revoke_and_ack)
* [Updating Fees: `update_fee`](#updating-fees-update_fee)
* [Message Retransmission: `channel_reestablish` message](#message-retransmission)
* [Channel Backup Storage](#channel-backup-storage)
* [Authors](#authors)

# Channel
Expand Down Expand Up @@ -1310,6 +1311,12 @@ messages are), they are independent of requirements here.
* [`32*byte`:`your_last_per_commitment_secret`]
* [`point`:`my_current_per_commitment_point`]

1. `tlv_stream`: `channel_reestablish_tlvs`
2. types:
1. type: 1 (`channel_backup`)
2. data:
* [`...*byte`:`backup_data`]

`next_commitment_number`: A commitment number is a 48-bit
incrementing counter for each commitment transaction; counters
are independent for each peer in the channel and start at 0.
Expand Down Expand Up @@ -1515,6 +1522,52 @@ fall-behind detection. An implementation can offer both, however, and
fall back to the `option_data_loss_protect` behavior if
`option_static_remotekey` is not negotiated.

## Channel Backup Storage

Nodes that advertise the `provide_peer_backup_storage` feature offer storing
arbitrary data for their peers. [BOLT #1](01-messaging.md#node-backup-storage)
describes how this allows storing per-node data, and the following section
describes how this allows storing per-channel data.

Nodes ask their peers to store data using the `update_channel_backup` message:

1. type: 120 (`update_channel_backup`)
2. data:
* [`u16`: `length`]
* [`length*byte`:`channel_backup`]

A node with `want_peer_backup_storage` activated:
- if its peer doesn't support `provide_peer_backup_storage`:
- MUST NOT send `update_channel_backup`
- otherwise:
- when it sends a message that completes an update of the channel state (`funding_signed`, `commitment_signed`, `revoke_and_ack`, `shutdown` or `closing_signed`):
- MAY send an `update_channel_backup` *before* the channel update message
- MUST limit its `channel_backup` to 32000 bytes
- when it receives `channel_reestablish` with an outdated or missing `channel_backup`:
- MAY send a warning

A node with `provide_peer_backup_storage` activated:
- when it receives `update_channel_backup`:
- if the `channel_backup` exceeds 32000 bytes:
- SHOULD send a warning
- MUST NOT store this backup data
- otherwise:
- SHOULD store this backup data before storing the next channel state
change
- when it sends `channel_reestablish`:
- MUST include the last `channel_backup` it stored for that channel
- when the channel is closed:
- SHOULD wait at least 2016 blocks before deleting the `channel_backup`

### Rationale

The `update_channel_backup` should be sent before the message that updates the
channel state (e.g. `commit_sig`) to ensure that the backup is stored with that
latest update, which can be retransmitted on reconnection. If the backup is
sent after the channel update, there is a risk that the channel update is
committed by the remote peer but not the corresponding backup, which may lead
to data loss.

# Authors

[ FIXME: Insert Author List ]
Expand Down
48 changes: 29 additions & 19 deletions 09-features.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,23 +26,25 @@ The Context column decodes as follows:
* `C+`: presented in the `channel_announcement` message, but always even (required).
* `9`: presented in [BOLT 11](11-payment-encoding.md) invoices.

| Bits | Name | Description | Context | Dependencies | Link |
|-------|----------------------------------|-----------------------------------------------------------|----------|-------------------|---------------------------------------|
| 0/1 | `option_data_loss_protect` | Requires or supports extra `channel_reestablish` fields | IN | | [BOLT #2][bolt02-retransmit] |
| 3 | `initial_routing_sync` | Sending node needs a complete routing information dump | I | | [BOLT #7][bolt07-sync] |
| 4/5 | `option_upfront_shutdown_script` | Commits to a shutdown scriptpubkey when opening channel | IN | | [BOLT #2][bolt02-open] |
| 6/7 | `gossip_queries` | More sophisticated gossip control | IN | | [BOLT #7][bolt07-query] |
| 8/9 | `var_onion_optin` | Requires/supports variable-length routing onion payloads | IN9 | | [Routing Onion Specification][bolt04] |
| 10/11 | `gossip_queries_ex` | Gossip queries can include additional information | IN | `gossip_queries` | [BOLT #7][bolt07-query] |
| 12/13 | `option_static_remotekey` | Static key for remote output | IN | | [BOLT #3](03-transactions.md) |
| 14/15 | `payment_secret` | Node supports `payment_secret` field | IN9 | `var_onion_optin` | [Routing Onion Specification][bolt04] |
| 16/17 | `basic_mpp` | Node can receive basic multi-part payments | IN9 | `payment_secret` | [BOLT #4][bolt04-mpp] |
| 18/19 | `option_support_large_channel` | Can create large channels | IN | | [BOLT #2](02-peer-protocol.md#the-open_channel-message) |
| 20/21 | `option_anchor_outputs` | Anchor outputs | IN | `option_static_remotekey` | [BOLT #3](03-transactions.md) |
| 22/23 | `option_anchors_zero_fee_htlc_tx` | Anchor commitment type with zero fee HTLC transactions | IN | `option_static_remotekey` | [BOLT #3][bolt03-htlc-tx], [lightning-dev][ml-sighash-single-harmful]|
| 26/27 | `option_shutdown_anysegwit` | Future segwit versions allowed in `shutdown` | IN | | [BOLT #2][bolt02-shutdown] |
| 44/45 | `option_channel_type` | Node supports the `channel_type` field in open/accept | IN | | [BOLT #2](02-peer-protocol.md#the-open_channel-message) |
| 48/49 | `option_payment_metadata` | Payment metadata in tlv record | 9 | | [BOLT #11](11-payment-encoding.md#tagged-fields)
| Bits | Name | Description | Context | Dependencies | Link |
|-------|-----------------------------------|-----------------------------------------------------------|----------|---------------------------|-----------------------------------------------------------------------|
| 0/1 | `option_data_loss_protect` | Requires or supports extra `channel_reestablish` fields | IN | | [BOLT #2][bolt02-retransmit] |
| 3 | `initial_routing_sync` | Sending node needs a complete routing information dump | I | | [BOLT #7][bolt07-sync] |
| 4/5 | `option_upfront_shutdown_script` | Commits to a shutdown scriptpubkey when opening channel | IN | | [BOLT #2][bolt02-open] |
| 6/7 | `gossip_queries` | More sophisticated gossip control | IN | | [BOLT #7][bolt07-query] |
| 8/9 | `var_onion_optin` | Requires/supports variable-length routing onion payloads | IN9 | | [Routing Onion Specification][bolt04] |
| 10/11 | `gossip_queries_ex` | Gossip queries can include additional information | IN | `gossip_queries` | [BOLT #7][bolt07-query] |
| 12/13 | `option_static_remotekey` | Static key for remote output | IN | | [BOLT #3](03-transactions.md) |
| 14/15 | `payment_secret` | Node supports `payment_secret` field | IN9 | `var_onion_optin` | [Routing Onion Specification][bolt04] |
| 16/17 | `basic_mpp` | Node can receive basic multi-part payments | IN9 | `payment_secret` | [BOLT #4][bolt04-mpp] |
| 18/19 | `option_support_large_channel` | Can create large channels | IN | | [BOLT #2](02-peer-protocol.md#the-open_channel-message) |
| 20/21 | `option_anchor_outputs` | Anchor outputs | IN | `option_static_remotekey` | [BOLT #3](03-transactions.md) |
| 22/23 | `option_anchors_zero_fee_htlc_tx` | Anchor commitment type with zero fee HTLC transactions | IN | `option_static_remotekey` | [BOLT #3][bolt03-htlc-tx], [lightning-dev][ml-sighash-single-harmful] |
| 26/27 | `option_shutdown_anysegwit` | Future segwit versions allowed in `shutdown` | IN | | [BOLT #2][bolt02-shutdown] |
| 40/41 | `want_peer_backup_storage` | Want to use other nodes to store encrypted backup data | IN | | [BOLT #1][bolt01-node-backup], [BOLT #2][bolt02-channel-backup] |
| 42/43 | `provide_peer_backup_storage` | Can store other nodes' encrypted backup data | IN | | [BOLT #1][bolt01-node-backup], [BOLT #2][bolt02-channel-backup] |
| 44/45 | `option_channel_type` | Node supports the `channel_type` field in open/accept | IN | | [BOLT #2](02-peer-protocol.md#the-open_channel-message) |
| 48/49 | `option_payment_metadata` | Payment metadata in tlv record | 9 | | [BOLT #11](11-payment-encoding.md#tagged-fields) |

## Definitions

Expand All @@ -53,7 +55,7 @@ We define `option_anchors` as `option_anchor_outputs || option_anchors_zero_fee_
The origin node:
* If it supports a feature above, SHOULD set the corresponding odd
bit in all feature fields indicated by the Context column unless
indicated that it must set the even feature bit instead.
indicated that it must set the even feature bit instead.
* If it requires a feature above, MUST set the corresponding even
feature bit in all feature fields indicated by the Context column,
unless indicated that it must set the odd feature bit instead.
Expand Down Expand Up @@ -82,16 +84,24 @@ well-formed feature vector. By validating all known dependencies up front, this
simplifies logic gated on a single feature bit; the feature's dependencies are
known to be set, and do not need to be validated at every feature gate.

Some features are asymmetric and come in pairs (e.g. `want_peer_backup_storage`
and `provide_peer_backup_storage`), where some peers may want to use a feature
that other peers provide, without providing it themselves. Nodes that want to
only connect to peers who provide such feature should set the `want_XXX` bit to
mandatory.

![Creative Commons License](https://i.creativecommons.org/l/by/4.0/88x31.png "License CC-BY")
<br>
This work is licensed under a [Creative Commons Attribution 4.0 International License](http://creativecommons.org/licenses/by/4.0/).

[bolt01-node-backup]: 01-messaging.md#node-backup-storage
[bolt02-retransmit]: 02-peer-protocol.md#message-retransmission
[bolt02-open]: 02-peer-protocol.md#the-open_channel-message
[bolt03-htlc-tx]: 03-transactions.md#htlc-timeout-and-htlc-success-transactions
[bolt02-shutdown]: 02-peer-protocol.md#closing-initiation-shutdown
[bolt02-channel-backup]: 02-peer-protocol.md#channel-backup-storage
[bolt04]: 04-onion-routing.md
[bolt04-mpp]: 04-onion-routing.md#basic-multi-part-payments
[bolt07-sync]: 07-routing-gossip.md#initial-sync
[bolt07-query]: 07-routing-gossip.md#query-messages
[bolt04-mpp]: 04-onion-routing.md#basic-multi-part-payments
[ml-sighash-single-harmful]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-September/002796.html