From eadc50d6724f6ed2123e7554133d1e287807e160 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Mon, 15 Jul 2024 19:20:03 +0300 Subject: [PATCH 01/24] draft Signed-off-by: Andrei Sandu --- text/0102-introduce-core-index-commitment.md | 191 +++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 text/0102-introduce-core-index-commitment.md diff --git a/text/0102-introduce-core-index-commitment.md b/text/0102-introduce-core-index-commitment.md new file mode 100644 index 000000000..d340270f4 --- /dev/null +++ b/text/0102-introduce-core-index-commitment.md @@ -0,0 +1,191 @@ +# RFC-0102: Introduce a `CoreIndex` commitment in candidate receipts + +| | | +| --------------- | ------------------------------------------------------------------------------------------- | +| **Start Date** | 15 July 2024 | +| **Description** | Constrain parachain block validity on a single core | +| **Authors** | Andrei Sandu | + +## Summary +The only requirement for collator nodes is to provide valid parachain blocks to the validators of a backing group and by definition the collator set is trustless. However, in the case of elastic scaling, for security reason, collators must be trusted - non-malicious. `CoreIndex` commitments are required to remove this limitation. + +## Motivation + +At present time misbehaving collator nodes can prevent their parachain from effecitvely using elastic scaling by providing the same valid block to all backing groups assigned to the parachain. This happens before the next parachain block is authored and will prevent the chain of candidates to be formed, reducing the throughput of the parachain to a single core. +There are no special requirements from collators to do it, just being a full node is sufficient and there are no methods of punishing or rewarding good behaviour. + +This RFC solves the problem by enabling a parachain to provide the core index information as part of it's PVF execution output and in the associated candidate receipt data structure. + +Once this RFC is implemented the validity of a parachain block depends on the core it is being executed on. + +## Stakeholders + +- Polkadot core developers. +- Cumulus node developers. +- Tooling, block explorer developers. + +This approach and alternatives have been considered and discussed in [this issue](https://github.com/polkadot-fellows/RFCs/issues/92). + +## Explanation + +The approach proposed below was chosen primarly because it minimizes the number of breaking changes, the complexity and takes far less implementation and testing time. The proposal is to free up space and introduce a new core index field in the `CandidateDescriptor` primitive and use the UMP queue as output for `CoreIndex` commitment. + +### Reclaiming unused space in the descriptor +The `CandidateDescriptor` currently includes `collator` and `signature` fields. The collator includes a signature on the following descriptor fields: parachain id, relay parent, validation data hash, validation code hash and the PoV hash. + +However, in practice, having a collator signature in the receipt on the relay chain does not provide any benefits as there is no mechanism to punish or reward collators that have provided bad parachain blocks. + +This proposal aims to remove the two fields and all the logic that checks the collator signatures. We reclaim the unused space as `reserved` fields and fill it with zeroes, so there is no change in the layout and lenght of the receipt. The new primitive binary compatible with the old one. + + +### Backwards compatibility + +There are two flavors of candidate receipts which are used in network protocols, runtime and node implementation: +- `CommittedCandidateReceipt` which includes the `CanidateDescriptor` and the `CandidateCommitments` +- `CandidateReceipt` which includes the `CanidateDescriptor` and just a hash of the commitments + +We want to support both the old and new versions in the runtime and node . The implementation must be able to detect the version of a given candidate receipt. + +This is easy to do in both cases: +- the reserved fields are zeroed +- the UMP queue contains the core index commitment that matches the core index in the descriptor. + + +### Polkadot Primitive changes + +#### New [CandidateDescriptor](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L482) + +- reclaim 32 bytes from `collator: CollatorId` and 64 bytes from `signature: CollatorSignature` as `reserved` +- use 4 bytes for a new `core_index: CoreIndex` field. +- the unused reclaimed space will be filled with zeroes + +Thew new primitive will look like this: +``` +pub struct CandidateDescriptor { + /// The ID of the para this is a candidate for. + pub para_id: Id, + /// The hash of the relay-chain block this is executed in the context of. + pub relay_parent: H, + /// The core index where the candidate is backed. + pub core_index: CoreIndex, + /// Reserved bytes. + pub reserved1: [u8; 28], + /// The blake2-256 hash of the persisted validation data. This is extra data derived from + /// relay-chain state which may vary based on bitfields included before the candidate. + /// Thus it cannot be derived entirely from the relay-parent. + pub persisted_validation_data_hash: Hash, + /// The blake2-256 hash of the PoV. + pub pov_hash: Hash, + /// The root of a block's erasure encoding Merkle tree. + pub erasure_root: Hash, + /// Reserved bytes. + pub reserved2: [u8; 64], + /// Hash of the para header that is being generated by this candidate. + pub para_head: Hash, + /// The blake2-256 hash of the validation code bytes. + pub validation_code_hash: ValidationCodeHash, +} + +``` + +In the future, parts of the `reserved1` and `reserved2` bytes can be used to include additional information in the descriptor. + +**Introduce new primitive for representing the `CoreIndex` commitment as an enum to allow future additions.** + + +``` +pub enum UMPSignal { + OnCore(CoreIndex), +} +``` +### Cumulus primitives + +Add a new version of the `ParachainInherentData` structure which includes an additional `core_index` field. +``` +pub struct ParachainInherentData { + pub validation_data: PersistedValidationData, + /// A storage proof of a predefined set of keys from the relay-chain. + /// + /// Specifically this witness contains the data for: + /// + /// - the current slot number at the given relay parent + /// - active host configuration as per the relay parent, + /// - the relay dispatch queue sizes + /// - the list of egress HRMP channels (in the list of recipients form) + /// - the metadata for the egress HRMP channels + pub relay_chain_state: sp_trie::StorageProof, + /// Downward messages in the order they were sent. + pub downward_messages: Vec, + /// HRMP messages grouped by channels. The messages in the inner vec must be in order they + /// were sent. In combination with the rule of no more than one message in a channel per block, + /// this means `sent_at` is **strictly** greater than the previous one (if any). + pub horizontal_messages: BTreeMap>, + /// The core index on which the parachain block must be backed + pub core_index: CoreIndex, +} +``` + +### UMP transport +[CandidateCommitments](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L652) remains unchanged as we will store scale encoded `UMPSignal` messages directly in the parachain UMP queue by outputing them in the [upward_messages](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L682). + + +The UMP queue layout is adjusted to allow the relay chain to receive both the XCM messages and `UMPSignal` messages. We will introduce a message separator that will be implemented as an empty `Vec`. + +The separator marks the end of the XCM messages and the begging of the `UMPSignal` messages. + +Example: +``` +[ XCM message1, XCM message2, ..., EMPTY message, UMPSignal::CoreIndex ] +``` + +### Parachain block validation +Backers will make use of the core index information to validate the blocks during backing and reject blocks if: +- the `core_index` in descriptor does not match the one in the `UMPSignal`. +- the `core_index` in the descriptor does not match the core the backing group is assigned to + +If core index information is not available (backers got an old candidate receipt), there will be no changes compared to current behaviour. + +## Drawbacks + +The only drawback is that further additions to the descriptor are limited to the amount of remaining unused space. + + +## Testing, Security, and Privacy + +Standard testing (unit tests, CI zombienet tests) for functionality and mandatory secuirty audit to ensure the implementation does not introduce any new security issues. + +Backwards compatibility of the implementation will be tested on testnets (Versi and Westend). + +There is no impact on privacy. + +## Performance + +The expectation is that performance impact is negligible for sending and processing the UMP message has negligible performance impact in the runtime as well as on the node side. + +## Ergonomics + +Parachain that use elastic scaling must send the separator empty message followed by the `UMPSignal::OnCore` only after sending all of the UMP XCM messages. + +## Compatibility + +To ensure a smooth transition the first step is to remove collator signature checking logic on the node side and upgrade validators. Any tooling that uses these fields will require similar changes as described above to support both formats. + +The runtime does not check the collator signature so there are no other specific concers for compatibility except adding the support for the new primitives. + + +`CoreIndex` commitments are mandatory only for parachains using elastic scaling. No new implementation of networking protocol versions for collation and validation are required. + + +## Prior Art and References + +Forum discussion about a new `CandidateReceipt` format: https://forum.polkadot.network/t/pre-rfc-discussion-candidate-receipt-format-v2/3738 + +## Unresolved Questions + +N/A + +## Future Directions and Related Material + +The implementation is extensible and future proof to some extent. With minimal or no breaking changes, additional fields can be added in the candidate descriptor until the reserved space is exhausted + +Once the reserved space is exhausted, versioning will be implemented. The candidate receipt format will be versioned. This will exteend to pvf execution which requires versioning for the validation function, inputs and outputs (`CandidateCommitments`). \ No newline at end of file From b20fd68654b90bb7f53666889f0ec6443fceedc5 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Mon, 15 Jul 2024 19:24:17 +0300 Subject: [PATCH 02/24] update number Signed-off-by: Andrei Sandu --- ...ex-commitment.md => 0103-introduce-core-index-commitment.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename text/{0102-introduce-core-index-commitment.md => 0103-introduce-core-index-commitment.md} (99%) diff --git a/text/0102-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md similarity index 99% rename from text/0102-introduce-core-index-commitment.md rename to text/0103-introduce-core-index-commitment.md index d340270f4..ec779311d 100644 --- a/text/0102-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -1,4 +1,4 @@ -# RFC-0102: Introduce a `CoreIndex` commitment in candidate receipts +# RFC-0103: Introduce a `CoreIndex` commitment in candidate receipts | | | | --------------- | ------------------------------------------------------------------------------------------- | From 9e754fc899ee89e431b10c95911f6f31a7a771d5 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Mon, 15 Jul 2024 19:25:37 +0300 Subject: [PATCH 03/24] fix Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index ec779311d..1b80a1244 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -3,7 +3,7 @@ | | | | --------------- | ------------------------------------------------------------------------------------------- | | **Start Date** | 15 July 2024 | -| **Description** | Constrain parachain block validity on a single core | +| **Description** | Constrain parachain block validity on a specific core | | **Authors** | Andrei Sandu | ## Summary From 7cff0ede6467287ca63eaf25f8cfde45323193d6 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Thu, 18 Jul 2024 02:31:47 +0300 Subject: [PATCH 04/24] correct error and rephrase Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 1b80a1244..e7b351990 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -168,12 +168,13 @@ Parachain that use elastic scaling must send the separator empty message followe ## Compatibility -To ensure a smooth transition the first step is to remove collator signature checking logic on the node side and upgrade validators. Any tooling that uses these fields will require similar changes as described above to support both formats. +To ensure a smooth transition the first step is to remove collator signature checking logic in the runtime and the node side, then upgrade validators. Any tooling that decodes UMP XCM messages needs an update to support the new UMP messages. -The runtime does not check the collator signature so there are no other specific concers for compatibility except adding the support for the new primitives. +`CoreIndex` commitments are needed only by parachains using elastic scaling. Just upgrading the collator node and runtime should be sufficient and possible with no manual changes. +No new implementation of networking protocol versions for collation and validation are required. -`CoreIndex` commitments are mandatory only for parachains using elastic scaling. No new implementation of networking protocol versions for collation and validation are required. +The runtime does check the collator signature in inclusion, so that should be removed as first step, before the new receipts are introduced. ## Prior Art and References From a404babc077d24bc6783c91b56b2267ba9b9eda7 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Mon, 29 Jul 2024 19:08:27 +0300 Subject: [PATCH 05/24] update Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 102 +++++++++++++------ 1 file changed, 73 insertions(+), 29 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index e7b351990..d8131aaaa 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -3,11 +3,11 @@ | | | | --------------- | ------------------------------------------------------------------------------------------- | | **Start Date** | 15 July 2024 | -| **Description** | Constrain parachain block validity on a specific core | +| **Description** | Constrain parachain block validity to a specific core and session | | **Authors** | Andrei Sandu | ## Summary -The only requirement for collator nodes is to provide valid parachain blocks to the validators of a backing group and by definition the collator set is trustless. However, in the case of elastic scaling, for security reason, collators must be trusted - non-malicious. `CoreIndex` commitments are required to remove this limitation. +The only requirement for collator nodes is to provide valid parachain blocks to the validators of a backing group and by definition the collator set is trustless. However, in the case of elastic scaling, for security reason, collators must be trusted - non-malicious. `CoreIndex` commitments are required to remove this limitation. Additionally we are introducing a `SessionIndex` field in the `CandidateReceipt` to make dispute resolution more secure and robust. ## Motivation @@ -28,14 +28,14 @@ This approach and alternatives have been considered and discussed in [this issue ## Explanation -The approach proposed below was chosen primarly because it minimizes the number of breaking changes, the complexity and takes far less implementation and testing time. The proposal is to free up space and introduce a new core index field in the `CandidateDescriptor` primitive and use the UMP queue as output for `CoreIndex` commitment. +The approach proposed below was chosen primarly because it minimizes the number of breaking changes, the complexity and takes far less implementation and testing time. The proposal is to free up space and introduce a core index and a session index field in the `CandidateDescriptor` primitive and use the UMP queue as output for `CoreIndex` commitment. ### Reclaiming unused space in the descriptor The `CandidateDescriptor` currently includes `collator` and `signature` fields. The collator includes a signature on the following descriptor fields: parachain id, relay parent, validation data hash, validation code hash and the PoV hash. However, in practice, having a collator signature in the receipt on the relay chain does not provide any benefits as there is no mechanism to punish or reward collators that have provided bad parachain blocks. -This proposal aims to remove the two fields and all the logic that checks the collator signatures. We reclaim the unused space as `reserved` fields and fill it with zeroes, so there is no change in the layout and lenght of the receipt. The new primitive binary compatible with the old one. +This proposal aims to remove the collator signature and all the logic that checks the collator signatures of candidate receupts. We use the first 6 bytes to represent the core and session index, and fill the rest with zeroes. So, there is no change in the layout and length of the receipt. The new primitive is binary compatible with the old one. ### Backwards compatibility @@ -44,60 +44,88 @@ There are two flavors of candidate receipts which are used in network protocols, - `CommittedCandidateReceipt` which includes the `CanidateDescriptor` and the `CandidateCommitments` - `CandidateReceipt` which includes the `CanidateDescriptor` and just a hash of the commitments -We want to support both the old and new versions in the runtime and node . The implementation must be able to detect the version of a given candidate receipt. +We want to support both the old and new versions in the runtime and node. The implementation must be able to detect the version of a given candidate receipt. This is easy to do in both cases: - the reserved fields are zeroed -- the UMP queue contains the core index commitment that matches the core index in the descriptor. - +- the UMP queue contains the core and session index commitments and the commitments value matching the ones in the descriptor. ### Polkadot Primitive changes #### New [CandidateDescriptor](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L482) -- reclaim 32 bytes from `collator: CollatorId` and 64 bytes from `signature: CollatorSignature` as `reserved` -- use 4 bytes for a new `core_index: CoreIndex` field. +- reclaim 32 bytes from `collator: CollatorId` and 64 bytes from `signature: CollatorSignature` and rename to `reserved1` and `reserved2` fields. +- take 2 bytes from `reserved1` for a new `core_index: u16` field. +- take 4 bytes from `reserved` for a new `session_index: u32` field. - the unused reclaimed space will be filled with zeroes Thew new primitive will look like this: ``` -pub struct CandidateDescriptor { +pub struct CandidateDescriptorV2 { /// The ID of the para this is a candidate for. - pub para_id: Id, + para_id: Id, /// The hash of the relay-chain block this is executed in the context of. - pub relay_parent: H, + relay_parent: H, /// The core index where the candidate is backed. - pub core_index: CoreIndex, + core_index: CoreIndex, + /// The session index in which the candidate is backed. + session_index: SessionIndex, /// Reserved bytes. - pub reserved1: [u8; 28], + reserved1: [u8; 26], /// The blake2-256 hash of the persisted validation data. This is extra data derived from /// relay-chain state which may vary based on bitfields included before the candidate. /// Thus it cannot be derived entirely from the relay-parent. - pub persisted_validation_data_hash: Hash, + persisted_validation_data_hash: Hash, /// The blake2-256 hash of the PoV. - pub pov_hash: Hash, + pov_hash: Hash, /// The root of a block's erasure encoding Merkle tree. - pub erasure_root: Hash, + erasure_root: Hash, /// Reserved bytes. - pub reserved2: [u8; 64], + reserved2: [u8; 64], /// Hash of the para header that is being generated by this candidate. - pub para_head: Hash, + para_head: Hash, /// The blake2-256 hash of the validation code bytes. - pub validation_code_hash: ValidationCodeHash, + validation_code_hash: ValidationCodeHash, +} +``` + +In future format versions, parts of the `reserved1` and `reserved2` bytes can be used to include additional information in the descriptor. + + +#### Versioned `CandidateReceipt` and `CommittedCandidateReceipt` primitives: + +We want to decouple the actual representation of the `CandidateReceipt` from the higher level code. This should make it easier to implement future format versions of this primitive. To hide the logic of versioning the descriptor fields will be private and accessor methods are provided. + +``` +pub enum VersionedCandidateReceipt { + V1(CandidateReceipt), + V2(CandidateReceiptV2), +} + +impl VersionedCandidateReceipt { + /// Returns the core index the candidate has commited to. + /// Returns `None` if the candidate receipt is the old version (v1). + fn core_index() -> Option; + + /// Returns the session index of the candidate relay parent. + fn session_index() -> Option; + + /// ... } ``` -In the future, parts of the `reserved1` and `reserved2` bytes can be used to include additional information in the descriptor. +A manual decode `Decode` implementation is required to account for version detection and constructing the appropriate variant. -**Introduce new primitive for representing the `CoreIndex` commitment as an enum to allow future additions.** +#### New primitive for representing the `CoreIndex` commitments.** ``` pub enum UMPSignal { OnCore(CoreIndex), } ``` + ### Cumulus primitives Add a new version of the `ParachainInherentData` structure which includes an additional `core_index` field. @@ -120,8 +148,8 @@ pub struct ParachainInherentData { /// were sent. In combination with the rule of no more than one message in a channel per block, /// this means `sent_at` is **strictly** greater than the previous one (if any). pub horizontal_messages: BTreeMap>, - /// The core index on which the parachain block must be backed - pub core_index: CoreIndex, + /// The core index on which the parachain block must be backed + pub core_index: CoreIndex, } ``` @@ -139,17 +167,18 @@ Example: ``` ### Parachain block validation + Backers will make use of the core index information to validate the blocks during backing and reject blocks if: - the `core_index` in descriptor does not match the one in the `UMPSignal`. - the `core_index` in the descriptor does not match the core the backing group is assigned to +- the `session_index` is equal to the session of the `relay_parent` in the descriptor -If core index information is not available (backers got an old candidate receipt), there will be no changes compared to current behaviour. +If core index (and session index) information is not available (backers got an old candidate receipt), there will be no changes compared to current behaviour. ## Drawbacks The only drawback is that further additions to the descriptor are limited to the amount of remaining unused space. - ## Testing, Security, and Privacy Standard testing (unit tests, CI zombienet tests) for functionality and mandatory secuirty audit to ensure the implementation does not introduce any new security issues. @@ -168,14 +197,29 @@ Parachain that use elastic scaling must send the separator empty message followe ## Compatibility -To ensure a smooth transition the first step is to remove collator signature checking logic in the runtime and the node side, then upgrade validators. Any tooling that decodes UMP XCM messages needs an update to support the new UMP messages. +### Runtime +The first step is to remove collator signature checking logic in the runtime, but keep the node side collator signature +checks. + +The runtime must be upgraded to the new primitives before any collator or node are allowed to use the new candidate receipts format. -`CoreIndex` commitments are needed only by parachains using elastic scaling. Just upgrading the collator node and runtime should be sufficient and possible with no manual changes. +### Validators + +To ensure a smooth launch, a new node feature is required. +The feature acts as a signal for supporting the new candidate receipts on the node side and can only be safely enabled if at least 2/3 of the validators are upgraded. + +Once enabled, the validators will skip checking the collator signature when processing the new candidate receipts. No new implementation of networking protocol versions for collation and validation are required. -The runtime does check the collator signature in inclusion, so that should be removed as first step, before the new receipts are introduced. +### Parachains + +`CoreIndex` commitments are needed only by parachains using elastic scaling. Just upgrading the collator node and runtime should be sufficient and possible without manual changes. + +### Tooling + +Any tooling that decodes UMP XCM messages needs an update to support or ignore the new UMP messages, but they should be fine to decode the regular XCM messages that come before the separator. ## Prior Art and References From 476d941c4064fea3a4a264d82954dd1b29a33f93 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Thu, 1 Aug 2024 17:50:05 +0300 Subject: [PATCH 06/24] feedback Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index d8131aaaa..28743308d 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -11,8 +11,7 @@ The only requirement for collator nodes is to provide valid parachain blocks to ## Motivation -At present time misbehaving collator nodes can prevent their parachain from effecitvely using elastic scaling by providing the same valid block to all backing groups assigned to the parachain. This happens before the next parachain block is authored and will prevent the chain of candidates to be formed, reducing the throughput of the parachain to a single core. -There are no special requirements from collators to do it, just being a full node is sufficient and there are no methods of punishing or rewarding good behaviour. +At present time misbehaving collator nodes, or anyone who has acquired a valid collation can prevent a parachain from effecitvely using elastic scaling by providing the same valid block to all backing groups assigned to the parachain. This happens before the next parachain block is authored and will prevent the chain of candidates to be formed, reducing the throughput of the parachain to a single core. This RFC solves the problem by enabling a parachain to provide the core index information as part of it's PVF execution output and in the associated candidate receipt data structure. @@ -35,7 +34,7 @@ The `CandidateDescriptor` currently includes `collator` and `signature` fields. However, in practice, having a collator signature in the receipt on the relay chain does not provide any benefits as there is no mechanism to punish or reward collators that have provided bad parachain blocks. -This proposal aims to remove the collator signature and all the logic that checks the collator signatures of candidate receupts. We use the first 6 bytes to represent the core and session index, and fill the rest with zeroes. So, there is no change in the layout and length of the receipt. The new primitive is binary compatible with the old one. +This proposal aims to remove the collator signature and all the logic that checks the collator signatures of candidate receipts. We use the first 6 bytes to represent the core and session index, and fill the rest with zeroes. So, there is no change in the layout and length of the receipt. The new primitive is binary compatible with the old one. ### Backwards compatibility @@ -56,8 +55,8 @@ This is easy to do in both cases: - reclaim 32 bytes from `collator: CollatorId` and 64 bytes from `signature: CollatorSignature` and rename to `reserved1` and `reserved2` fields. - take 2 bytes from `reserved1` for a new `core_index: u16` field. -- take 4 bytes from `reserved` for a new `session_index: u32` field. -- the unused reclaimed space will be filled with zeroes +- take 4 bytes from `reserved2` for a new `session_index: u32` field. +- the `reserved1` and `reserved2` fields are zeroed Thew new primitive will look like this: ``` @@ -208,7 +207,7 @@ The runtime must be upgraded to the new primitives before any collator or node a To ensure a smooth launch, a new node feature is required. The feature acts as a signal for supporting the new candidate receipts on the node side and can only be safely enabled if at least 2/3 of the validators are upgraded. -Once enabled, the validators will skip checking the collator signature when processing the new candidate receipts. +Once enabled, the validators will skip checking the collator signature when processing the candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if present in the receipit. No new implementation of networking protocol versions for collation and validation are required. From d429299e3201275da72693747ef434ead03ef108 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Mon, 5 Aug 2024 13:49:28 +0300 Subject: [PATCH 07/24] feedback Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 115 +++++++++---------- 1 file changed, 56 insertions(+), 59 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 28743308d..89ff5bb03 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -11,9 +11,9 @@ The only requirement for collator nodes is to provide valid parachain blocks to ## Motivation -At present time misbehaving collator nodes, or anyone who has acquired a valid collation can prevent a parachain from effecitvely using elastic scaling by providing the same valid block to all backing groups assigned to the parachain. This happens before the next parachain block is authored and will prevent the chain of candidates to be formed, reducing the throughput of the parachain to a single core. +At present time misbehaving collator nodes, or anyone who has acquired a valid collation can prevent a parachain from effecitvely using elastic scaling by providing the same collation to all backing groups assigned to the parachain. This happens before the next parachain block is authored and will prevent the chain of candidates to be formed, reducing the throughput of the parachain to a single core. -This RFC solves the problem by enabling a parachain to provide the core index information as part of it's PVF execution output and in the associated candidate receipt data structure. +This RFC solves the problem by enabling a parachain to provide a core index commitment as part of it's PVF execution output and in the associated candidate receipt data structure. Once this RFC is implemented the validity of a parachain block depends on the core it is being executed on. @@ -27,15 +27,14 @@ This approach and alternatives have been considered and discussed in [this issue ## Explanation -The approach proposed below was chosen primarly because it minimizes the number of breaking changes, the complexity and takes far less implementation and testing time. The proposal is to free up space and introduce a core index and a session index field in the `CandidateDescriptor` primitive and use the UMP queue as output for `CoreIndex` commitment. +The approach proposed below was chosen primarly because it minimizes the number of breaking changes, the complexity and takes less implementation and testing time. The proposal is to change the existing primitives while keeping binary compatibility with the older versions. We repurpose unused fields to introduce core index and a session index information in the `CandidateDescriptor` and extend the UMP usage to output core index information. ### Reclaiming unused space in the descriptor The `CandidateDescriptor` currently includes `collator` and `signature` fields. The collator includes a signature on the following descriptor fields: parachain id, relay parent, validation data hash, validation code hash and the PoV hash. However, in practice, having a collator signature in the receipt on the relay chain does not provide any benefits as there is no mechanism to punish or reward collators that have provided bad parachain blocks. -This proposal aims to remove the collator signature and all the logic that checks the collator signatures of candidate receipts. We use the first 6 bytes to represent the core and session index, and fill the rest with zeroes. So, there is no change in the layout and length of the receipt. The new primitive is binary compatible with the old one. - +This proposal aims to remove the collator signature and all the logic that checks the collator signatures of candidate receipts. We use the first 6 reclaimed bytes to represent the core and session index, and fill the rest with zeroes. So, there is no change in the layout and length of the receipt. The new primitive is binary compatible with the old one. ### Backwards compatibility @@ -45,9 +44,57 @@ There are two flavors of candidate receipts which are used in network protocols, We want to support both the old and new versions in the runtime and node. The implementation must be able to detect the version of a given candidate receipt. -This is easy to do in both cases: +`CandidateDescriptor` is at version 2 if: - the reserved fields are zeroed -- the UMP queue contains the core and session index commitments and the commitments value matching the ones in the descriptor. +- the session index matches the session index of the relay parent +- the UMP queue contains a core index commitment and it the one in the descriptor. + + +### UMP transport + +[CandidateCommitments](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L652) remains unchanged as we will store scale encoded `UMPSignal` messages directly in the parachain UMP queue by outputing them in the [upward_messages](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L682). + +The UMP queue layout is changed to allow the relay chain to receive both the XCM messages and `UMPSignal` messages. An empty message (empty `Vec`) is used to mark the end XCM messages and the start of `UMPSignal` messages. + +This way of representing the new messages has been choose over introducing an enum wrapper to minimize breaking changes of XCM message decoding in tools like Subscan for example. + +Example: +``` +[ XCM message1, XCM message2, ..., EMPTY message, UMPSignal::CoreSelector ] +``` + +#### `UMPSignal` messages + +``` +/// A strictly increasing sequence number, tipically this would be the parachain block number. +pub struct CoreSelector(pub BlockNumber); + +/// An offset in the relay chain claim queue. +pub struct ClaimQueueOffset(pub u8); + +/// Default claim queue offset +pub const DEFAULT_CLAIM_QUEUE_OFFSET: ClaimQueueOffset = ClaimQueueOffset(1); + +pub enum UMPSignal { + /// A message sent by a parachain to select the core the candidate is commited to. + /// Relay chain validators, in particular backers, use the `CoreSelector` and `ClaimQueueOffset` + /// to compute the index of the core the candidate has commited to. + /// + SelectCore(CoreSelector, ClaimQueueOffset), +} +``` + +As we dont want to have a claim queue snapshot in the parachain runtime, we need to set `ClaimQueueOffset` +statically to some sane value. Parachains should prefer to have a static value that makes sense for their usecase which can be changed by governance at some future point. Changing the value dynamically can be a friction point. It will work out fine to decrease the value to build more into the present. But if the value is increased to build more into the future, a relay chain block will be skipped. + +Considering `para_assigned_cores` is a sorted vec of core indices assigned to a parachain at the +specified claim queue offset, validators will determine the committed core index like this: + +``` +let assigned_core_index = core_selector % para_assigned_cores.len(); +let committed_core_index = para_assigned_cores[assigned_core_index]; +``` + ### Polkadot Primitive changes @@ -90,7 +137,6 @@ pub struct CandidateDescriptorV2 { In future format versions, parts of the `reserved1` and `reserved2` bytes can be used to include additional information in the descriptor. - #### Versioned `CandidateReceipt` and `CommittedCandidateReceipt` primitives: We want to decouple the actual representation of the `CandidateReceipt` from the higher level code. This should make it easier to implement future format versions of this primitive. To hide the logic of versioning the descriptor fields will be private and accessor methods are provided. @@ -116,61 +162,12 @@ impl VersionedCandidateReceipt { A manual decode `Decode` implementation is required to account for version detection and constructing the appropriate variant. - -#### New primitive for representing the `CoreIndex` commitments.** - -``` -pub enum UMPSignal { - OnCore(CoreIndex), -} -``` - -### Cumulus primitives - -Add a new version of the `ParachainInherentData` structure which includes an additional `core_index` field. -``` -pub struct ParachainInherentData { - pub validation_data: PersistedValidationData, - /// A storage proof of a predefined set of keys from the relay-chain. - /// - /// Specifically this witness contains the data for: - /// - /// - the current slot number at the given relay parent - /// - active host configuration as per the relay parent, - /// - the relay dispatch queue sizes - /// - the list of egress HRMP channels (in the list of recipients form) - /// - the metadata for the egress HRMP channels - pub relay_chain_state: sp_trie::StorageProof, - /// Downward messages in the order they were sent. - pub downward_messages: Vec, - /// HRMP messages grouped by channels. The messages in the inner vec must be in order they - /// were sent. In combination with the rule of no more than one message in a channel per block, - /// this means `sent_at` is **strictly** greater than the previous one (if any). - pub horizontal_messages: BTreeMap>, - /// The core index on which the parachain block must be backed - pub core_index: CoreIndex, -} -``` - -### UMP transport -[CandidateCommitments](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L652) remains unchanged as we will store scale encoded `UMPSignal` messages directly in the parachain UMP queue by outputing them in the [upward_messages](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L682). - - -The UMP queue layout is adjusted to allow the relay chain to receive both the XCM messages and `UMPSignal` messages. We will introduce a message separator that will be implemented as an empty `Vec`. - -The separator marks the end of the XCM messages and the begging of the `UMPSignal` messages. - -Example: -``` -[ XCM message1, XCM message2, ..., EMPTY message, UMPSignal::CoreIndex ] -``` - ### Parachain block validation Backers will make use of the core index information to validate the blocks during backing and reject blocks if: -- the `core_index` in descriptor does not match the one in the `UMPSignal`. +- the `core_index` in descriptor does not match the one determined by the `UMPSignal::SelectCore` message - the `core_index` in the descriptor does not match the core the backing group is assigned to -- the `session_index` is equal to the session of the `relay_parent` in the descriptor +- the `session_index` is not equal to the session of the `relay_parent` in the descriptor If core index (and session index) information is not available (backers got an old candidate receipt), there will be no changes compared to current behaviour. From f1d1a603be5bdc3b21180b43969aa6a129f9a39c Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Mon, 5 Aug 2024 15:35:30 +0300 Subject: [PATCH 08/24] Add future versioning field details Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 89ff5bb03..5a8cccd74 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -189,10 +189,15 @@ The expectation is that performance impact is negligible for sending and process ## Ergonomics -Parachain that use elastic scaling must send the separator empty message followed by the `UMPSignal::OnCore` only after sending all of the UMP XCM messages. +Parachain that use elastic scaling must send the separator empty message followed by the `UMPSignal::SelectCore` only after sending all of the UMP XCM messages. ## Compatibility +### Versioning + +At this point there is a simple way to determine the version of the receipt, by testing for zeroed reserved bytes in the +descriptor. Supporting future changes will require a `u8` version field to be introduced in the reserved space. We consider the current version to be 0 and the version check implicitly done when checking for reserved space to be zeroed. + ### Runtime The first step is to remove collator signature checking logic in the runtime, but keep the node side collator signature checks. @@ -208,7 +213,6 @@ Once enabled, the validators will skip checking the collator signature when proc No new implementation of networking protocol versions for collation and validation are required. - ### Parachains `CoreIndex` commitments are needed only by parachains using elastic scaling. Just upgrading the collator node and runtime should be sufficient and possible without manual changes. From f3e81671095885f1cca2cbb94bc5d6ed28549e71 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Mon, 5 Aug 2024 15:39:10 +0300 Subject: [PATCH 09/24] fix typo and rephrase. Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 5a8cccd74..68910d531 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -233,4 +233,4 @@ N/A The implementation is extensible and future proof to some extent. With minimal or no breaking changes, additional fields can be added in the candidate descriptor until the reserved space is exhausted -Once the reserved space is exhausted, versioning will be implemented. The candidate receipt format will be versioned. This will exteend to pvf execution which requires versioning for the validation function, inputs and outputs (`CandidateCommitments`). \ No newline at end of file +Once the reserved space is exhausted, versioning will be implemented. The candidate receipt format will be versioned. Versioning should also be implemented for the validation function, inputs and outputs (`CandidateCommitments`). \ No newline at end of file From c78ddd53827483e3b32ff18a4e8e88dc461612af Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Tue, 13 Aug 2024 18:08:46 +0300 Subject: [PATCH 10/24] feedback Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 96 ++++++++++++-------- 1 file changed, 56 insertions(+), 40 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 68910d531..0651d4943 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -7,15 +7,15 @@ | **Authors** | Andrei Sandu | ## Summary -The only requirement for collator nodes is to provide valid parachain blocks to the validators of a backing group and by definition the collator set is trustless. However, in the case of elastic scaling, for security reason, collators must be trusted - non-malicious. `CoreIndex` commitments are required to remove this limitation. Additionally we are introducing a `SessionIndex` field in the `CandidateReceipt` to make dispute resolution more secure and robust. + +Elastic scaling is not resiliet against griefing attacks without a way for a PoV (Proof of Validity) to commit to the particular core index it was intedened for. This RFC proposes a way to include core index information in the candidate commitments and the `CandidateDescriptor` data strcuture in a backwards compatible way. Additionally it proposes the addition of a `SessionIndex` field in the `CandidateDescriptor` to make dispute resolution more secure and robust. + ## Motivation At present time misbehaving collator nodes, or anyone who has acquired a valid collation can prevent a parachain from effecitvely using elastic scaling by providing the same collation to all backing groups assigned to the parachain. This happens before the next parachain block is authored and will prevent the chain of candidates to be formed, reducing the throughput of the parachain to a single core. -This RFC solves the problem by enabling a parachain to provide a core index commitment as part of it's PVF execution output and in the associated candidate receipt data structure. - -Once this RFC is implemented the validity of a parachain block depends on the core it is being executed on. +The session index of candidates is important for the disputes protocol as it is used to lookup validator keys and check dispute vote signatures. By adding a `SessionIndex` in the `CandidateDescriptor`, validators no longer have to trust the `Sessionindex` provided by the validator raising a dispute. It can happen that the dispute concerns a relay chain block not yet imported by a validator. In this case validators can safely assume the session index refers to the session the candidate has appeared in, otherwise the chain would have rejected candidate. ## Stakeholders @@ -30,11 +30,12 @@ This approach and alternatives have been considered and discussed in [this issue The approach proposed below was chosen primarly because it minimizes the number of breaking changes, the complexity and takes less implementation and testing time. The proposal is to change the existing primitives while keeping binary compatibility with the older versions. We repurpose unused fields to introduce core index and a session index information in the `CandidateDescriptor` and extend the UMP usage to output core index information. ### Reclaiming unused space in the descriptor + The `CandidateDescriptor` currently includes `collator` and `signature` fields. The collator includes a signature on the following descriptor fields: parachain id, relay parent, validation data hash, validation code hash and the PoV hash. However, in practice, having a collator signature in the receipt on the relay chain does not provide any benefits as there is no mechanism to punish or reward collators that have provided bad parachain blocks. -This proposal aims to remove the collator signature and all the logic that checks the collator signatures of candidate receipts. We use the first 6 reclaimed bytes to represent the core and session index, and fill the rest with zeroes. So, there is no change in the layout and length of the receipt. The new primitive is binary compatible with the old one. +This proposal aims to remove the collator signature and all the logic that checks the collator signatures of candidate receipts. We use the first 7 reclaimed bytes to represent version, the core and session index, and fill the rest with zeroes. So, there is no change in the layout and length of the receipt. The new primitive is binary compatible with the old one. ### Backwards compatibility @@ -44,10 +45,11 @@ There are two flavors of candidate receipts which are used in network protocols, We want to support both the old and new versions in the runtime and node. The implementation must be able to detect the version of a given candidate receipt. -`CandidateDescriptor` is at version 2 if: +`CandidateDescriptor` is a valid version 2 descriptor, if: +- version field is 0 - the reserved fields are zeroed - the session index matches the session index of the relay parent -- the UMP queue contains a core index commitment and it the one in the descriptor. +- the UMP queue contains a core index commitment and it matches the one in the descriptor. ### UMP transport @@ -56,7 +58,7 @@ We want to support both the old and new versions in the runtime and node. The im The UMP queue layout is changed to allow the relay chain to receive both the XCM messages and `UMPSignal` messages. An empty message (empty `Vec`) is used to mark the end XCM messages and the start of `UMPSignal` messages. -This way of representing the new messages has been choose over introducing an enum wrapper to minimize breaking changes of XCM message decoding in tools like Subscan for example. +This way of representing the new messages has been chosen over introducing an enum wrapper to minimize breaking changes of XCM message decoding in tools like Subscan for example. Example: ``` @@ -66,8 +68,8 @@ Example: #### `UMPSignal` messages ``` -/// A strictly increasing sequence number, tipically this would be the parachain block number. -pub struct CoreSelector(pub BlockNumber); +/// An `u8` wrap around sequence number. Typically this would be the least significant byte of the parachain block number. +pub struct CoreSelector(pub u8); /// An offset in the relay chain claim queue. pub struct ClaimQueueOffset(pub u8); @@ -101,23 +103,28 @@ let committed_core_index = para_assigned_cores[assigned_core_index]; #### New [CandidateDescriptor](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L482) - reclaim 32 bytes from `collator: CollatorId` and 64 bytes from `signature: CollatorSignature` and rename to `reserved1` and `reserved2` fields. -- take 2 bytes from `reserved1` for a new `core_index: u16` field. -- take 4 bytes from `reserved2` for a new `session_index: u32` field. -- the `reserved1` and `reserved2` fields are zeroed +- take 1 bytes from `reserved1` for a new `version: u8` field. +- take 2 bytes from `reserved1` for a new `core_index: u16` field. +- take 4 bytes from `reserved1` for a new `session_index: u32` field. +- the remaining `reserved1` and `reserved2` fields are zeroed Thew new primitive will look like this: -``` + +```rust pub struct CandidateDescriptorV2 { /// The ID of the para this is a candidate for. - para_id: Id, + para_id: ParaId, /// The hash of the relay-chain block this is executed in the context of. relay_parent: H, + /// Version field. The raw value here is not exposed, instead it is used + /// to determine the `CandidateDescriptorVersion`, see `fn version()` + version: InternalVersion, /// The core index where the candidate is backed. - core_index: CoreIndex, - /// The session index in which the candidate is backed. + core_index: u16, + /// The session index of the candidate relay parent. session_index: SessionIndex, /// Reserved bytes. - reserved1: [u8; 26], + reserved25b: [u8; 25], /// The blake2-256 hash of the persisted validation data. This is extra data derived from /// relay-chain state which may vary based on bitfields included before the candidate. /// Thus it cannot be derived entirely from the relay-parent. @@ -127,7 +134,7 @@ pub struct CandidateDescriptorV2 { /// The root of a block's erasure encoding Merkle tree. erasure_root: Hash, /// Reserved bytes. - reserved2: [u8; 64], + reserved64b: [u8; 64], /// Hash of the para header that is being generated by this candidate. para_head: Hash, /// The blake2-256 hash of the validation code bytes. @@ -137,23 +144,24 @@ pub struct CandidateDescriptorV2 { In future format versions, parts of the `reserved1` and `reserved2` bytes can be used to include additional information in the descriptor. -#### Versioned `CandidateReceipt` and `CommittedCandidateReceipt` primitives: +#### Candidate descriptor API -We want to decouple the actual representation of the `CandidateReceipt` from the higher level code. This should make it easier to implement future format versions of this primitive. To hide the logic of versioning the descriptor fields will be private and accessor methods are provided. +We want to decouple the actual representation of the `CandidateDescriptor` from the higher level code. This should make it easier to implement future format versions of this primitive. To hide the logic of versioning the descriptor fields will be private and getter methods are provided for all the fields. -``` -pub enum VersionedCandidateReceipt { - V1(CandidateReceipt), - V2(CandidateReceiptV2), -} +```rust +impl CandidateDescriptorV2 { + + /// Returns the collator id in the descriptor. Returns `None` if descriptor is at verision 2. + pub fn collator(&self) -> Option; + + /// Returns the collator signature in the descriptor. Returns `None` if descriptor is at verision 2. + pub fn signature(&self) -> Option; -impl VersionedCandidateReceipt { - /// Returns the core index the candidate has commited to. - /// Returns `None` if the candidate receipt is the old version (v1). - fn core_index() -> Option; + /// Returns the core index of the descriptor. Returns `None` if the descriptor is at version 1. + pub fn core_index(&self) -> Option; - /// Returns the session index of the candidate relay parent. - fn session_index() -> Option; + /// Returns the session index of the descriptor. Returns `None` if the descriptor is at version 1. + pub fn session_index(&self) -> Option; /// ... } @@ -164,6 +172,8 @@ A manual decode `Decode` implementation is required to account for version dete ### Parachain block validation +#### Node + Backers will make use of the core index information to validate the blocks during backing and reject blocks if: - the `core_index` in descriptor does not match the one determined by the `UMPSignal::SelectCore` message - the `core_index` in the descriptor does not match the core the backing group is assigned to @@ -171,6 +181,10 @@ Backers will make use of the core index information to validate the blocks durin If core index (and session index) information is not available (backers got an old candidate receipt), there will be no changes compared to current behaviour. +#### Runtime + +The runtime will also perform the above checks and reject invalid candidates. + ## Drawbacks The only drawback is that further additions to the descriptor are limited to the amount of remaining unused space. @@ -189,16 +203,17 @@ The expectation is that performance impact is negligible for sending and process ## Ergonomics -Parachain that use elastic scaling must send the separator empty message followed by the `UMPSignal::SelectCore` only after sending all of the UMP XCM messages. +It is mandatory for elastic parachains to switch to the new receipt format. It is optional but desired that all parachains +switch to the new receipts for providing the session index for disputes. -## Compatibility +Once this RFC is implemented the parachain runtime and node must not require any manual changes to use it, except if the parachain wants to change the `ClaimQueueOffset` that is used to determine the core index. -### Versioning +## Compatibility -At this point there is a simple way to determine the version of the receipt, by testing for zeroed reserved bytes in the -descriptor. Supporting future changes will require a `u8` version field to be introduced in the reserved space. We consider the current version to be 0 and the version check implicitly done when checking for reserved space to be zeroed. +The proposed changes are backwards compatible in general, but additional care must be taken by waiting for enough validators to upgrade before the validators and runtime start accepting the new candidate receipts. ### Runtime + The first step is to remove collator signature checking logic in the runtime, but keep the node side collator signature checks. @@ -209,13 +224,13 @@ The runtime must be upgraded to the new primitives before any collator or node a To ensure a smooth launch, a new node feature is required. The feature acts as a signal for supporting the new candidate receipts on the node side and can only be safely enabled if at least 2/3 of the validators are upgraded. -Once enabled, the validators will skip checking the collator signature when processing the candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if present in the receipit. +Once enabled, the validators will skip checking the collator signature when processing the candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if present in the receipt. No new implementation of networking protocol versions for collation and validation are required. ### Parachains -`CoreIndex` commitments are needed only by parachains using elastic scaling. Just upgrading the collator node and runtime should be sufficient and possible without manual changes. +The implementation of this RFC will supersede the Elastic MVP feature that relies on injecting a core index in the `validator_indices` fields of the `BackedCandidate` primitive. Elastic parachains must upgrade to use the new receipt format. ### Tooling @@ -233,4 +248,5 @@ N/A The implementation is extensible and future proof to some extent. With minimal or no breaking changes, additional fields can be added in the candidate descriptor until the reserved space is exhausted -Once the reserved space is exhausted, versioning will be implemented. The candidate receipt format will be versioned. Versioning should also be implemented for the validation function, inputs and outputs (`CandidateCommitments`). \ No newline at end of file +At this point there is a simple way to determine the version of the receipt, by testing for zeroed reserved bytes in the +descriptor. Future versions of the receipt can be implemented and identified by using the `version` field of the descirptor introduced in this RFC. \ No newline at end of file From 952a03a0f44d8c2edcf636730e124946a0f07fc5 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Tue, 13 Aug 2024 18:10:00 +0300 Subject: [PATCH 11/24] ```rust Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 0651d4943..8329cd5a4 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -61,13 +61,13 @@ The UMP queue layout is changed to allow the relay chain to receive both the XCM This way of representing the new messages has been chosen over introducing an enum wrapper to minimize breaking changes of XCM message decoding in tools like Subscan for example. Example: -``` +```rust [ XCM message1, XCM message2, ..., EMPTY message, UMPSignal::CoreSelector ] ``` #### `UMPSignal` messages -``` +```rust /// An `u8` wrap around sequence number. Typically this would be the least significant byte of the parachain block number. pub struct CoreSelector(pub u8); @@ -92,7 +92,7 @@ statically to some sane value. Parachains should prefer to have a static value t Considering `para_assigned_cores` is a sorted vec of core indices assigned to a parachain at the specified claim queue offset, validators will determine the committed core index like this: -``` +```rust let assigned_core_index = core_selector % para_assigned_cores.len(); let committed_core_index = para_assigned_cores[assigned_core_index]; ``` From bab26c4ba46d49b2025860d04a996f0e030162df Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Fri, 16 Aug 2024 00:44:46 +0300 Subject: [PATCH 12/24] more feedback Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 125 +++++++++---------- 1 file changed, 57 insertions(+), 68 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 8329cd5a4..d4f7f5834 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -1,4 +1,4 @@ -# RFC-0103: Introduce a `CoreIndex` commitment in candidate receipts +# RFC-0103: Introduce a `CoreIndex` commitment and a `SessionIndex` field in candidate receipts | | | | --------------- | ------------------------------------------------------------------------------------------- | @@ -8,14 +8,15 @@ ## Summary -Elastic scaling is not resiliet against griefing attacks without a way for a PoV (Proof of Validity) to commit to the particular core index it was intedened for. This RFC proposes a way to include core index information in the candidate commitments and the `CandidateDescriptor` data strcuture in a backwards compatible way. Additionally it proposes the addition of a `SessionIndex` field in the `CandidateDescriptor` to make dispute resolution more secure and robust. +Elastic scaling is not resilient against griefing attacks without a way for a PoV (Proof of Validity) to commit to the particular core index it was intended for. This RFC proposes a way to include core index information in the candidate commitments and the `CandidateDescriptor` data structure in a backwards compatible way. Additionally it proposes the addition of a `SessionIndex` field in the `CandidateDescriptor` to make dispute resolution more secure and robust. ## Motivation -At present time misbehaving collator nodes, or anyone who has acquired a valid collation can prevent a parachain from effecitvely using elastic scaling by providing the same collation to all backing groups assigned to the parachain. This happens before the next parachain block is authored and will prevent the chain of candidates to be formed, reducing the throughput of the parachain to a single core. +This RFC proposes a way to solve two different problems: -The session index of candidates is important for the disputes protocol as it is used to lookup validator keys and check dispute vote signatures. By adding a `SessionIndex` in the `CandidateDescriptor`, validators no longer have to trust the `Sessionindex` provided by the validator raising a dispute. It can happen that the dispute concerns a relay chain block not yet imported by a validator. In this case validators can safely assume the session index refers to the session the candidate has appeared in, otherwise the chain would have rejected candidate. +1. For Elastic Scaling, it prevents anyone who has acquired a valid collation to DoS the parachain by providing the same collation to all backing groups assigned to the parachain. This can happen before the next valid parachain block is authored and will prevent the chain of candidates to be formed, reducing the throughput of the parachain to a single core. +2. The dispute protocol relies on validators trusting the session index provided by other valdiators when initiating and participating in disputes. It is used to lookup validator keys and check dispute vote signatures. By adding a `SessionIndex` in the `CandidateDescriptor`, validators no longer have to trust the `Sessionindex` provided by the validator raising a dispute. It can happen that the dispute concerns a relay chain block not yet imported by a validator. In this case validators can safely assume the session index refers to the session the candidate has appeared in, otherwise the chain would have rejected candidate. ## Stakeholders @@ -27,7 +28,7 @@ This approach and alternatives have been considered and discussed in [this issue ## Explanation -The approach proposed below was chosen primarly because it minimizes the number of breaking changes, the complexity and takes less implementation and testing time. The proposal is to change the existing primitives while keeping binary compatibility with the older versions. We repurpose unused fields to introduce core index and a session index information in the `CandidateDescriptor` and extend the UMP usage to output core index information. +The approach proposed below was chosen primarly because it minimizes the number of breaking changes, the complexity and takes less implementation and testing time. The proposal is to change the existing primitives while keeping binary compatibility with the older versions. We repurpose unused fields to introduce core index and a session index information in the `CandidateDescriptor` and extend the UMP for transporting non-XCM messages. ### Reclaiming unused space in the descriptor @@ -37,24 +38,9 @@ However, in practice, having a collator signature in the receipt on the relay ch This proposal aims to remove the collator signature and all the logic that checks the collator signatures of candidate receipts. We use the first 7 reclaimed bytes to represent version, the core and session index, and fill the rest with zeroes. So, there is no change in the layout and length of the receipt. The new primitive is binary compatible with the old one. -### Backwards compatibility - -There are two flavors of candidate receipts which are used in network protocols, runtime and node implementation: -- `CommittedCandidateReceipt` which includes the `CanidateDescriptor` and the `CandidateCommitments` -- `CandidateReceipt` which includes the `CanidateDescriptor` and just a hash of the commitments - -We want to support both the old and new versions in the runtime and node. The implementation must be able to detect the version of a given candidate receipt. - -`CandidateDescriptor` is a valid version 2 descriptor, if: -- version field is 0 -- the reserved fields are zeroed -- the session index matches the session index of the relay parent -- the UMP queue contains a core index commitment and it matches the one in the descriptor. - - ### UMP transport -[CandidateCommitments](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L652) remains unchanged as we will store scale encoded `UMPSignal` messages directly in the parachain UMP queue by outputing them in the [upward_messages](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L682). +[CandidateCommitments](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L682) remains unchanged as we will store scale encoded `UMPSignal` messages directly in the parachain UMP queue by outputing them in [upward_messages](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L684). The UMP queue layout is changed to allow the relay chain to receive both the XCM messages and `UMPSignal` messages. An empty message (empty `Vec`) is used to mark the end XCM messages and the start of `UMPSignal` messages. @@ -62,7 +48,7 @@ This way of representing the new messages has been chosen over introducing an en Example: ```rust -[ XCM message1, XCM message2, ..., EMPTY message, UMPSignal::CoreSelector ] +[ XCM message1, XCM message2, ..., EMPTY message, UMPSignal::SelectCore ] ``` #### `UMPSignal` messages @@ -86,21 +72,41 @@ pub enum UMPSignal { } ``` -As we dont want to have a claim queue snapshot in the parachain runtime, we need to set `ClaimQueueOffset` -statically to some sane value. Parachains should prefer to have a static value that makes sense for their usecase which can be changed by governance at some future point. Changing the value dynamically can be a friction point. It will work out fine to decrease the value to build more into the present. But if the value is increased to build more into the future, a relay chain block will be skipped. +The parachain runtime is not concerned with the actual `CoreIndex` the candidate is intended for, +but must provide enough information for the validators and collators to compute it. `CoreSelector` +and `ClaimQueueOffset` use only 2 bytes and fullfil the requirement. -Considering `para_assigned_cores` is a sorted vec of core indices assigned to a parachain at the -specified claim queue offset, validators will determine the committed core index like this: +**Example:** + +`cq_offset = 1` and `core_selector = 3` + +The table below represents a snapshot of the claim queue: + +| | offset = 0 | offset = 1 | offset = 2 | +| :--: | :--: | :--: | :--: | +| Core 1 | **Para A** | **Para A** | **Para A** | +| Core 2 | **Para A** | Para B | **Para A** | +| Core 3 | Para B | **Para A** | **Para A** | + +The purpose of `ClaimQueueOffset` is to select the column from the above table. +For `cq_offset = 1` we get `[ Para A, Para B, Para A]` and use as input to create +a sorted vec with the cores A is assigned to: `[ Core 1, Core 2]` and call it `para_assigned_cores`. +We use `core_selector` and determine the commited core index is `Core 2` like this: ```rust -let assigned_core_index = core_selector % para_assigned_cores.len(); -let committed_core_index = para_assigned_cores[assigned_core_index]; +let committed_core_index = para_assigned_cores[core_selector % para_assigned_cores.len()]; ``` +Parachains should prefer to have a static `ClaimQueueOffset` value that makes sense for their +usecase which can be changed by governance at some future point. Changing the value dynamically +can be a friction point. It will work out fine to decrease the value to build more into the +present. But if the value is increased to build more into the future, a relay chain block will +be skipped. + ### Polkadot Primitive changes -#### New [CandidateDescriptor](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/primitives/src/v7/mod.rs#L482) +#### New [CandidateDescriptor](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L512) - reclaim 32 bytes from `collator: CollatorId` and 64 bytes from `signature: CollatorSignature` and rename to `reserved1` and `reserved2` fields. - take 1 bytes from `reserved1` for a new `version: u8` field. @@ -117,14 +123,14 @@ pub struct CandidateDescriptorV2 { /// The hash of the relay-chain block this is executed in the context of. relay_parent: H, /// Version field. The raw value here is not exposed, instead it is used - /// to determine the `CandidateDescriptorVersion`, see `fn version()` + /// to determine the `CandidateDescriptorVersion` version: InternalVersion, /// The core index where the candidate is backed. core_index: u16, /// The session index of the candidate relay parent. session_index: SessionIndex, /// Reserved bytes. - reserved25b: [u8; 25], + reserved1: [u8; 25], /// The blake2-256 hash of the persisted validation data. This is extra data derived from /// relay-chain state which may vary based on bitfields included before the candidate. /// Thus it cannot be derived entirely from the relay-parent. @@ -134,7 +140,7 @@ pub struct CandidateDescriptorV2 { /// The root of a block's erasure encoding Merkle tree. erasure_root: Hash, /// Reserved bytes. - reserved64b: [u8; 64], + reserved2: [u8; 64], /// Hash of the para header that is being generated by this candidate. para_head: Hash, /// The blake2-256 hash of the validation code bytes. @@ -144,46 +150,28 @@ pub struct CandidateDescriptorV2 { In future format versions, parts of the `reserved1` and `reserved2` bytes can be used to include additional information in the descriptor. -#### Candidate descriptor API - -We want to decouple the actual representation of the `CandidateDescriptor` from the higher level code. This should make it easier to implement future format versions of this primitive. To hide the logic of versioning the descriptor fields will be private and getter methods are provided for all the fields. - -```rust -impl CandidateDescriptorV2 { - - /// Returns the collator id in the descriptor. Returns `None` if descriptor is at verision 2. - pub fn collator(&self) -> Option; - - /// Returns the collator signature in the descriptor. Returns `None` if descriptor is at verision 2. - pub fn signature(&self) -> Option; - - /// Returns the core index of the descriptor. Returns `None` if the descriptor is at version 1. - pub fn core_index(&self) -> Option; - - /// Returns the session index of the descriptor. Returns `None` if the descriptor is at version 1. - pub fn session_index(&self) -> Option; - - /// ... -} - -``` - -A manual decode `Decode` implementation is required to account for version detection and constructing the appropriate variant. - ### Parachain block validation -#### Node +If the candidate descriptor is version 1, there are no changes. -Backers will make use of the core index information to validate the blocks during backing and reject blocks if: -- the `core_index` in descriptor does not match the one determined by the `UMPSignal::SelectCore` message +For version 2, backers and the runtime must check the validity of `core_index` and `session_index` fields. +A candidate must not be backed if any of the following are true: - the `core_index` in the descriptor does not match the core the backing group is assigned to - the `session_index` is not equal to the session of the `relay_parent` in the descriptor +- the `core_index` in descriptor does not match the one determined by the `UMPSignal::SelectCore` message -If core index (and session index) information is not available (backers got an old candidate receipt), there will be no changes compared to current behaviour. -#### Runtime +### Backwards compatibility + +There are two flavors of candidate receipts which are used in network protocols, runtime and node implementation: +- `CommittedCandidateReceipt` which includes the `CanidateDescriptor` and the `CandidateCommitments` +- `CandidateReceipt` which includes the `CanidateDescriptor` and just a hash of the commitments + +We want to support both the old and new versions in the runtime and node. The implementation must be able to detect the version of a given candidate receipt. -The runtime will also perform the above checks and reject invalid candidates. +The version of the descriptor is detected by checking if the `version` field is `0` and the +reserved fields are zerored. If this is true it means the descriptor is version 2, +otherwise we consider it is version 1. ## Drawbacks @@ -206,13 +194,14 @@ The expectation is that performance impact is negligible for sending and process It is mandatory for elastic parachains to switch to the new receipt format. It is optional but desired that all parachains switch to the new receipts for providing the session index for disputes. -Once this RFC is implemented the parachain runtime and node must not require any manual changes to use it, except if the parachain wants to change the `ClaimQueueOffset` that is used to determine the core index. +The implementation of this RFC itself must not introduce any breaking changes for the parachain runtime or collator nodes. +Collator must compute the core index from the ## Compatibility -The proposed changes are backwards compatible in general, but additional care must be taken by waiting for enough validators to upgrade before the validators and runtime start accepting the new candidate receipts. +The proposed changes are backwards compatible in general, but additional care must be taken by waiting for at least `2/3 + 1` validators to upgrade. Validators that have not upgraded will not back candidates using the new descriptor format and will also initiate disputes against. -### Runtime +### Relay chain runtime The first step is to remove collator signature checking logic in the runtime, but keep the node side collator signature checks. @@ -222,7 +211,7 @@ The runtime must be upgraded to the new primitives before any collator or node a ### Validators To ensure a smooth launch, a new node feature is required. -The feature acts as a signal for supporting the new candidate receipts on the node side and can only be safely enabled if at least 2/3 of the validators are upgraded. +The feature acts as a signal for supporting the new candidate receipts on the node side and can only be safely enabled if at least `2/3 + 1` of the validators are upgraded. Once enabled, the validators will skip checking the collator signature when processing the candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if present in the receipt. From 1b200f3d6862b6422a47c5536bd040fe91ccd601 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Fri, 16 Aug 2024 00:50:11 +0300 Subject: [PATCH 13/24] . Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index d4f7f5834..163324b86 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -195,7 +195,6 @@ It is mandatory for elastic parachains to switch to the new receipt format. It i switch to the new receipts for providing the session index for disputes. The implementation of this RFC itself must not introduce any breaking changes for the parachain runtime or collator nodes. -Collator must compute the core index from the ## Compatibility From 4d92c14eb33c06ac8eb234bb9634cc74a815556b Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Fri, 16 Aug 2024 01:55:59 +0300 Subject: [PATCH 14/24] fix example Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 163324b86..49c63dcfb 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -90,8 +90,8 @@ The table below represents a snapshot of the claim queue: The purpose of `ClaimQueueOffset` is to select the column from the above table. For `cq_offset = 1` we get `[ Para A, Para B, Para A]` and use as input to create -a sorted vec with the cores A is assigned to: `[ Core 1, Core 2]` and call it `para_assigned_cores`. -We use `core_selector` and determine the commited core index is `Core 2` like this: +a sorted vec with the cores A is assigned to: `[ Core 1, Core 3]` and call it `para_assigned_cores`. +We use `core_selector` and determine the commited core index is `Core 3` like this: ```rust let committed_core_index = para_assigned_cores[core_selector % para_assigned_cores.len()]; From 56e0b3922ccef1e4e64b54666d711109ecbb1b86 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Fri, 16 Aug 2024 15:24:00 +0300 Subject: [PATCH 15/24] update performance Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 49c63dcfb..85f96bee7 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -103,7 +103,6 @@ can be a friction point. It will work out fine to decrease the value to build mo present. But if the value is increased to build more into the future, a relay chain block will be skipped. - ### Polkadot Primitive changes #### New [CandidateDescriptor](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L512) @@ -189,6 +188,11 @@ There is no impact on privacy. The expectation is that performance impact is negligible for sending and processing the UMP message has negligible performance impact in the runtime as well as on the node side. +The `ClaimQueueOffset` along with the relay parent choice allows parachains to optimize their block production +for either throughput or latency. A value of `0` with newest relay parent provides best latency while picking +older relay parents avoids re-orgs. + + ## Ergonomics It is mandatory for elastic parachains to switch to the new receipt format. It is optional but desired that all parachains From 13c6ae9c6deea4217c08942c223b07644afcca5f Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Fri, 16 Aug 2024 15:37:32 +0300 Subject: [PATCH 16/24] fix spelling and formatting Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 141 ++++++++++++------- 1 file changed, 93 insertions(+), 48 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 85f96bee7..4631db227 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -8,15 +8,28 @@ ## Summary -Elastic scaling is not resilient against griefing attacks without a way for a PoV (Proof of Validity) to commit to the particular core index it was intended for. This RFC proposes a way to include core index information in the candidate commitments and the `CandidateDescriptor` data structure in a backwards compatible way. Additionally it proposes the addition of a `SessionIndex` field in the `CandidateDescriptor` to make dispute resolution more secure and robust. +Elastic scaling is not resilient against griefing attacks without a way for a PoV (Proof of Validity) +to commit to the particular core index it was intended for. This RFC proposes a way to include +core index information in the candidate commitments and the `CandidateDescriptor` data structure +in a backwards compatible way. Additionally it proposes the addition of a `SessionIndex` field in +the `CandidateDescriptor` to make dispute resolution more secure and robust. ## Motivation This RFC proposes a way to solve two different problems: -1. For Elastic Scaling, it prevents anyone who has acquired a valid collation to DoS the parachain by providing the same collation to all backing groups assigned to the parachain. This can happen before the next valid parachain block is authored and will prevent the chain of candidates to be formed, reducing the throughput of the parachain to a single core. -2. The dispute protocol relies on validators trusting the session index provided by other valdiators when initiating and participating in disputes. It is used to lookup validator keys and check dispute vote signatures. By adding a `SessionIndex` in the `CandidateDescriptor`, validators no longer have to trust the `Sessionindex` provided by the validator raising a dispute. It can happen that the dispute concerns a relay chain block not yet imported by a validator. In this case validators can safely assume the session index refers to the session the candidate has appeared in, otherwise the chain would have rejected candidate. +1. For Elastic Scaling, it prevents anyone who has acquired a valid collation to DoS the parachain + by providing the same collation to all backing groups assigned to the parachain. This can + happen before the next valid parachain block is authored and will prevent the chain of + candidates to be formed, reducing the throughput of the parachain to a single core. +2. The dispute protocol relies on validators trusting the session index provided by other + validators when initiating and participating in disputes. It is used to lookup validator keys + and check dispute vote signatures. By adding a `SessionIndex` in the `CandidateDescriptor`, + validators no longer have to trust the `Sessionindex` provided by the validator raising a + dispute. It can happen that the dispute concerns a relay chain block not yet imported by a + validator. In this case validators can safely assume the session index refers to the session + the candidate has appeared in, otherwise the chain would have rejected candidate. ## Stakeholders @@ -28,23 +41,39 @@ This approach and alternatives have been considered and discussed in [this issue ## Explanation -The approach proposed below was chosen primarly because it minimizes the number of breaking changes, the complexity and takes less implementation and testing time. The proposal is to change the existing primitives while keeping binary compatibility with the older versions. We repurpose unused fields to introduce core index and a session index information in the `CandidateDescriptor` and extend the UMP for transporting non-XCM messages. +The approach proposed below was chosen primarily because it minimizes the number of breaking +changes, the complexity and takes less implementation and testing time. The proposal is to change +the existing primitives while keeping binary compatibility with the older versions. We repurpose +unused fields to introduce core index and a session index information in the `CandidateDescriptor` +and extend the UMP for transporting non-XCM messages. ### Reclaiming unused space in the descriptor -The `CandidateDescriptor` currently includes `collator` and `signature` fields. The collator includes a signature on the following descriptor fields: parachain id, relay parent, validation data hash, validation code hash and the PoV hash. +The `CandidateDescriptor` currently includes `collator` and `signature` fields. The collator +includes a signature on the following descriptor fields: parachain id, relay parent, validation +data hash, validation code hash and the PoV hash. -However, in practice, having a collator signature in the receipt on the relay chain does not provide any benefits as there is no mechanism to punish or reward collators that have provided bad parachain blocks. +However, in practice, having a collator signature in the receipt on the relay chain does not +provide any benefits as there is no mechanism to punish or reward collators that have provided +bad parachain blocks. -This proposal aims to remove the collator signature and all the logic that checks the collator signatures of candidate receipts. We use the first 7 reclaimed bytes to represent version, the core and session index, and fill the rest with zeroes. So, there is no change in the layout and length of the receipt. The new primitive is binary compatible with the old one. +This proposal aims to remove the collator signature and all the logic that checks the collator +signatures of candidate receipts. We use the first 7 reclaimed bytes to represent version, +the core and session index, and fill the rest with zeroes. So, there is no change in the layout +and length of the receipt. The new primitive is binary compatible with the old one. ### UMP transport -[CandidateCommitments](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L682) remains unchanged as we will store scale encoded `UMPSignal` messages directly in the parachain UMP queue by outputing them in [upward_messages](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L684). +[CandidateCommitments](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L682) +remains unchanged as we will store scale encoded `UMPSignal` messages directly in the parachain +UMP queue by outputing them in [upward_messages](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L684). -The UMP queue layout is changed to allow the relay chain to receive both the XCM messages and `UMPSignal` messages. An empty message (empty `Vec`) is used to mark the end XCM messages and the start of `UMPSignal` messages. +The UMP queue layout is changed to allow the relay chain to receive both the XCM messages and +`UMPSignal` messages. An empty message (empty `Vec`) is used to mark the end XCM messages and +the start of `UMPSignal` messages. -This way of representing the new messages has been chosen over introducing an enum wrapper to minimize breaking changes of XCM message decoding in tools like Subscan for example. +This way of representing the new messages has been chosen over introducing an enum wrapper to +minimize breaking changes of XCM message decoding in tools like Subscan for example. Example: ```rust @@ -54,19 +83,18 @@ Example: #### `UMPSignal` messages ```rust -/// An `u8` wrap around sequence number. Typically this would be the least significant byte of the parachain block number. +/// An `u8` wrap around sequence number. Typically this would be the least significant byte of the +/// parachain block number. pub struct CoreSelector(pub u8); /// An offset in the relay chain claim queue. pub struct ClaimQueueOffset(pub u8); -/// Default claim queue offset -pub const DEFAULT_CLAIM_QUEUE_OFFSET: ClaimQueueOffset = ClaimQueueOffset(1); - +/// Signals sent by a parachain to the relay chain. pub enum UMPSignal { - /// A message sent by a parachain to select the core the candidate is commited to. + /// A message sent by a parachain to select the core the candidate is committed to. /// Relay chain validators, in particular backers, use the `CoreSelector` and `ClaimQueueOffset` - /// to compute the index of the core the candidate has commited to. + /// to compute the index of the core the candidate has committed to. /// SelectCore(CoreSelector, ClaimQueueOffset), } @@ -74,7 +102,7 @@ pub enum UMPSignal { The parachain runtime is not concerned with the actual `CoreIndex` the candidate is intended for, but must provide enough information for the validators and collators to compute it. `CoreSelector` -and `ClaimQueueOffset` use only 2 bytes and fullfil the requirement. +and `ClaimQueueOffset` use only 2 bytes and fulfill the requirement. **Example:** @@ -91,7 +119,7 @@ The table below represents a snapshot of the claim queue: The purpose of `ClaimQueueOffset` is to select the column from the above table. For `cq_offset = 1` we get `[ Para A, Para B, Para A]` and use as input to create a sorted vec with the cores A is assigned to: `[ Core 1, Core 3]` and call it `para_assigned_cores`. -We use `core_selector` and determine the commited core index is `Core 3` like this: +We use `core_selector` and determine the committed core index is `Core 3` like this: ```rust let committed_core_index = para_assigned_cores[core_selector % para_assigned_cores.len()]; @@ -107,7 +135,8 @@ be skipped. #### New [CandidateDescriptor](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L512) -- reclaim 32 bytes from `collator: CollatorId` and 64 bytes from `signature: CollatorSignature` and rename to `reserved1` and `reserved2` fields. +- reclaim 32 bytes from `collator: CollatorId` and 64 bytes from `signature: CollatorSignature` + and rename to `reserved1` and `reserved2` fields. - take 1 bytes from `reserved1` for a new `version: u8` field. - take 2 bytes from `reserved1` for a new `core_index: u16` field. - take 4 bytes from `reserved1` for a new `session_index: u32` field. @@ -147,7 +176,8 @@ pub struct CandidateDescriptorV2 { } ``` -In future format versions, parts of the `reserved1` and `reserved2` bytes can be used to include additional information in the descriptor. +In future format versions, parts of the `reserved1` and `reserved2` bytes can be used to include +additional information in the descriptor. ### Parachain block validation @@ -157,28 +187,32 @@ For version 2, backers and the runtime must check the validity of `core_index` a A candidate must not be backed if any of the following are true: - the `core_index` in the descriptor does not match the core the backing group is assigned to - the `session_index` is not equal to the session of the `relay_parent` in the descriptor -- the `core_index` in descriptor does not match the one determined by the `UMPSignal::SelectCore` message +- the `core_index` in the descriptor does not match the one determined by the `UMPSignal::SelectCore` message ### Backwards compatibility -There are two flavors of candidate receipts which are used in network protocols, runtime and node implementation: +There are two flavors of candidate receipts that are used in network protocols, runtime and node +implementation: - `CommittedCandidateReceipt` which includes the `CanidateDescriptor` and the `CandidateCommitments` - `CandidateReceipt` which includes the `CanidateDescriptor` and just a hash of the commitments -We want to support both the old and new versions in the runtime and node. The implementation must be able to detect the version of a given candidate receipt. +We want to support both the old and new versions in the runtime and node. The implementation must +be able to detect the version of a given candidate receipt. The version of the descriptor is detected by checking if the `version` field is `0` and the -reserved fields are zerored. If this is true it means the descriptor is version 2, +reserved fields are zeroed. If this is true it means the descriptor is version 2, otherwise we consider it is version 1. ## Drawbacks -The only drawback is that further additions to the descriptor are limited to the amount of remaining unused space. +The only drawback is that further additions to the descriptor are limited to the amount of +remaining unused space. ## Testing, Security, and Privacy -Standard testing (unit tests, CI zombienet tests) for functionality and mandatory secuirty audit to ensure the implementation does not introduce any new security issues. +Standard testing (unit tests, CI zombienet tests) for functionality and mandatory security audit +to ensure the implementation does not introduce any new security issues. Backwards compatibility of the implementation will be tested on testnets (Versi and Westend). @@ -186,51 +220,59 @@ There is no impact on privacy. ## Performance -The expectation is that performance impact is negligible for sending and processing the UMP message has negligible performance impact in the runtime as well as on the node side. +The expected performance impact is negligible for sending and processing the new UMP +message in the runtime as well as on the node side. -The `ClaimQueueOffset` along with the relay parent choice allows parachains to optimize their block production -for either throughput or latency. A value of `0` with newest relay parent provides best latency while picking -older relay parents avoids re-orgs. +The `ClaimQueueOffset` along with the relay parent choice allows parachains to optimize their +block production for either throughput or latency. A value of `0` with newest relay parent +provides best latency while picking older relay parents avoids re-orgs. ## Ergonomics -It is mandatory for elastic parachains to switch to the new receipt format. It is optional but desired that all parachains -switch to the new receipts for providing the session index for disputes. +It is mandatory for elastic parachains to switch to the new receipt format. It is optional but +desired that all parachains switch to the new receipts for providing the session index for +disputes. -The implementation of this RFC itself must not introduce any breaking changes for the parachain runtime or collator nodes. +The implementation of this RFC itself must not introduce any breaking changes for the parachain +runtime or collator nodes. ## Compatibility -The proposed changes are backwards compatible in general, but additional care must be taken by waiting for at least `2/3 + 1` validators to upgrade. Validators that have not upgraded will not back candidates using the new descriptor format and will also initiate disputes against. +The proposed changes are backwards compatible in general, but additional care must be taken by +waiting for at least `2/3 + 1` validators to upgrade. Validators that have not upgraded will not +back candidates using the new descriptor format and will also initiate disputes against. ### Relay chain runtime -The first step is to remove collator signature checking logic in the runtime, but keep the node side collator signature +The first step is to remove collator signature checking logic in the runtime, but keep the node +side collator signature checks. -The runtime must be upgraded to the new primitives before any collator or node are allowed to use the new candidate receipts format. +The runtime must be upgraded to the new primitives before any collator or node is allowed to use +the new candidate receipts format. ### Validators To ensure a smooth launch, a new node feature is required. -The feature acts as a signal for supporting the new candidate receipts on the node side and can only be safely enabled if at least `2/3 + 1` of the validators are upgraded. - -Once enabled, the validators will skip checking the collator signature when processing the candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if present in the receipt. - -No new implementation of networking protocol versions for collation and validation are required. +The feature acts as a signal for supporting the new candidate receipts on the node side and can +only be safely enabled if at least `2/3 + 1` of the validators are upgraded. -### Parachains +Once enabled, the validators will skip checking the collator signature when processing the +candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if present in the receipt. -The implementation of this RFC will supersede the Elastic MVP feature that relies on injecting a core index in the `validator_indices` fields of the `BackedCandidate` primitive. Elastic parachains must upgrade to use the new receipt format. +No new implementation of networking protocol versions for collation and validation is required. ### Tooling -Any tooling that decodes UMP XCM messages needs an update to support or ignore the new UMP messages, but they should be fine to decode the regular XCM messages that come before the separator. +Any tooling that decodes UMP XCM messages needs an update to support or ignore the new UMP +messages, but they should be fine to decode the regular XCM messages that come before the +separator. ## Prior Art and References -Forum discussion about a new `CandidateReceipt` format: https://forum.polkadot.network/t/pre-rfc-discussion-candidate-receipt-format-v2/3738 +Forum discussion about a new `CandidateReceipt` format: + https://forum.polkadot.network/t/pre-rfc-discussion-candidate-receipt-format-v2/3738 ## Unresolved Questions @@ -238,7 +280,10 @@ N/A ## Future Directions and Related Material -The implementation is extensible and future proof to some extent. With minimal or no breaking changes, additional fields can be added in the candidate descriptor until the reserved space is exhausted +The implementation is extensible and future proof to some extent. With minimal or no breaking +changes, additional fields can be added in the candidate descriptor until the reserved space is +exhausted -At this point there is a simple way to determine the version of the receipt, by testing for zeroed reserved bytes in the -descriptor. Future versions of the receipt can be implemented and identified by using the `version` field of the descirptor introduced in this RFC. \ No newline at end of file +At this point there is a simple way to determine the version of the receipt, by testing for zeroed +reserved bytes in the descriptor. Future versions of the receipt can be implemented and identified +by using the `version` field of the descirptor introduced in this RFC. \ No newline at end of file From 029cc11e8e4a8292687a752ba69e8e32dab780cd Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Tue, 20 Aug 2024 13:37:24 +0300 Subject: [PATCH 17/24] stricter runtime check Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 30 ++++++++++++++++++-- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 4631db227..fefe523d7 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -183,11 +183,34 @@ additional information in the descriptor. If the candidate descriptor is version 1, there are no changes. -For version 2, backers and the runtime must check the validity of `core_index` and `session_index` fields. +Backers must check the validity of `core_index` and `session_index` fields. A candidate must not be backed if any of the following are true: -- the `core_index` in the descriptor does not match the core the backing group is assigned to +- the `core_index` in the descriptor does not match the core the backer is assigned to - the `session_index` is not equal to the session of the `relay_parent` in the descriptor -- the `core_index` in the descriptor does not match the one determined by the `UMPSignal::SelectCore` message +- the `core_index` in the descriptor does not match the one determined by the + `UMPSignal::SelectCore` message + + +### On-chain backing + +If the candidate descriptor is version 1, there are no changes. + +For version 2 descriptors the runtime will determine the `core_index` similarly to backers but +will always ignore the committed claim queue offset and use `0`, as it only cares +about what candidates can be backed at current block. + +As the chain advances the claims also advance to the top of the queue. The runtime will only back +a candidate if the claimed core selected by it's claim queue offset has reached the top of the queue +at the current relay chain block: `current_block_num - relay_parent_num - 1 == claim_queue_offset`. + +The impact of this change is that candidates built into the future (`claim queue offset > 0`) +can no longer be backed earlier even if the core is free and the core is assigned to the +parachain. + +Introducing this additional check is required to ensure the runtime computes the core index +determinstically. For example, some collator has missed his slot and the core is now used +to back a candidate with a higher claim queue offset. The number of assigned cores can +be different at these two queue offsets and the committed core indices would be different. ### Backwards compatibility @@ -204,6 +227,7 @@ The version of the descriptor is detected by checking if the `version` field is reserved fields are zeroed. If this is true it means the descriptor is version 2, otherwise we consider it is version 1. + ## Drawbacks The only drawback is that further additions to the descriptor are limited to the amount of From a19d31849db442cbbde48ee447b5c27a7db1d5c7 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Thu, 29 Aug 2024 14:16:49 +0300 Subject: [PATCH 18/24] update Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index fefe523d7..e352e20af 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -155,7 +155,7 @@ pub struct CandidateDescriptorV2 { version: InternalVersion, /// The core index where the candidate is backed. core_index: u16, - /// The session index of the candidate relay parent. + /// The session in which the candidate is backed. session_index: SessionIndex, /// Reserved bytes. reserved1: [u8; 25], @@ -278,9 +278,9 @@ the new candidate receipts format. ### Validators -To ensure a smooth launch, a new node feature is required. +To ensure a smooth launch, a new node feature is required. The feature acts as a signal for supporting the new candidate receipts on the node side and can -only be safely enabled if at least `2/3 + 1` of the validators are upgraded. +only be safely enabled if at least `2/3 + 1` of the validators are upgraded. Node implementions need to decode the new candidate descriptor once the feature is enabled otherwise they might raise disputes and get slashed. Once enabled, the validators will skip checking the collator signature when processing the candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if present in the receipt. From 7b6dfc668be5e9d06a7c2f1667b283a0d00b1593 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Fri, 30 Aug 2024 13:22:52 +0300 Subject: [PATCH 19/24] updates Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 63 ++++++++------------ 1 file changed, 26 insertions(+), 37 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index e352e20af..85bf5be47 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -126,10 +126,7 @@ let committed_core_index = para_assigned_cores[core_selector % para_assigned_cor ``` Parachains should prefer to have a static `ClaimQueueOffset` value that makes sense for their -usecase which can be changed by governance at some future point. Changing the value dynamically -can be a friction point. It will work out fine to decrease the value to build more into the -present. But if the value is increased to build more into the future, a relay chain block will -be skipped. +usecase which can be changed by governance at some future point. ### Polkadot Primitive changes @@ -186,7 +183,7 @@ If the candidate descriptor is version 1, there are no changes. Backers must check the validity of `core_index` and `session_index` fields. A candidate must not be backed if any of the following are true: - the `core_index` in the descriptor does not match the core the backer is assigned to -- the `session_index` is not equal to the session of the `relay_parent` in the descriptor +- the `session_index` is not equal to the session index the candidate is backed in - the `core_index` in the descriptor does not match the one determined by the `UMPSignal::SelectCore` message @@ -195,37 +192,26 @@ A candidate must not be backed if any of the following are true: If the candidate descriptor is version 1, there are no changes. -For version 2 descriptors the runtime will determine the `core_index` similarly to backers but -will always ignore the committed claim queue offset and use `0`, as it only cares -about what candidates can be backed at current block. - -As the chain advances the claims also advance to the top of the queue. The runtime will only back -a candidate if the claimed core selected by it's claim queue offset has reached the top of the queue -at the current relay chain block: `current_block_num - relay_parent_num - 1 == claim_queue_offset`. - -The impact of this change is that candidates built into the future (`claim queue offset > 0`) -can no longer be backed earlier even if the core is free and the core is assigned to the -parachain. - -Introducing this additional check is required to ensure the runtime computes the core index -determinstically. For example, some collator has missed his slot and the core is now used -to back a candidate with a higher claim queue offset. The number of assigned cores can -be different at these two queue offsets and the committed core indices would be different. +For version 2 descriptors the runtime will determine the `core_index` using the same inputs +as backers did off-chain. It currently stores the claim queue at the newest allowed +relay parent corresponding to claim queue offset `0`. The runtime needs to be changed to store +a claim queue snapshot at all allowed relay parents. ### Backwards compatibility There are two flavors of candidate receipts that are used in network protocols, runtime and node implementation: -- `CommittedCandidateReceipt` which includes the `CanidateDescriptor` and the `CandidateCommitments` -- `CandidateReceipt` which includes the `CanidateDescriptor` and just a hash of the commitments +- `CommittedCandidateReceipt` which includes the `CandidateDescriptor` and the `CandidateCommitments` +- `CandidateReceipt` which includes the `CandidateDescriptor` and just a hash of the commitments -We want to support both the old and new versions in the runtime and node. The implementation must +We want to support both the old and new versions in the runtime and node, so the implementation must be able to detect the version of a given candidate receipt. -The version of the descriptor is detected by checking if the `version` field is `0` and the -reserved fields are zeroed. If this is true it means the descriptor is version 2, -otherwise we consider it is version 1. +The version of the descriptor is detected by checking the reserved fields. +If they are not zerored, it means it is a version 1 descriptor. Otherwise the `version` field +is used further to determine the version. It should be `0` for version 2 descriptors. If it is not +the descriptor has an unknown version and should be considered invalid. ## Drawbacks @@ -244,8 +230,8 @@ There is no impact on privacy. ## Performance -The expected performance impact is negligible for sending and processing the new UMP -message in the runtime as well as on the node side. +Overall performance will be improved by not checking the collator signatures in runtime and nodes. +The impact on the UMP queue and candidate receipt processing is negligible. The `ClaimQueueOffset` along with the relay parent choice allows parachains to optimize their block production for either throughput or latency. A value of `0` with newest relay parent @@ -265,25 +251,28 @@ runtime or collator nodes. The proposed changes are backwards compatible in general, but additional care must be taken by waiting for at least `2/3 + 1` validators to upgrade. Validators that have not upgraded will not -back candidates using the new descriptor format and will also initiate disputes against. +back candidates using the new descriptor format and will also initiate disputes against these +candidates. ### Relay chain runtime The first step is to remove collator signature checking logic in the runtime, but keep the node -side collator signature -checks. +side collator signature checks. -The runtime must be upgraded to the new primitives before any collator or node is allowed to use -the new candidate receipts format. +The runtime must be upgraded to support the new primitives before any collator or node is allowed +to use the new candidate receipts format. ### Validators To ensure a smooth launch, a new node feature is required. The feature acts as a signal for supporting the new candidate receipts on the node side and can -only be safely enabled if at least `2/3 + 1` of the validators are upgraded. Node implementions need to decode the new candidate descriptor once the feature is enabled otherwise they might raise disputes and get slashed. +only be safely enabled if at least `2/3 + 1` of the validators are upgraded. Node implementions +need to decode the new candidate descriptor once the feature is enabled otherwise they might +raise disputes and get slashed. -Once enabled, the validators will skip checking the collator signature when processing the -candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if present in the receipt. +Once the feature is enabled, the validators will skip checking the collator signature when +processing the candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if +present in the receipt. No new implementation of networking protocol versions for collation and validation is required. From ea485d4c424c708d2442f68fbd0ea230320554ff Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Fri, 30 Aug 2024 14:10:10 +0300 Subject: [PATCH 20/24] spell check and lint Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 258 +++++++++---------- 1 file changed, 128 insertions(+), 130 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 85bf5be47..a59292f31 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -2,34 +2,33 @@ | | | | --------------- | ------------------------------------------------------------------------------------------- | -| **Start Date** | 15 July 2024 | +| **Start Date** | 15 July 2024 | | **Description** | Constrain parachain block validity to a specific core and session | -| **Authors** | Andrei Sandu | +| **Authors** | Andrei Sandu | ## Summary Elastic scaling is not resilient against griefing attacks without a way for a PoV (Proof of Validity) -to commit to the particular core index it was intended for. This RFC proposes a way to include -core index information in the candidate commitments and the `CandidateDescriptor` data structure -in a backwards compatible way. Additionally it proposes the addition of a `SessionIndex` field in -the `CandidateDescriptor` to make dispute resolution more secure and robust. - +to commit to the particular core index it was intended for. This RFC proposes a way to include +core index information in the candidate commitments and the `CandidateDescriptor` data structure +in a backward compatible way. Additionally, it proposes the addition of a `SessionIndex` field in +the `CandidateDescriptor` to make dispute resolution more secure and robust. ## Motivation This RFC proposes a way to solve two different problems: -1. For Elastic Scaling, it prevents anyone who has acquired a valid collation to DoS the parachain - by providing the same collation to all backing groups assigned to the parachain. This can - happen before the next valid parachain block is authored and will prevent the chain of - candidates to be formed, reducing the throughput of the parachain to a single core. -2. The dispute protocol relies on validators trusting the session index provided by other - validators when initiating and participating in disputes. It is used to lookup validator keys - and check dispute vote signatures. By adding a `SessionIndex` in the `CandidateDescriptor`, - validators no longer have to trust the `Sessionindex` provided by the validator raising a - dispute. It can happen that the dispute concerns a relay chain block not yet imported by a - validator. In this case validators can safely assume the session index refers to the session - the candidate has appeared in, otherwise the chain would have rejected candidate. +1. For Elastic Scaling, it prevents anyone who has acquired a valid collation to DoS the parachain + by providing the same collation to all backing groups assigned to the parachain. This can + happen before the next valid parachain block is authored and will prevent the chain of + candidates from being formed, reducing the throughput of the parachain to a single core. +2. The dispute protocol relies on validators trusting the session index provided by other + validators when initiating and participating in disputes. It is used to look up validator keys + and check dispute vote signatures. By adding a `SessionIndex` in the `CandidateDescriptor`, + validators no longer have to trust the `Sessionindex` provided by the validator raising a + dispute. The dispute may concern a relay chain block not yet imported by a + validator. In this case, validators can safely assume the session index refers to the session + the candidate has appeared in, otherwise, the chain would have rejected the candidate. ## Stakeholders @@ -41,41 +40,42 @@ This approach and alternatives have been considered and discussed in [this issue ## Explanation -The approach proposed below was chosen primarily because it minimizes the number of breaking -changes, the complexity and takes less implementation and testing time. The proposal is to change -the existing primitives while keeping binary compatibility with the older versions. We repurpose -unused fields to introduce core index and a session index information in the `CandidateDescriptor` -and extend the UMP for transporting non-XCM messages. +The approach proposed below was chosen primarily because it minimizes the number of breaking +changes, the complexity and takes less implementation and testing time. The proposal is to change +the existing primitives while keeping binary compatibility with the older versions. We repurpose +unused fields to introduce core index and a session index information in the `CandidateDescriptor` +and extend the UMP to transport non-XCM messages. ### Reclaiming unused space in the descriptor -The `CandidateDescriptor` currently includes `collator` and `signature` fields. The collator -includes a signature on the following descriptor fields: parachain id, relay parent, validation -data hash, validation code hash and the PoV hash. +The `CandidateDescriptor` includes `collator` and `signature` fields. The collator +includes a signature on the following descriptor fields: parachain id, relay parent, validation +data hash, validation code hash, and the PoV hash. -However, in practice, having a collator signature in the receipt on the relay chain does not -provide any benefits as there is no mechanism to punish or reward collators that have provided +However, in practice, having a collator signature in the receipt on the relay chain does not +provide any benefits as there is no mechanism to punish or reward collators that have provided bad parachain blocks. -This proposal aims to remove the collator signature and all the logic that checks the collator -signatures of candidate receipts. We use the first 7 reclaimed bytes to represent version, -the core and session index, and fill the rest with zeroes. So, there is no change in the layout -and length of the receipt. The new primitive is binary compatible with the old one. +This proposal aims to remove the collator signature and all the logic that checks the collator +signatures of candidate receipts. We use the first 7 reclaimed bytes to represent the version, +the core, session index, and fill the rest with zeroes. So, there is no change in the layout +and length of the receipt. The new primitive is binary-compatible with the old one. ### UMP transport -[CandidateCommitments](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L682) -remains unchanged as we will store scale encoded `UMPSignal` messages directly in the parachain -UMP queue by outputing them in [upward_messages](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L684). +[CandidateCommitments](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L682) +remains unchanged as we will store scale encoded `UMPSignal` messages directly in the parachain +UMP queue by outputting them in [upward_messages](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L684). -The UMP queue layout is changed to allow the relay chain to receive both the XCM messages and -`UMPSignal` messages. An empty message (empty `Vec`) is used to mark the end XCM messages and +The UMP queue layout is changed to allow the relay chain to receive both the XCM messages and +`UMPSignal` messages. An empty message (empty `Vec`) is used to mark the end of XCM messages and the start of `UMPSignal` messages. -This way of representing the new messages has been chosen over introducing an enum wrapper to -minimize breaking changes of XCM message decoding in tools like Subscan for example. +This way of representing the new messages has been chosen over introducing an enum wrapper to +minimize breaking changes of XCM message decoding in tools like Subscan for example. + +Example: -Example: ```rust [ XCM message1, XCM message2, ..., EMPTY message, UMPSignal::SelectCore ] ``` @@ -83,7 +83,7 @@ Example: #### `UMPSignal` messages ```rust -/// An `u8` wrap around sequence number. Typically this would be the least significant byte of the +/// An `u8` wrap-around sequence number. Typically this would be the least significant byte of the /// parachain block number. pub struct CoreSelector(pub u8); @@ -92,17 +92,17 @@ pub struct ClaimQueueOffset(pub u8); /// Signals sent by a parachain to the relay chain. pub enum UMPSignal { - /// A message sent by a parachain to select the core the candidate is committed to. - /// Relay chain validators, in particular backers, use the `CoreSelector` and `ClaimQueueOffset` - /// to compute the index of the core the candidate has committed to. - /// - SelectCore(CoreSelector, ClaimQueueOffset), + /// A message sent by a parachain to select the core the candidate is committed to. + /// Relay chain validators, in particular backers, use the `CoreSelector` and `ClaimQueueOffset` + /// to compute the index of the core the candidate has committed to. + /// + SelectCore(CoreSelector, ClaimQueueOffset), } ``` -The parachain runtime is not concerned with the actual `CoreIndex` the candidate is intended for, -but must provide enough information for the validators and collators to compute it. `CoreSelector` -and `ClaimQueueOffset` use only 2 bytes and fulfill the requirement. +The parachain runtime is not concerned with the actual `CoreIndex` the candidate is intended for, +but must provide enough information for the validators and collators to compute it. `CoreSelector` +and `ClaimQueueOffset` use only 2 bytes and fulfills the requirement. **Example:** @@ -111,13 +111,13 @@ and `ClaimQueueOffset` use only 2 bytes and fulfill the requirement. The table below represents a snapshot of the claim queue: | | offset = 0 | offset = 1 | offset = 2 | -| :--: | :--: | :--: | :--: | -| Core 1 | **Para A** | **Para A** | **Para A** | -| Core 2 | **Para A** | Para B | **Para A** | -| Core 3 | Para B | **Para A** | **Para A** | +| :--: | :--: | :--: | :--: | +| Core 1 | **Para A** | **Para A** | **Para A** | +| Core 2 | **Para A** | Para B | **Para A** | +| Core 3 | Para B | **Para A** | **Para A** | -The purpose of `ClaimQueueOffset` is to select the column from the above table. -For `cq_offset = 1` we get `[ Para A, Para B, Para A]` and use as input to create +The purpose of `ClaimQueueOffset` is to select the column from the above table. +For `cq_offset = 1` we get `[ Para A, Para B, Para A]` and use as input to create a sorted vec with the cores A is assigned to: `[ Core 1, Core 3]` and call it `para_assigned_cores`. We use `core_selector` and determine the committed core index is `Core 3` like this: @@ -125,106 +125,105 @@ We use `core_selector` and determine the committed core index is `Core 3` like t let committed_core_index = para_assigned_cores[core_selector % para_assigned_cores.len()]; ``` -Parachains should prefer to have a static `ClaimQueueOffset` value that makes sense for their +Parachains should prefer to have a static `ClaimQueueOffset` value that makes sense for their usecase which can be changed by governance at some future point. ### Polkadot Primitive changes #### New [CandidateDescriptor](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L512) -- reclaim 32 bytes from `collator: CollatorId` and 64 bytes from `signature: CollatorSignature` - and rename to `reserved1` and `reserved2` fields. +- reclaim 32 bytes from `collator: CollatorId` and 64 bytes from `signature: CollatorSignature` + and rename to `reserved1` and `reserved2` fields. - take 1 bytes from `reserved1` for a new `version: u8` field. - take 2 bytes from `reserved1` for a new `core_index: u16` field. - take 4 bytes from `reserved1` for a new `session_index: u32` field. - the remaining `reserved1` and `reserved2` fields are zeroed -Thew new primitive will look like this: +The new primitive will look like this: ```rust pub struct CandidateDescriptorV2 { - /// The ID of the para this is a candidate for. - para_id: ParaId, - /// The hash of the relay-chain block this is executed in the context of. - relay_parent: H, - /// Version field. The raw value here is not exposed, instead it is used - /// to determine the `CandidateDescriptorVersion` - version: InternalVersion, - /// The core index where the candidate is backed. - core_index: u16, - /// The session in which the candidate is backed. - session_index: SessionIndex, - /// Reserved bytes. - reserved1: [u8; 25], - /// The blake2-256 hash of the persisted validation data. This is extra data derived from - /// relay-chain state which may vary based on bitfields included before the candidate. - /// Thus it cannot be derived entirely from the relay-parent. - persisted_validation_data_hash: Hash, - /// The blake2-256 hash of the PoV. - pov_hash: Hash, - /// The root of a block's erasure encoding Merkle tree. - erasure_root: Hash, - /// Reserved bytes. - reserved2: [u8; 64], - /// Hash of the para header that is being generated by this candidate. - para_head: Hash, - /// The blake2-256 hash of the validation code bytes. - validation_code_hash: ValidationCodeHash, + /// The ID of the para this is a candidate for. + para_id: ParaId, + /// The hash of the relay-chain block this is executed in the context of. + relay_parent: H, + /// Version field. The raw value here is not exposed, instead, it is used + /// to determine the `CandidateDescriptorVersion` + version: InternalVersion, + /// The core index where the candidate is backed. + core_index: u16, + /// The session in which the candidate is backed. + session_index: SessionIndex, + /// Reserved bytes. + reserved1: [u8; 25], + /// The blake2-256 hash of the persisted validation data. This is extra data derived from + /// relay-chain state which may vary based on bitfields included before the candidate. + /// Thus it cannot be derived entirely from the relay parent. + persisted_validation_data_hash: Hash, + /// The blake2-256 hash of the PoV. + pov_hash: Hash, + /// The root of a block's erasure encoding Merkle tree. + erasure_root: Hash, + /// Reserved bytes. + reserved2: [u8; 64], + /// Hash of the para header that is being generated by this candidate. + para_head: Hash, + /// The blake2-256 hash of the validation code bytes. + validation_code_hash: ValidationCodeHash, } ``` -In future format versions, parts of the `reserved1` and `reserved2` bytes can be used to include +In future format versions, parts of the `reserved1` and `reserved2` bytes can be used to include additional information in the descriptor. ### Parachain block validation If the candidate descriptor is version 1, there are no changes. -Backers must check the validity of `core_index` and `session_index` fields. +Backers must check the validity of `core_index` and `session_index` fields. A candidate must not be backed if any of the following are true: + - the `core_index` in the descriptor does not match the core the backer is assigned to - the `session_index` is not equal to the session index the candidate is backed in -- the `core_index` in the descriptor does not match the one determined by the +- the `core_index` in the descriptor does not match the one determined by the `UMPSignal::SelectCore` message - ### On-chain backing If the candidate descriptor is version 1, there are no changes. -For version 2 descriptors the runtime will determine the `core_index` using the same inputs -as backers did off-chain. It currently stores the claim queue at the newest allowed -relay parent corresponding to claim queue offset `0`. The runtime needs to be changed to store +For version 2 descriptors the runtime will determine the `core_index` using the same inputs +as backers did off-chain. It currently stores the claim queue at the newest allowed +relay parent corresponding to the claim queue offset `0`. The runtime needs to be changed to store a claim queue snapshot at all allowed relay parents. - ### Backwards compatibility -There are two flavors of candidate receipts that are used in network protocols, runtime and node +Two flavors of candidate receipts are used in network protocols, runtime and node implementation: -- `CommittedCandidateReceipt` which includes the `CandidateDescriptor` and the `CandidateCommitments` + +- `CommittedCandidateReceipt` which includes the `CandidateDescriptor` and the `CandidateCommitments` - `CandidateReceipt` which includes the `CandidateDescriptor` and just a hash of the commitments -We want to support both the old and new versions in the runtime and node, so the implementation must +We want to support both the old and new versions in the runtime and node, so the implementation must be able to detect the version of a given candidate receipt. The version of the descriptor is detected by checking the reserved fields. -If they are not zerored, it means it is a version 1 descriptor. Otherwise the `version` field -is used further to determine the version. It should be `0` for version 2 descriptors. If it is not +If they are not zeroed, it means it is a version 1 descriptor. Otherwise the `version` field +is used further to determine the version. It should be `0` for version 2 descriptors. If it is not the descriptor has an unknown version and should be considered invalid. - ## Drawbacks -The only drawback is that further additions to the descriptor are limited to the amount of +The only drawback is that further additions to the descriptor are limited to the amount of remaining unused space. ## Testing, Security, and Privacy -Standard testing (unit tests, CI zombienet tests) for functionality and mandatory security audit +Standard testing (unit tests, CI zombienet tests) for functionality and mandatory security audit to ensure the implementation does not introduce any new security issues. -Backwards compatibility of the implementation will be tested on testnets (Versi and Westend). +Backward compatibility of the implementation will be tested on testnets (Versi and Westend). There is no impact on privacy. @@ -233,59 +232,58 @@ There is no impact on privacy. Overall performance will be improved by not checking the collator signatures in runtime and nodes. The impact on the UMP queue and candidate receipt processing is negligible. -The `ClaimQueueOffset` along with the relay parent choice allows parachains to optimize their -block production for either throughput or latency. A value of `0` with newest relay parent -provides best latency while picking older relay parents avoids re-orgs. - +The `ClaimQueueOffset` along with the relay parent choice allows parachains to optimize their +block production for either throughput or latency. A value of `0` with the newest relay parent +provides the best latency while picking older relay parents avoids re-orgs. ## Ergonomics -It is mandatory for elastic parachains to switch to the new receipt format. It is optional but -desired that all parachains switch to the new receipts for providing the session index for +It is mandatory for elastic parachains to switch to the new receipt format. It is optional but +desired that all parachains switch to the new receipts for providing the session index for disputes. -The implementation of this RFC itself must not introduce any breaking changes for the parachain +The implementation of this RFC itself must not introduce any breaking changes for the parachain runtime or collator nodes. ## Compatibility -The proposed changes are backwards compatible in general, but additional care must be taken by -waiting for at least `2/3 + 1` validators to upgrade. Validators that have not upgraded will not -back candidates using the new descriptor format and will also initiate disputes against these +The proposed changes are backward compatible in general, but additional care must be taken by +waiting for at least `2/3 + 1` validators to upgrade. Validators that have not upgraded will not +back candidates using the new descriptor format and will also initiate disputes against these candidates. ### Relay chain runtime -The first step is to remove collator signature checking logic in the runtime, but keep the node -side collator signature checks. +The first step is to remove collator signature checking logic in the runtime but keep the node +side collator signature checks. -The runtime must be upgraded to support the new primitives before any collator or node is allowed -to use the new candidate receipts format. +The runtime must be upgraded to support the new primitives before any collator or node is allowed +to use the new candidate receipts format. ### Validators To ensure a smooth launch, a new node feature is required. -The feature acts as a signal for supporting the new candidate receipts on the node side and can -only be safely enabled if at least `2/3 + 1` of the validators are upgraded. Node implementions -need to decode the new candidate descriptor once the feature is enabled otherwise they might +The feature acts as a signal for supporting the new candidate receipts on the node side and can +only be safely enabled if at least `2/3 + 1` of the validators are upgraded. Node implementations +need to decode the new candidate descriptor once the feature is enabled otherwise they might raise disputes and get slashed. -Once the feature is enabled, the validators will skip checking the collator signature when -processing the candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if +Once the feature is enabled, the validators will skip checking the collator signature when +processing the candidate receipts and verify the `CoreIndex` and `SessionIndex` fields if present in the receipt. No new implementation of networking protocol versions for collation and validation is required. ### Tooling -Any tooling that decodes UMP XCM messages needs an update to support or ignore the new UMP -messages, but they should be fine to decode the regular XCM messages that come before the +Any tooling that decodes UMP XCM messages needs an update to support or ignore the new UMP +messages, but they should be fine to decode the regular XCM messages that come before the separator. ## Prior Art and References Forum discussion about a new `CandidateReceipt` format: - https://forum.polkadot.network/t/pre-rfc-discussion-candidate-receipt-format-v2/3738 + ## Unresolved Questions @@ -293,10 +291,10 @@ N/A ## Future Directions and Related Material -The implementation is extensible and future proof to some extent. With minimal or no breaking -changes, additional fields can be added in the candidate descriptor until the reserved space is +The implementation is extensible and future-proof to some extent. With minimal or no breaking +changes, additional fields can be added in the candidate descriptor until the reserved space is exhausted -At this point there is a simple way to determine the version of the receipt, by testing for zeroed -reserved bytes in the descriptor. Future versions of the receipt can be implemented and identified -by using the `version` field of the descirptor introduced in this RFC. \ No newline at end of file +At this point, there is a simple way to determine the version of the receipt, by testing for zeroed +reserved bytes in the descriptor. Future versions of the receipt can be implemented and identified +by using the `version` field of the descriptor introduced in this RFC. From d6d35b991e90ad461525b0169864d2b3421dab15 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Tue, 3 Sep 2024 14:26:45 +0300 Subject: [PATCH 21/24] feedback Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 66 ++++++++++---------- 1 file changed, 33 insertions(+), 33 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index a59292f31..45768b7d9 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -58,7 +58,7 @@ bad parachain blocks. This proposal aims to remove the collator signature and all the logic that checks the collator signatures of candidate receipts. We use the first 7 reclaimed bytes to represent the version, -the core, session index, and fill the rest with zeroes. So, there is no change in the layout +the core, session index, and fill the rest with zeroes. So, there is no change in the layout and length of the receipt. The new primitive is binary-compatible with the old one. ### UMP transport @@ -83,11 +83,13 @@ Example: #### `UMPSignal` messages ```rust -/// An `u8` wrap-around sequence number. Typically this would be the least significant byte of the -/// parachain block number. +/// The selector that determines the core index. pub struct CoreSelector(pub u8); -/// An offset in the relay chain claim queue. +/// The offset in the relay chain claim queue. +/// +/// The state of the claim queue is given by the relay chain block +/// that is used as context for the `PoV`. pub struct ClaimQueueOffset(pub u8); /// Signals sent by a parachain to the relay chain. @@ -95,14 +97,13 @@ pub enum UMPSignal { /// A message sent by a parachain to select the core the candidate is committed to. /// Relay chain validators, in particular backers, use the `CoreSelector` and `ClaimQueueOffset` /// to compute the index of the core the candidate has committed to. - /// SelectCore(CoreSelector, ClaimQueueOffset), } ``` -The parachain runtime is not concerned with the actual `CoreIndex` the candidate is intended for, -but must provide enough information for the validators and collators to compute it. `CoreSelector` -and `ClaimQueueOffset` use only 2 bytes and fulfills the requirement. +The `CoreSelector` together with the `ClaimQueueOffset` are used to index the claim queue. This way +the validators can compute the `CoreIndex` and ensure that the collator put the correct `CoreIndex` +into the `CandidateDescriptor`. **Example:** @@ -117,17 +118,14 @@ The table below represents a snapshot of the claim queue: | Core 3 | Para B | **Para A** | **Para A** | The purpose of `ClaimQueueOffset` is to select the column from the above table. -For `cq_offset = 1` we get `[ Para A, Para B, Para A]` and use as input to create -a sorted vec with the cores A is assigned to: `[ Core 1, Core 3]` and call it `para_assigned_cores`. +For `cq_offset = 1` we get `[Para A, Para B, Para A]` and use as input to create +a sorted vec with the cores A is assigned to: `[Core 1, Core 3]` and call it `para_assigned_cores`. We use `core_selector` and determine the committed core index is `Core 3` like this: ```rust let committed_core_index = para_assigned_cores[core_selector % para_assigned_cores.len()]; ``` -Parachains should prefer to have a static `ClaimQueueOffset` value that makes sense for their -usecase which can be changed by governance at some future point. - ### Polkadot Primitive changes #### New [CandidateDescriptor](https://github.com/paritytech/polkadot-sdk/blob/b5029eb4fd6c7ffd8164b2fe12b71bad0c59c9f2/polkadot/primitives/src/v7/mod.rs#L512) @@ -176,6 +174,22 @@ pub struct CandidateDescriptorV2 { In future format versions, parts of the `reserved1` and `reserved2` bytes can be used to include additional information in the descriptor. +### Backwards compatibility + +Two flavors of candidate receipts are used in network protocols, runtime and node +implementation: + +- `CommittedCandidateReceipt` which includes the `CandidateDescriptor` and the `CandidateCommitments` +- `CandidateReceipt` which includes the `CandidateDescriptor` and just a hash of the commitments + +We want to support both the old and new versions in the runtime and node, so the implementation must +be able to detect the version of a given candidate receipt. + +The version of the descriptor is detected by checking the reserved fields. +If they are not zeroed, it means it is a version 1 descriptor. Otherwise the `version` field +is used further to determine the version. It should be `0` for version 2 descriptors. If it is not +the descriptor has an unknown version and should be considered invalid. + ### Parachain block validation If the candidate descriptor is version 1, there are no changes. @@ -197,22 +211,6 @@ as backers did off-chain. It currently stores the claim queue at the newest allo relay parent corresponding to the claim queue offset `0`. The runtime needs to be changed to store a claim queue snapshot at all allowed relay parents. -### Backwards compatibility - -Two flavors of candidate receipts are used in network protocols, runtime and node -implementation: - -- `CommittedCandidateReceipt` which includes the `CandidateDescriptor` and the `CandidateCommitments` -- `CandidateReceipt` which includes the `CandidateDescriptor` and just a hash of the commitments - -We want to support both the old and new versions in the runtime and node, so the implementation must -be able to detect the version of a given candidate receipt. - -The version of the descriptor is detected by checking the reserved fields. -If they are not zeroed, it means it is a version 1 descriptor. Otherwise the `version` field -is used further to determine the version. It should be `0` for version 2 descriptors. If it is not -the descriptor has an unknown version and should be considered invalid. - ## Drawbacks The only drawback is that further additions to the descriptor are limited to the amount of @@ -247,10 +245,12 @@ runtime or collator nodes. ## Compatibility -The proposed changes are backward compatible in general, but additional care must be taken by -waiting for at least `2/3 + 1` validators to upgrade. Validators that have not upgraded will not -back candidates using the new descriptor format and will also initiate disputes against these -candidates. +The proposed changes are not fully backward compatible, because older validators verify the +collator signature of candidate descriptors. + +Additional care must be taken before enabling the new descriptors by waiting for at least +`2/3 + 1` validators to upgrade. Validators that have not upgraded will not back candidates +using the new descriptor format and will also initiate disputes against these candidates. ### Relay chain runtime From e01dd950d18b6b931320f1d7babc74c9b04f36ee Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Tue, 3 Sep 2024 18:06:49 +0300 Subject: [PATCH 22/24] make commitment optional Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 45768b7d9..67fbe0052 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -69,7 +69,8 @@ UMP queue by outputting them in [upward_messages](https://github.com/paritytech/ The UMP queue layout is changed to allow the relay chain to receive both the XCM messages and `UMPSignal` messages. An empty message (empty `Vec`) is used to mark the end of XCM messages and -the start of `UMPSignal` messages. +the start of `UMPSignal` messages.The `UMPSignal` is optional and can be omitted by parachains +not using elastic scaling. This way of representing the new messages has been chosen over introducing an enum wrapper to minimize breaking changes of XCM message decoding in tools like Subscan for example. @@ -236,9 +237,9 @@ provides the best latency while picking older relay parents avoids re-orgs. ## Ergonomics -It is mandatory for elastic parachains to switch to the new receipt format. It is optional but -desired that all parachains switch to the new receipts for providing the session index for -disputes. +It is mandatory for elastic parachains to switch to the new receipt format and commit to a +core by sending the `UMPSignal::SelectCore` message. It is optional but desired that all +parachains switch to the new receipts for providing the session index for disputes. The implementation of this RFC itself must not introduce any breaking changes for the parachain runtime or collator nodes. From 5962c06a192c6ea9517b39c81ec846e3e1155c61 Mon Sep 17 00:00:00 2001 From: Andrei Sandu Date: Tue, 3 Sep 2024 18:10:35 +0300 Subject: [PATCH 23/24] latency == XCM latency Signed-off-by: Andrei Sandu --- text/0103-introduce-core-index-commitment.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 67fbe0052..694f42ee8 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -232,8 +232,9 @@ Overall performance will be improved by not checking the collator signatures in The impact on the UMP queue and candidate receipt processing is negligible. The `ClaimQueueOffset` along with the relay parent choice allows parachains to optimize their -block production for either throughput or latency. A value of `0` with the newest relay parent -provides the best latency while picking older relay parents avoids re-orgs. +block production for either throughput or lower XCM message processing latency. A value of `0` +with the newest relay parent provides the best latency while picking older relay parents avoids +re-orgs. ## Ergonomics From 01719f7b8e74839056a3285e722658823743c81f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bastian=20K=C3=B6cher?= Date: Tue, 3 Sep 2024 20:42:04 +0200 Subject: [PATCH 24/24] Update text/0103-introduce-core-index-commitment.md --- text/0103-introduce-core-index-commitment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0103-introduce-core-index-commitment.md b/text/0103-introduce-core-index-commitment.md index 694f42ee8..660adca02 100644 --- a/text/0103-introduce-core-index-commitment.md +++ b/text/0103-introduce-core-index-commitment.md @@ -69,7 +69,7 @@ UMP queue by outputting them in [upward_messages](https://github.com/paritytech/ The UMP queue layout is changed to allow the relay chain to receive both the XCM messages and `UMPSignal` messages. An empty message (empty `Vec`) is used to mark the end of XCM messages and -the start of `UMPSignal` messages.The `UMPSignal` is optional and can be omitted by parachains +the start of `UMPSignal` messages. The `UMPSignal` is optional and can be omitted by parachains not using elastic scaling. This way of representing the new messages has been chosen over introducing an enum wrapper to