From c23b15cad1383d440df6cf95f4a94344f0c50c10 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Fri, 17 Nov 2023 13:48:13 +0100 Subject: [PATCH 01/33] WIP: yield execution --- neps/nep-0000-yield-execution.md | 136 +++++++++++++++++++++++++++++++ 1 file changed, 136 insertions(+) create mode 100644 neps/nep-0000-yield-execution.md diff --git a/neps/nep-0000-yield-execution.md b/neps/nep-0000-yield-execution.md new file mode 100644 index 000000000..693de9697 --- /dev/null +++ b/neps/nep-0000-yield-execution.md @@ -0,0 +1,136 @@ +--- +NEP: 0 +Title: Yield Execution +Authors: Akhi Singhania +Status: Draft +DiscussionsTo: https://github.com/nearprotocol/neps/pull/0000 +Type: Protocol +Version: 0.0.0 +Created: 2023-11-17 +LastUpdated: 2023-11-17 +--- + +[This is a NEP (NEAR Enhancement Proposal) template, as described in [NEP-0001](https://github.com/near/NEPs/blob/master/neps/nep-0001.md). Use this when creating a new NEP. The author should delete or replace all the comments or commented brackets when merging their NEP.] + + + +## Summary + +[Provide a short human-readable (~200 words) description of the proposal. A reader should get from this section a high-level understanding about the issue this NEP is addressing.] + +## Motivation + +[Explain why this proposal is necessary, how it will benefit the NEAR protocol or community, and what problems it solves. Also describe why the existing protocol specification is inadequate to address the problem that this NEP solves, and what potential use cases or outcomes.] + +## Specification + +[Explain the proposal as if you were teaching it to another developer. This generally means describing the syntax and semantics, naming new concepts, and providing clear examples. The specification needs to include sufficient detail to allow interoperable implementations getting built by following only the provided specification. In cases where it is infeasible to specify all implementation details upfront, broadly describe what they are.] + +## Reference Implementation + +[This technical section is required for Protocol proposals but optional for other categories. A draft implementation should demonstrate a minimal implementation that assists in understanding or implementing this proposal. Explain the design in sufficient detail that: + +* Its interaction with other features is clear. +* Where possible, include a Minimum Viable Interface subsection expressing the required behavior and types in a target programming language. (ie. traits and structs for rust, interfaces and classes for javascript, function signatures and structs for c, etc.) +* It is reasonably clear how the feature would be implemented. +* Corner cases are dissected by example. +* For protocol changes: A link to a draft PR on nearcore that shows how it can be integrated in the current code. It should at least solve the key technical challenges. + +The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work.] + +## Security Implications + +[Explicitly outline any security concerns in relation to the NEP, and potential ways to resolve or mitigate them. At the very least, well-known relevant threats must be covered, e.g. person-in-the-middle, double-spend, XSS, CSRF, etc.] + +## Alternatives + +[Explain any alternative designs that were considered and the rationale for not choosing them. Why your design is superior?] + +## Future possibilities + +[Describe any natural extensions and evolutions to the NEP proposal, and how they would impact the project. Use this section as a tool to help fully consider all possible interactions with the project in your proposal. This is also a good place to "dump ideas"; if they are out of scope for the NEP but otherwise related. Note that having something written down in the future-possibilities section is not a reason to accept the current or a future NEP. Such notes should be in the section on motivation or rationale in this or subsequent NEPs. The section merely provides additional information.] + +## Consequences + +[This section describes the consequences, after applying the decision. All consequences should be summarized here, not just the "positive" ones. Record any concerns raised throughout the NEP discussion.] + +### Positive + +* p1 + +### Neutral + +* n1 + +### Negative + +* n1 + +### Backwards Compatibility + +[All NEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. Author must explain a proposes to deal with these incompatibilities. Submissions without a sufficient backwards compatibility treatise may be rejected outright.] + +## Unresolved Issues (Optional) + +[Explain any issues that warrant further discussion. Considerations + +* What parts of the design do you expect to resolve through the NEP process before this gets merged? +* What parts of the design do you expect to resolve through the implementation of this feature before stabilization? +* What related issues do you consider out of scope for this NEP that could be addressed in the future independently of the solution that comes out of this NEP?] + +## Changelog + +[The changelog section provides historical context for how the NEP developed over time. Initial NEP submission should start with version 1.0.0, and all subsequent NEP extensions must follow [Semantic Versioning](https://semver.org/). Every version should have the benefits and concerns raised during the review. The author does not need to fill out this section for the initial draft. Instead, the assigned reviewers (Subject Matter Experts) should create the first version during the first technical review. After the final public call, the author should then finalize the last version of the decision context.] + +### 1.0.0 - Initial Version + +> Placeholder for the context about when and who approved this NEP version. + +#### Benefits + +> List of benefits filled by the Subject Matter Experts while reviewing this version: + +* Benefit 1 +* Benefit 2 + +#### Concerns + +> Template for Subject Matter Experts review for this version: +> Status: New | Ongoing | Resolved + +| # | Concern | Resolution | Status | +| --: | :------ | :--------- | -----: | +| 1 | | | | +| 2 | | | | + +## Copyright + +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). From 1f8dbbd474b803b515c11332cdec172aa2ab7585 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Fri, 17 Nov 2023 13:59:30 +0100 Subject: [PATCH 02/33] rename file as per instructions --- neps/{nep-0000-yield-execution.md => nep-519-yield-execution.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename neps/{nep-0000-yield-execution.md => nep-519-yield-execution.md} (100%) diff --git a/neps/nep-0000-yield-execution.md b/neps/nep-519-yield-execution.md similarity index 100% rename from neps/nep-0000-yield-execution.md rename to neps/nep-519-yield-execution.md From 0931acd907fa2c5d4526e47cd0c254ed4226a1db Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Fri, 17 Nov 2023 14:06:37 +0100 Subject: [PATCH 03/33] Add a summary and update some metadata --- neps/nep-519-yield-execution.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 693de9697..fcf4a3a8f 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -1,17 +1,15 @@ --- -NEP: 0 +NEP: 519 Title: Yield Execution Authors: Akhi Singhania Status: Draft -DiscussionsTo: https://github.com/nearprotocol/neps/pull/0000 +DiscussionsTo: https://github.com/near/NEPs/pull/519 Type: Protocol Version: 0.0.0 Created: 2023-11-17 LastUpdated: 2023-11-17 --- -[This is a NEP (NEAR Enhancement Proposal) template, as described in [NEP-0001](https://github.com/near/NEPs/blob/master/neps/nep-0001.md). Use this when creating a new NEP. The author should delete or replace all the comments or commented brackets when merging their NEP.] - ## Summary -[Provide a short human-readable (~200 words) description of the proposal. A reader should get from this section a high-level understanding about the issue this NEP is addressing.] +Today, when a smart contract is called by a user or another contract, it has no sensible way to delay responding to the caller. There exist use cases where contracts would benefit from being able to delay responding till some arbitrary time in the future. This proposal introduces such possibility into the NEAR protocol. ## Motivation From 5b54b470e06b663dd4491ce23ff21bdf6ae48881 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Mon, 20 Nov 2023 14:10:47 +0100 Subject: [PATCH 04/33] minor cleanup --- neps/nep-519-yield-execution.md | 32 +------------------------------- 1 file changed, 1 insertion(+), 31 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index fcf4a3a8f..41b50bba3 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -7,39 +7,9 @@ DiscussionsTo: https://github.com/near/NEPs/pull/519 Type: Protocol Version: 0.0.0 Created: 2023-11-17 -LastUpdated: 2023-11-17 +LastUpdated: 2023-11-20 --- - - ## Summary Today, when a smart contract is called by a user or another contract, it has no sensible way to delay responding to the caller. There exist use cases where contracts would benefit from being able to delay responding till some arbitrary time in the future. This proposal introduces such possibility into the NEAR protocol. From 20c993ff186ffdb615dd7990d3e52c321b2d9f96 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Mon, 20 Nov 2023 14:29:34 +0100 Subject: [PATCH 05/33] add motivation --- neps/nep-519-yield-execution.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 41b50bba3..260751625 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -16,7 +16,20 @@ Today, when a smart contract is called by a user or another contract, it has no ## Motivation -[Explain why this proposal is necessary, how it will benefit the NEAR protocol or community, and what problems it solves. Also describe why the existing protocol specification is inadequate to address the problem that this NEP solves, and what potential use cases or outcomes.] +There exist some situations where when a smart contract on NEAR is called, it will only be able to provide an answer at some arbitrary time in the future. So the callee needs a way to defer replying to the caller till this time in future. + +Examples include when a smart contract (`S`) provides MPC signing capabilities parties external to the NEAR protocol are computing the signature. The rough steps are: +1. Signer contract provides a function `fn sign_payload(Payload, ...)`. +2. When called, the contract updates some contract state which is being monitored by external indexers to indicate that a new signing request has been received. It also defers replying to the caller. +3. The indexers observe the new signing request, they compute a signature and call another function `fn signature_available(Signature, ...)` on the signer contract. +4. The signer contract validates the signature and if validate, replies to the original caller. + +Today, the NEAR protocol has no sensible way to defer replying to the caller in step 2 above. This proposal proposes adding two following new host functions to the NEAR protocol: + +- `yield`: this can be called by a contract to indicate to the protocol that it is not ready yet to reply to its caller. +- `resume`: a contract can use this mechanism to indicate to a protocol that it is now ready to reply to a caller that it had deferred earlier. + +If these two host functions were available, then `yield` would be used in step 2 above and `resume` would be used in step 4 above. ## Specification From b8d882c3772eb99bd911767e6aa133a2968d7153 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Tue, 5 Dec 2023 19:57:01 +0100 Subject: [PATCH 06/33] add draft of host function calls --- neps/nep-519-yield-execution.md | 49 ++++++++++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 260751625..322fdfad3 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -33,7 +33,54 @@ If these two host functions were available, then `yield` would be used in step 2 ## Specification -[Explain the proposal as if you were teaching it to another developer. This generally means describing the syntax and semantics, naming new concepts, and providing clear examples. The specification needs to include sufficient detail to allow interoperable implementations getting built by following only the provided specification. In cases where it is infeasible to specify all implementation details upfront, broadly describe what they are.] +The proposal is to add the following host functions to the NEAR protocol: + + +```rust +/// Instructs the protocol that the smart contract is not ready yet to respond +/// to its caller yet. The smart contract promises to call `yield_resume()` +/// within `yield_num_blocks` blocks. When `yield_resume()` is called, the +/// protocol will call the method on the smart contract that is identified by +/// `method_name_len` and `method_name_ptr` and this method will be expected to +/// either respond to the caller or create another promise. +/// +/// If the contract fails to call `yield_resume()` within `yield_num_blocks`, +/// then the protocol will call the method on the smart contract that is +/// identified by `method_name_len` and `method_name_ptr` with a timeout error. +/// +/// `gas_for_resumed_method` is the prepayment of Gas that will be used to +/// execute the method identified by `method_name_len` and `method_name_ptr`. +/// +/// `gas_weight`: as specified in +/// [this](https://github.com/near/NEPs/blob/master/neps/nep-0264.md) NEP, this +/// improves the devX by allowing the developer to specify how to divide up the +/// remaining gas. +/// +/// Return value: u64: Similar to the +/// [promise_create](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1281) +/// host function, this function also create a promise and returns an index to +/// the promise. +pub fn yield_create( + method_name_len: u64, + method_name_ptr: u64, + yield_num_blocks: u64, + gas_for_resumed_method: Gas, + gas_weight: u64, +) -> u64; + +/// When a smart contract has postponed replying to its caller earlier, it can +/// use this function to indicate that it may now be ready to reply to it. When +/// this is called, then the protocol will call the method that the smart +/// contract referred to in the earlier `yield_create()` call. +/// +/// `promise_index`: the index that was returned from the promise that created +/// from an earlier call to `yield_create()`. +/// +/// `arguments_len` and `arguments_ptr`: the smart contract can provide an +/// optional list of arguments that should be passed to the method that will be +/// resumed. +pub fn yield_resume(promise_index: u64, arguments_len: u64, argument_ptr: u64) -> (); +``` ## Reference Implementation From fcb52308628bfbf88ffb7b9b0e86e0ca670a6f33 Mon Sep 17 00:00:00 2001 From: Akhilesh Singhania Date: Tue, 12 Dec 2023 10:37:39 +0100 Subject: [PATCH 07/33] Apply suggestions from code review Co-authored-by: DavidM-D --- neps/nep-519-yield-execution.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 322fdfad3..af4e28774 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -41,8 +41,8 @@ The proposal is to add the following host functions to the NEAR protocol: /// to its caller yet. The smart contract promises to call `yield_resume()` /// within `yield_num_blocks` blocks. When `yield_resume()` is called, the /// protocol will call the method on the smart contract that is identified by -/// `method_name_len` and `method_name_ptr` and this method will be expected to -/// either respond to the caller or create another promise. +/// `method_name_len` and `method_name_ptr` and this method may respond to the caller. +/// Once the method has responded, `yield_resume` can no longer be called on this promise. /// /// If the contract fails to call `yield_resume()` within `yield_num_blocks`, /// then the protocol will call the method on the smart contract that is From bc6708bda70cfe5fb1b342efcedb489674ed2c40 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 14 Feb 2024 17:13:07 +0100 Subject: [PATCH 08/33] update the API --- neps/nep-519-yield-execution.md | 66 +++++++++++++++++++++------------ 1 file changed, 42 insertions(+), 24 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index af4e28774..036891db3 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -37,49 +37,67 @@ The proposal is to add the following host functions to the NEAR protocol: ```rust -/// Instructs the protocol that the smart contract is not ready yet to respond -/// to its caller yet. The smart contract promises to call `yield_resume()` -/// within `yield_num_blocks` blocks. When `yield_resume()` is called, the -/// protocol will call the method on the smart contract that is identified by -/// `method_name_len` and `method_name_ptr` and this method may respond to the caller. -/// Once the method has responded, `yield_resume` can no longer be called on this promise. +/// Instructs the protocol that the smart contract is not yet ready to respond +/// to its caller yet. The smart contract promises to call +/// `promise_yield_resume()` within X (TBD) blocks. When +/// `promise_yield_resume()` is called, the protocol will call the method on the +/// smart contract that is identified by `method_name_len` and `method_name_ptr` +/// and this method may or may not respond to the caller. /// -/// If the contract fails to call `yield_resume()` within `yield_num_blocks`, -/// then the protocol will call the method on the smart contract that is -/// identified by `method_name_len` and `method_name_ptr` with a timeout error. +/// `arguments_len` and `arguments_ptr` provide an initial set of arguments that +/// will be passed to the method. /// -/// `gas_for_resumed_method` is the prepayment of Gas that will be used to -/// execute the method identified by `method_name_len` and `method_name_ptr`. +/// If the contract fails to call `promise_yield_resume()` within X blocks, then +/// the protocol will call the method with a timeout error. +/// +/// Similar to the `gas` parameter in +/// [promise_create](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1281), +/// the `gas` parameter is a prepayment for the gas that would be used to +/// execute the method. /// /// `gas_weight`: as specified in /// [this](https://github.com/near/NEPs/blob/master/neps/nep-0264.md) NEP, this -/// improves the devX by allowing the developer to specify how to divide up the -/// remaining gas. +/// improves the devX of specifying a portion of the remaining gas for executing +/// the method instead of specifying a precise amount. +/// +/// `register_id`: is used to identify the register that will be used by the +/// protocol to return unique token referring to this yielded execution to the +/// contract. The contract will have to pass this value when it calls +/// `promise_yield_resume`. /// /// Return value: u64: Similar to the /// [promise_create](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1281) /// host function, this function also create a promise and returns an index to -/// the promise. -pub fn yield_create( +/// the promise. This index can be used to create a chain of promises. +pub fn promise_yield_create( method_name_len: u64, method_name_ptr: u64, - yield_num_blocks: u64, - gas_for_resumed_method: Gas, + arguments_len: u64, + arguments_ptr: u64, + gas: u64, gas_weight: u64, + register_id: u64, ) -> u64; /// When a smart contract has postponed replying to its caller earlier, it can /// use this function to indicate that it may now be ready to reply to it. When /// this is called, then the protocol will call the method that the smart -/// contract referred to in the earlier `yield_create()` call. +/// contract referred to in the earlier `promise_yield_create()` call. /// -/// `promise_index`: the index that was returned from the promise that created -/// from an earlier call to `yield_create()`. +/// `data_id_len` and `data_it_ptr`: This value was returned in an earlier call +/// to `promise_yield_create` and uniquely identifies which yielded execution +/// should be resumed. /// -/// `arguments_len` and `arguments_ptr`: the smart contract can provide an -/// optional list of arguments that should be passed to the method that will be -/// resumed. -pub fn yield_resume(promise_index: u64, arguments_len: u64, argument_ptr: u64) -> (); +/// `payload_len` and `payload_ptr`: the smart contract can provide an optional +/// list of arguments that should be passed to the method that will be resumed. +/// These will be appended to the list that was provided in +/// `promise_yield_create` +pub fn promise_yield_resume( + data_id_len: u64, + data_id_ptr: u64, + payload_len: u64, + payload_ptr: u64, +) -> (); ``` ## Reference Implementation From aef122ac20bb9fc49d84331bb7b263eff1dd9344 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 14 Feb 2024 17:39:07 +0100 Subject: [PATCH 09/33] add alternatives --- neps/nep-519-yield-execution.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 036891db3..1b65452b8 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -118,7 +118,17 @@ The section should return to the examples given in the previous section, and exp ## Alternatives -[Explain any alternative designs that were considered and the rationale for not choosing them. Why your design is superior?] +Two alternatives have been identified. + +### Self calls to delay replying + +In the `fn sign_payload(Payload, ...)` function, instead of calling `yield`, the contract can keep calling itself in a loop till external indexer replies with the signature. This would work but would be very fragile and expensive. The contract would have to pay for all the calls and function executions while it is waiting for the response. Also depending on the congestion on the network; if the shard is not busy at all, some self calls could happen within the same block meaning that the contract might not actually wait for as long as it hoped for and if the network is very busy then the call from the external indexer might be arbitrarily delayed. + +### Change the flow of calls + +The general flow of cross contract calls in NEAR is that a contract `A` sends a request to another contract `B` to perform a service and `B` replies to `A` with the response. This flow could be altered. When a contract `A` calls `B` to perform a service, `B` could respond with a "promise to call it later with the answer". Then when the signature is eventually available, `B` can then send `A` a request with the signature. + +There are some problems with this approach though. After the change of flow of calls; `B` is now going to be paying for gas for various executions that `A` should have been paying for. Due to bugs or malicious intent, `B` could forget to call `A` with the signature. If `A` is calling `B` deep in a call tree and `B` replies to it without actually providing an answer, then `A` would need a mechanism to keep the call tree alive while it waits for `B` to call it with the signature in effect running into the same problem that this NEP is attempting to solve. ## Future possibilities From 5ce8652eba237a988d2ba59727d3d6b2f3455bbc Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Mon, 19 Feb 2024 10:53:26 +0100 Subject: [PATCH 10/33] address some review comments --- neps/nep-519-yield-execution.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 1b65452b8..abb54e694 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -12,16 +12,17 @@ LastUpdated: 2023-11-20 ## Summary -Today, when a smart contract is called by a user or another contract, it has no sensible way to delay responding to the caller. There exist use cases where contracts would benefit from being able to delay responding till some arbitrary time in the future. This proposal introduces such possibility into the NEAR protocol. +Today, when a smart contract is called by a user or another contract, it has no sensible way to delay responding to the caller till it has observed another future transaction. This proposal introduces this possibility into the NEAR protocol. ## Motivation There exist some situations where when a smart contract on NEAR is called, it will only be able to provide an answer at some arbitrary time in the future. So the callee needs a way to defer replying to the caller till this time in future. Examples include when a smart contract (`S`) provides MPC signing capabilities parties external to the NEAR protocol are computing the signature. The rough steps are: + 1. Signer contract provides a function `fn sign_payload(Payload, ...)`. -2. When called, the contract updates some contract state which is being monitored by external indexers to indicate that a new signing request has been received. It also defers replying to the caller. -3. The indexers observe the new signing request, they compute a signature and call another function `fn signature_available(Signature, ...)` on the signer contract. +2. When called, the contract defers replying to the caller. +3. External indexers are monitoring the transactions on the contract; they observe the new signing request, compute a signature, and call another function `fn signature_available(Signature, ...)` on the signer contract. 4. The signer contract validates the signature and if validate, replies to the original caller. Today, the NEAR protocol has no sensible way to defer replying to the caller in step 2 above. This proposal proposes adding two following new host functions to the NEAR protocol: @@ -44,8 +45,8 @@ The proposal is to add the following host functions to the NEAR protocol: /// smart contract that is identified by `method_name_len` and `method_name_ptr` /// and this method may or may not respond to the caller. /// -/// `arguments_len` and `arguments_ptr` provide an initial set of arguments that -/// will be passed to the method. +/// `arguments_len` and `arguments_ptr` provide an initial blob of arguments +/// that will be passed to the method. /// /// If the contract fails to call `promise_yield_resume()` within X blocks, then /// the protocol will call the method with a timeout error. @@ -88,10 +89,10 @@ pub fn promise_yield_create( /// to `promise_yield_create` and uniquely identifies which yielded execution /// should be resumed. /// -/// `payload_len` and `payload_ptr`: the smart contract can provide an optional -/// list of arguments that should be passed to the method that will be resumed. -/// These will be appended to the list that was provided in -/// `promise_yield_create` +/// `payload_len` and `payload_ptr`: the smart contract can provide an +/// additional optional blob of arguments that should be passed to the method +/// that will be resumed. These will be appended to the list that was provided +/// in `promise_yield_create` pub fn promise_yield_resume( data_id_len: u64, data_id_ptr: u64, From 4cc7d8828eb78b4dbc372450c700256543488909 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Mon, 19 Feb 2024 14:28:04 +0100 Subject: [PATCH 11/33] add discussion of security implications --- neps/nep-519-yield-execution.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index abb54e694..2c346402f 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -115,7 +115,10 @@ The section should return to the examples given in the previous section, and exp ## Security Implications -[Explicitly outline any security concerns in relation to the NEP, and potential ways to resolve or mitigate them. At the very least, well-known relevant threats must be covered, e.g. person-in-the-middle, double-spend, XSS, CSRF, etc.] +Some potential security issues have been identified and are covered below: + +* Smart contracts using this functionality have to be careful not to let just any party trigger a call to `promise_yield_resume`. In the example above, it is possible that a malicious actor may pretend to be an external signer and call the `signature_available()` function with an incorrect signature. Hence contracts should be taking precautions by only letting select callers call the function (by using [this](https://github.com/aurora-is-near/near-plugins/blob/master/near-plugins/src/access_controllable.rs) service for example) and validating the payload before acting upon it. +* This mechanism introduces a new way to create delayed receipts in the protocol. When the protocol is under conditions of congestion, this mechanism could be used to further aggravate the situation. This is deemed as not a terrible issue as the existing mechanisms of using promises and etc. can also be used to further exacerbate the situation. ## Alternatives From a57d2a53aa8b0b8a1949bfe03efa436964001a66 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Mon, 19 Feb 2024 14:33:59 +0100 Subject: [PATCH 12/33] fix lint issue --- neps/nep-519-yield-execution.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 2c346402f..bf39662a4 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -117,8 +117,8 @@ The section should return to the examples given in the previous section, and exp Some potential security issues have been identified and are covered below: -* Smart contracts using this functionality have to be careful not to let just any party trigger a call to `promise_yield_resume`. In the example above, it is possible that a malicious actor may pretend to be an external signer and call the `signature_available()` function with an incorrect signature. Hence contracts should be taking precautions by only letting select callers call the function (by using [this](https://github.com/aurora-is-near/near-plugins/blob/master/near-plugins/src/access_controllable.rs) service for example) and validating the payload before acting upon it. -* This mechanism introduces a new way to create delayed receipts in the protocol. When the protocol is under conditions of congestion, this mechanism could be used to further aggravate the situation. This is deemed as not a terrible issue as the existing mechanisms of using promises and etc. can also be used to further exacerbate the situation. +- Smart contracts using this functionality have to be careful not to let just any party trigger a call to `promise_yield_resume`. In the example above, it is possible that a malicious actor may pretend to be an external signer and call the `signature_available()` function with an incorrect signature. Hence contracts should be taking precautions by only letting select callers call the function (by using [this](https://github.com/aurora-is-near/near-plugins/blob/master/near-plugins/src/access_controllable.rs) service for example) and validating the payload before acting upon it. +- This mechanism introduces a new way to create delayed receipts in the protocol. When the protocol is under conditions of congestion, this mechanism could be used to further aggravate the situation. This is deemed as not a terrible issue as the existing mechanisms of using promises and etc. can also be used to further exacerbate the situation. ## Alternatives From ce2a28dfa512aaac36323bf3be79918f1d9d61c4 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Mon, 19 Feb 2024 15:15:46 +0100 Subject: [PATCH 13/33] add a reference implementation --- neps/nep-519-yield-execution.md | 78 ++++++++++++++++++++++++++++++--- 1 file changed, 71 insertions(+), 7 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index bf39662a4..98df371f8 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -103,15 +103,79 @@ pub fn promise_yield_resume( ## Reference Implementation -[This technical section is required for Protocol proposals but optional for other categories. A draft implementation should demonstrate a minimal implementation that assists in understanding or implementing this proposal. Explain the design in sufficient detail that: +The reference implementation against the nearcore repository can be found in this [PR](https://github.com/near/nearcore/pull/10415). Below is a pseudocode reproduction of the relevant pieces. -* Its interaction with other features is clear. -* Where possible, include a Minimum Viable Interface subsection expressing the required behavior and types in a target programming language. (ie. traits and structs for rust, interfaces and classes for javascript, function signatures and structs for c, etc.) -* It is reasonably clear how the feature would be implemented. -* Corner cases are dissected by example. -* For protocol changes: A link to a draft PR on nearcore that shows how it can be integrated in the current code. It should at least solve the key technical challenges. +```rust +pub fn promise_yield_create( + &mut self, + method_name_len: u64, + method_name_ptr: u64, + arguments_len: u64, + arguments_ptr: u64, + gas: Gas, + gas_weight: u64, + register_id: u64, +) -> Result { + // Look up and validate the passed data + let (method_name, arguments) = lookup_and_validate( + method_name_len, + method_name_ptr, + arguments_len, + arguments_ptr, + ); + + let current_account_id = self.context.current_account_id.clone(); + + // Pay all gas fees for using the host function + self.gas_counter.pay_base(yield_create_base); + let num_bytes = method_name.len() + arguments.len(); + self.gas_counter.pay_per(yield_create_bytes, num_bytes); + + // Create a receipt with a single data dependency which will be resolved by + // either by the resume call or the timeout. + let data_id = self.generate_data_id(); + let receipt = ReceiptMetadata { + output_data_receivers: vec![], + input_data_ids: vec![data_id], + actions: vec![], + }; + let receipt_index = self.action_receipts.push((current_account_id, receipt)); + let promise_index = Promise::Receipt(receipt_index); + + self.registers.set(register_id, *data_id.as_bytes())?; + Ok(promise_index) +} -The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work.] +pub fn promise_yield_resume( + &mut self, + data_id_len: u64, + data_id_ptr: u64, + payload_len: u64, + payload_ptr: u64, +) -> Result<(), VMLogicError> { + // Look up and validate the passed data + let (data_id, payload) = + lookup_and_validate(data_id_len, data_id_ptr, payload_len, payload_ptr); + + let current_account_id = self.context.current_account_id.clone(); + + // Pay all gas fees for using the host function + self.gas_counter.pay_base(yield_resume_base); + self.gas_counter.pay_per(yield_submit_byte, payload_len); + + // Look up the previously yielded promise from the trie + let yielded_promise = get_yielded_promise(data_id); + + // Yields can only be resumed by the account which created them + if yielded_promise.account_id != current_account_id { + return Error; + } + + // Create a data receipt and it on the the queue of data receipts so that + // eventually the function call will be processed. + return self.receipt_manager.create_data_receipt(data_id, payload); +} +``` ## Security Implications From 3fa309a119bc5c6bae9cf842612e99c81e3ae334 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Mon, 19 Feb 2024 18:37:04 +0100 Subject: [PATCH 14/33] fix some more lints --- neps/nep-519-yield-execution.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 98df371f8..298fd0f18 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -208,15 +208,15 @@ There are some problems with this approach though. After the change of flow of ### Positive -* p1 +- p1 ### Neutral -* n1 +- n1 ### Negative -* n1 +- n1 ### Backwards Compatibility @@ -226,9 +226,9 @@ There are some problems with this approach though. After the change of flow of [Explain any issues that warrant further discussion. Considerations -* What parts of the design do you expect to resolve through the NEP process before this gets merged? -* What parts of the design do you expect to resolve through the implementation of this feature before stabilization? -* What related issues do you consider out of scope for this NEP that could be addressed in the future independently of the solution that comes out of this NEP?] +- What parts of the design do you expect to resolve through the NEP process before this gets merged? +- What parts of the design do you expect to resolve through the implementation of this feature before stabilization? +- What related issues do you consider out of scope for this NEP that could be addressed in the future independently of the solution that comes out of this NEP?] ## Changelog @@ -242,8 +242,8 @@ There are some problems with this approach though. After the change of flow of > List of benefits filled by the Subject Matter Experts while reviewing this version: -* Benefit 1 -* Benefit 2 +- Benefit 1 +- Benefit 2 #### Concerns From 4f0bf493807ae2e6e7762292da01db2a28467b5b Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Thu, 22 Feb 2024 15:33:25 +0100 Subject: [PATCH 15/33] Add Saketh as author --- neps/nep-519-yield-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 298fd0f18..f556ecbb4 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -1,7 +1,7 @@ --- NEP: 519 Title: Yield Execution -Authors: Akhi Singhania +Authors: Akhi Singhania ; Saketh Are Status: Draft DiscussionsTo: https://github.com/near/NEPs/pull/519 Type: Protocol From 48757e4bdb379efd9ba6159a36a542644cf01c32 Mon Sep 17 00:00:00 2001 From: Akhilesh Singhania Date: Wed, 28 Feb 2024 09:34:10 +0100 Subject: [PATCH 16/33] Update neps/nep-519-yield-execution.md Co-authored-by: Michael Birch --- neps/nep-519-yield-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index f556ecbb4..6671379f4 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -27,7 +27,7 @@ Examples include when a smart contract (`S`) provides MPC signing capabilities p Today, the NEAR protocol has no sensible way to defer replying to the caller in step 2 above. This proposal proposes adding two following new host functions to the NEAR protocol: -- `yield`: this can be called by a contract to indicate to the protocol that it is not ready yet to reply to its caller. +- `promise_yield_create`: this can be called by a contract to indicate to the protocol that it is not ready yet to reply to its caller. - `resume`: a contract can use this mechanism to indicate to a protocol that it is now ready to reply to a caller that it had deferred earlier. If these two host functions were available, then `yield` would be used in step 2 above and `resume` would be used in step 4 above. From 60da3e17d4fdaaddabbbba89488a6e385532ef70 Mon Sep 17 00:00:00 2001 From: Akhilesh Singhania Date: Wed, 28 Feb 2024 09:34:26 +0100 Subject: [PATCH 17/33] Update neps/nep-519-yield-execution.md Co-authored-by: Michael Birch --- neps/nep-519-yield-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 6671379f4..485959d54 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -28,7 +28,7 @@ Examples include when a smart contract (`S`) provides MPC signing capabilities p Today, the NEAR protocol has no sensible way to defer replying to the caller in step 2 above. This proposal proposes adding two following new host functions to the NEAR protocol: - `promise_yield_create`: this can be called by a contract to indicate to the protocol that it is not ready yet to reply to its caller. -- `resume`: a contract can use this mechanism to indicate to a protocol that it is now ready to reply to a caller that it had deferred earlier. +- `promise_yield_resume`: a contract can use this mechanism to indicate to a protocol that it is now ready to reply to a caller that it had deferred earlier. If these two host functions were available, then `yield` would be used in step 2 above and `resume` would be used in step 4 above. From 48bd2a9d01a2527085006876a89f801840793bb9 Mon Sep 17 00:00:00 2001 From: Akhilesh Singhania Date: Wed, 28 Feb 2024 09:34:47 +0100 Subject: [PATCH 18/33] Update neps/nep-519-yield-execution.md Co-authored-by: Michael Birch --- neps/nep-519-yield-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 485959d54..a1f97f3b1 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -30,7 +30,7 @@ Today, the NEAR protocol has no sensible way to defer replying to the caller in - `promise_yield_create`: this can be called by a contract to indicate to the protocol that it is not ready yet to reply to its caller. - `promise_yield_resume`: a contract can use this mechanism to indicate to a protocol that it is now ready to reply to a caller that it had deferred earlier. -If these two host functions were available, then `yield` would be used in step 2 above and `resume` would be used in step 4 above. +If these two host functions were available, then `promise_yield_create` would be used in step 2 above and `promise_yield_resume` would be used in step 4 above. ## Specification From 64b5fb6f92fc3a4e6a89faa5e478e48f7fdb03b1 Mon Sep 17 00:00:00 2001 From: Akhilesh Singhania Date: Wed, 28 Feb 2024 09:35:53 +0100 Subject: [PATCH 19/33] Update neps/nep-519-yield-execution.md Co-authored-by: Simonas Kazlauskas --- neps/nep-519-yield-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index a1f97f3b1..d5080f05b 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -23,7 +23,7 @@ Examples include when a smart contract (`S`) provides MPC signing capabilities p 1. Signer contract provides a function `fn sign_payload(Payload, ...)`. 2. When called, the contract defers replying to the caller. 3. External indexers are monitoring the transactions on the contract; they observe the new signing request, compute a signature, and call another function `fn signature_available(Signature, ...)` on the signer contract. -4. The signer contract validates the signature and if validate, replies to the original caller. +4. The signer contract validates the signature and replies to the original caller. Today, the NEAR protocol has no sensible way to defer replying to the caller in step 2 above. This proposal proposes adding two following new host functions to the NEAR protocol: From c0586140051207da52bd84fcf97c880f5c4abae1 Mon Sep 17 00:00:00 2001 From: Akhilesh Singhania Date: Wed, 28 Feb 2024 09:36:36 +0100 Subject: [PATCH 20/33] Update neps/nep-519-yield-execution.md Co-authored-by: Simonas Kazlauskas --- neps/nep-519-yield-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index d5080f05b..dcb6d9e37 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -16,7 +16,7 @@ Today, when a smart contract is called by a user or another contract, it has no ## Motivation -There exist some situations where when a smart contract on NEAR is called, it will only be able to provide an answer at some arbitrary time in the future. So the callee needs a way to defer replying to the caller till this time in future. +There exist some situations where when a smart contract on NEAR is called, it will only be able to provide an answer at some time in the future. The callee needs a way to defer replying to the caller while the response is being prepared. Examples include when a smart contract (`S`) provides MPC signing capabilities parties external to the NEAR protocol are computing the signature. The rough steps are: From 708447328cb4aa58d2beb866464d9da1e8d8fe0b Mon Sep 17 00:00:00 2001 From: Akhilesh Singhania Date: Wed, 28 Feb 2024 09:37:38 +0100 Subject: [PATCH 21/33] Update neps/nep-519-yield-execution.md Co-authored-by: Simonas Kazlauskas --- neps/nep-519-yield-execution.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index dcb6d9e37..29545de88 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -61,10 +61,10 @@ The proposal is to add the following host functions to the NEAR protocol: /// improves the devX of specifying a portion of the remaining gas for executing /// the method instead of specifying a precise amount. /// -/// `register_id`: is used to identify the register that will be used by the -/// protocol to return unique token referring to this yielded execution to the -/// contract. The contract will have to pass this value when it calls -/// `promise_yield_resume`. +/// `register_id`: is used to identify the register that will be filled +/// with a unique resumption token. This token is used with +/// `promise_yield_resume` to resolve the continuation receipt set up by this +/// function. /// /// Return value: u64: Similar to the /// [promise_create](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1281) From 8acb8dae5541e4c7e7d0d7be9695b651abdd22a5 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 28 Feb 2024 15:12:51 +0100 Subject: [PATCH 22/33] specify blocks --- neps/nep-519-yield-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 29545de88..b261098cc 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -40,7 +40,7 @@ The proposal is to add the following host functions to the NEAR protocol: ```rust /// Instructs the protocol that the smart contract is not yet ready to respond /// to its caller yet. The smart contract promises to call -/// `promise_yield_resume()` within X (TBD) blocks. When +/// `promise_yield_resume()` within 200 blocks. When /// `promise_yield_resume()` is called, the protocol will call the method on the /// smart contract that is identified by `method_name_len` and `method_name_ptr` /// and this method may or may not respond to the caller. From 7adce0e91e4921e412b8549d303bdfe6aa927b20 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 28 Feb 2024 15:14:29 +0100 Subject: [PATCH 23/33] specify blocks again --- neps/nep-519-yield-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index b261098cc..f31b4d828 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -48,7 +48,7 @@ The proposal is to add the following host functions to the NEAR protocol: /// `arguments_len` and `arguments_ptr` provide an initial blob of arguments /// that will be passed to the method. /// -/// If the contract fails to call `promise_yield_resume()` within X blocks, then +/// If the contract fails to call `promise_yield_resume()` within 200 blocks, then /// the protocol will call the method with a timeout error. /// /// Similar to the `gas` parameter in From 09cc2e054c34978bcedfbf4320bee154f3a2afcf Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 28 Feb 2024 15:23:39 +0100 Subject: [PATCH 24/33] cleanup templates that are not needed --- neps/nep-519-yield-execution.md | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index f31b4d828..a6e5af84d 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -200,7 +200,7 @@ There are some problems with this approach though. After the change of flow of ## Future possibilities -[Describe any natural extensions and evolutions to the NEP proposal, and how they would impact the project. Use this section as a tool to help fully consider all possible interactions with the project in your proposal. This is also a good place to "dump ideas"; if they are out of scope for the NEP but otherwise related. Note that having something written down in the future-possibilities section is not a reason to accept the current or a future NEP. Such notes should be in the section on motivation or rationale in this or subsequent NEPs. The section merely provides additional information.] +One potential future possibility is to allow contracts to specify how long the protocol should wait (up to a certain limit) for the contract to call `promise_yield_resume`. If contracts specify a smaller value, they would potentially be charged a smaller gas fee. This would make contracts more efficient. This enhancement does lead to a more complex implementation and could even allow malicious contracts to more easily concentrate a lot of callbacks to occur at the same time increasing the congestion on the network. Hence, we decided not to include this feature for the time being. ## Consequences @@ -220,20 +220,10 @@ There are some problems with this approach though. After the change of flow of ### Backwards Compatibility -[All NEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. Author must explain a proposes to deal with these incompatibilities. Submissions without a sufficient backwards compatibility treatise may be rejected outright.] - -## Unresolved Issues (Optional) - -[Explain any issues that warrant further discussion. Considerations - -- What parts of the design do you expect to resolve through the NEP process before this gets merged? -- What parts of the design do you expect to resolve through the implementation of this feature before stabilization? -- What related issues do you consider out of scope for this NEP that could be addressed in the future independently of the solution that comes out of this NEP?] +We believe this can be implemented with full backwards compatibility. ## Changelog -[The changelog section provides historical context for how the NEP developed over time. Initial NEP submission should start with version 1.0.0, and all subsequent NEP extensions must follow [Semantic Versioning](https://semver.org/). Every version should have the benefits and concerns raised during the review. The author does not need to fill out this section for the initial draft. Instead, the assigned reviewers (Subject Matter Experts) should create the first version during the first technical review. After the final public call, the author should then finalize the last version of the decision context.] - ### 1.0.0 - Initial Version > Placeholder for the context about when and who approved this NEP version. From 8facd6b022e94fcca9111f984dc4016bde783044 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 28 Feb 2024 15:29:46 +0100 Subject: [PATCH 25/33] remove ref impl --- neps/nep-519-yield-execution.md | 74 +-------------------------------- 1 file changed, 1 insertion(+), 73 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index a6e5af84d..318e209cf 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -103,79 +103,7 @@ pub fn promise_yield_resume( ## Reference Implementation -The reference implementation against the nearcore repository can be found in this [PR](https://github.com/near/nearcore/pull/10415). Below is a pseudocode reproduction of the relevant pieces. - -```rust -pub fn promise_yield_create( - &mut self, - method_name_len: u64, - method_name_ptr: u64, - arguments_len: u64, - arguments_ptr: u64, - gas: Gas, - gas_weight: u64, - register_id: u64, -) -> Result { - // Look up and validate the passed data - let (method_name, arguments) = lookup_and_validate( - method_name_len, - method_name_ptr, - arguments_len, - arguments_ptr, - ); - - let current_account_id = self.context.current_account_id.clone(); - - // Pay all gas fees for using the host function - self.gas_counter.pay_base(yield_create_base); - let num_bytes = method_name.len() + arguments.len(); - self.gas_counter.pay_per(yield_create_bytes, num_bytes); - - // Create a receipt with a single data dependency which will be resolved by - // either by the resume call or the timeout. - let data_id = self.generate_data_id(); - let receipt = ReceiptMetadata { - output_data_receivers: vec![], - input_data_ids: vec![data_id], - actions: vec![], - }; - let receipt_index = self.action_receipts.push((current_account_id, receipt)); - let promise_index = Promise::Receipt(receipt_index); - - self.registers.set(register_id, *data_id.as_bytes())?; - Ok(promise_index) -} - -pub fn promise_yield_resume( - &mut self, - data_id_len: u64, - data_id_ptr: u64, - payload_len: u64, - payload_ptr: u64, -) -> Result<(), VMLogicError> { - // Look up and validate the passed data - let (data_id, payload) = - lookup_and_validate(data_id_len, data_id_ptr, payload_len, payload_ptr); - - let current_account_id = self.context.current_account_id.clone(); - - // Pay all gas fees for using the host function - self.gas_counter.pay_base(yield_resume_base); - self.gas_counter.pay_per(yield_submit_byte, payload_len); - - // Look up the previously yielded promise from the trie - let yielded_promise = get_yielded_promise(data_id); - - // Yields can only be resumed by the account which created them - if yielded_promise.account_id != current_account_id { - return Error; - } - - // Create a data receipt and it on the the queue of data receipts so that - // eventually the function call will be processed. - return self.receipt_manager.create_data_receipt(data_id, payload); -} -``` +The reference implementation against the nearcore repository can be found in this [PR](https://github.com/near/nearcore/pull/10415). ## Security Implications From 43fa6f351263771f0653c087d29410a68f6fce84 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 28 Feb 2024 17:02:04 +0100 Subject: [PATCH 26/33] clarify how the args are available and some formatting --- neps/nep-519-yield-execution.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 318e209cf..78ea1df19 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -40,16 +40,17 @@ The proposal is to add the following host functions to the NEAR protocol: ```rust /// Instructs the protocol that the smart contract is not yet ready to respond /// to its caller yet. The smart contract promises to call -/// `promise_yield_resume()` within 200 blocks. When -/// `promise_yield_resume()` is called, the protocol will call the method on the -/// smart contract that is identified by `method_name_len` and `method_name_ptr` -/// and this method may or may not respond to the caller. +/// `promise_yield_resume()` within 200 blocks. When `promise_yield_resume()` +/// is called, the protocol will call the method on the smart contract that is +/// identified by `method_name_len` and `method_name_ptr` and this method may or +/// may not respond to the caller. /// /// `arguments_len` and `arguments_ptr` provide an initial blob of arguments -/// that will be passed to the method. +/// that will be passed to the method. These will be available via the `input` +/// host function. /// -/// If the contract fails to call `promise_yield_resume()` within 200 blocks, then -/// the protocol will call the method with a timeout error. +/// If the contract fails to call `promise_yield_resume()` within 200 blocks, +/// then the protocol will call the method with a timeout error. /// /// Similar to the `gas` parameter in /// [promise_create](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1281), @@ -61,10 +62,9 @@ The proposal is to add the following host functions to the NEAR protocol: /// improves the devX of specifying a portion of the remaining gas for executing /// the method instead of specifying a precise amount. /// -/// `register_id`: is used to identify the register that will be filled -/// with a unique resumption token. This token is used with -/// `promise_yield_resume` to resolve the continuation receipt set up by this -/// function. +/// `register_id`: is used to identify the register that will be filled with a +/// unique resumption token. This token is used with `promise_yield_resume` to +/// resolve the continuation receipt set up by this function. /// /// Return value: u64: Similar to the /// [promise_create](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1281) @@ -91,14 +91,15 @@ pub fn promise_yield_create( /// /// `payload_len` and `payload_ptr`: the smart contract can provide an /// additional optional blob of arguments that should be passed to the method -/// that will be resumed. These will be appended to the list that was provided -/// in `promise_yield_create` +/// that will be resumed. These are available via the `promise_result` host +/// function. pub fn promise_yield_resume( data_id_len: u64, data_id_ptr: u64, payload_len: u64, payload_ptr: u64, ) -> (); + ``` ## Reference Implementation From cd9bc850d2bbebefda7c197a28fec135b4b7d02e Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 28 Feb 2024 17:06:40 +0100 Subject: [PATCH 27/33] minor improv english --- neps/nep-519-yield-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 78ea1df19..23ff22710 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -18,7 +18,7 @@ Today, when a smart contract is called by a user or another contract, it has no There exist some situations where when a smart contract on NEAR is called, it will only be able to provide an answer at some time in the future. The callee needs a way to defer replying to the caller while the response is being prepared. -Examples include when a smart contract (`S`) provides MPC signing capabilities parties external to the NEAR protocol are computing the signature. The rough steps are: +Examples include a smart contract (`S`) that provides the MPC signing capability. It relies on indexers external to the NEAR protocol for computing the signatures. The rough steps are: 1. Signer contract provides a function `fn sign_payload(Payload, ...)`. 2. When called, the contract defers replying to the caller. From 812cff345d9e2f04253d93139be3c582197017dd Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 28 Feb 2024 17:11:25 +0100 Subject: [PATCH 28/33] apply suggestion from review --- neps/nep-519-yield-execution.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 23ff22710..e723113da 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -27,8 +27,8 @@ Examples include a smart contract (`S`) that provides the MPC signing capability Today, the NEAR protocol has no sensible way to defer replying to the caller in step 2 above. This proposal proposes adding two following new host functions to the NEAR protocol: -- `promise_yield_create`: this can be called by a contract to indicate to the protocol that it is not ready yet to reply to its caller. -- `promise_yield_resume`: a contract can use this mechanism to indicate to a protocol that it is now ready to reply to a caller that it had deferred earlier. +- `promise_yield_create`: allows setting up a continuation function that should only be executed after `promise_yield_resume` is invoked. Together with `promise_return` this allows delaying the reply to the caller; +- `promise_yield_resume`: indicates to the protocol that the continuation to the yield may now be executed. If these two host functions were available, then `promise_yield_create` would be used in step 2 above and `promise_yield_resume` would be used in step 4 above. From 2728727e6da0851c2c547d70f07a3660224e9d46 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 28 Feb 2024 17:13:13 +0100 Subject: [PATCH 29/33] apply suggestion from review --- neps/nep-519-yield-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index e723113da..06d3ed6b8 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -30,7 +30,7 @@ Today, the NEAR protocol has no sensible way to defer replying to the caller in - `promise_yield_create`: allows setting up a continuation function that should only be executed after `promise_yield_resume` is invoked. Together with `promise_return` this allows delaying the reply to the caller; - `promise_yield_resume`: indicates to the protocol that the continuation to the yield may now be executed. -If these two host functions were available, then `promise_yield_create` would be used in step 2 above and `promise_yield_resume` would be used in step 4 above. +If these two host functions were available, then `promise_yield_create` would be used to implement step 2 above and `promise_yield_resume` would be used for step 3 of the motivating example above. ## Specification From 577970ea9d45ee11cc84bc7c0f0be0025081ece6 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Wed, 28 Feb 2024 19:28:04 +0100 Subject: [PATCH 30/33] Improve specification --- neps/nep-519-yield-execution.md | 56 ++++++++++++++++++--------------- 1 file changed, 30 insertions(+), 26 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 06d3ed6b8..38055c8fc 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -38,29 +38,32 @@ The proposal is to add the following host functions to the NEAR protocol: ```rust -/// Instructs the protocol that the smart contract is not yet ready to respond -/// to its caller yet. The smart contract promises to call -/// `promise_yield_resume()` within 200 blocks. When `promise_yield_resume()` -/// is called, the protocol will call the method on the smart contract that is -/// identified by `method_name_len` and `method_name_ptr` and this method may or -/// may not respond to the caller. +/// Smart contracts can use this host function along with +/// `promise_yield_resume()` to delay replying to their caller for up to 200 +/// blocks. This host function allows the contract to provide a callback to the +/// protocol that will be executed after either contract calls +/// `promise_yield_resume()` or after 200 blocks have been executed. The +/// callback then has the opportunity to either reply to the caller or to delay +/// replying again. /// -/// `arguments_len` and `arguments_ptr` provide an initial blob of arguments -/// that will be passed to the method. These will be available via the `input` -/// host function. +/// `method_name_len` and `method_name_ptr`: Identify the callback method that +/// should be executed either after the contract calls `promise_yield_resume()` +/// or after 200 blocks have been executed. /// -/// If the contract fails to call `promise_yield_resume()` within 200 blocks, -/// then the protocol will call the method with a timeout error. +/// `arguments_len` and `arguments_ptr` provide an initial blob of arguments +/// that will be passed to the callback. These will be available via the +/// `input` host function. /// -/// Similar to the `gas` parameter in +/// `gas`: Similar to the `gas` parameter in /// [promise_create](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1281), /// the `gas` parameter is a prepayment for the gas that would be used to -/// execute the method. +/// execute the callback. /// -/// `gas_weight`: as specified in -/// [this](https://github.com/near/NEPs/blob/master/neps/nep-0264.md) NEP, this -/// improves the devX of specifying a portion of the remaining gas for executing -/// the method instead of specifying a precise amount. +/// `gas_weight`: Similar to the `gas_weight` parameter in +/// [promise_batch_action_function_call_weight](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1699), +/// this improves the devX for the smart contract. It allows a contract to +/// specify a portion of the remaining gas for executing the callback instead of +/// specifying a precise amount. /// /// `register_id`: is used to identify the register that will be filled with a /// unique resumption token. This token is used with `promise_yield_resume` to @@ -80,17 +83,19 @@ pub fn promise_yield_create( register_id: u64, ) -> u64; -/// When a smart contract has postponed replying to its caller earlier, it can -/// use this function to indicate that it may now be ready to reply to it. When -/// this is called, then the protocol will call the method that the smart -/// contract referred to in the earlier `promise_yield_create()` call. +/// See `promise_yield_create()` for more details. This host function can be +/// used to resolve the continuation that was set up by +/// `promise_yield_create()`. The contract calling this function must be the +/// same contract that called `promise_yield_create()` earlier. This host +/// function cannot be called for the same resumption token twice or if the +/// callback specified in `promise_yield_create()` has already executed. /// -/// `data_id_len` and `data_it_ptr`: This value was returned in an earlier call -/// to `promise_yield_create` and uniquely identifies which yielded execution -/// should be resumed. +/// `data_id_len` and `data_it_ptr`: Used to pass the unique resumption token +/// that was returned to the smart contract in the `promise_yield_create()` +/// function. /// /// `payload_len` and `payload_ptr`: the smart contract can provide an -/// additional optional blob of arguments that should be passed to the method +/// additional optional blob of arguments that should be passed to the callback /// that will be resumed. These are available via the `promise_result` host /// function. pub fn promise_yield_resume( @@ -99,7 +104,6 @@ pub fn promise_yield_resume( payload_len: u64, payload_ptr: u64, ) -> (); - ``` ## Reference Implementation From 13ebeb6c687e00ae9be0f3e62fd2056fa5366537 Mon Sep 17 00:00:00 2001 From: Akhilesh Singhania Date: Thu, 7 Mar 2024 18:45:23 +0100 Subject: [PATCH 31/33] Update neps/nep-519-yield-execution.md Co-authored-by: Simonas Kazlauskas --- neps/nep-519-yield-execution.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 38055c8fc..66d33110a 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -98,12 +98,18 @@ pub fn promise_yield_create( /// additional optional blob of arguments that should be passed to the callback /// that will be resumed. These are available via the `promise_result` host /// function. +/// +/// The function returns `1` if submitting the payload was successful. This +/// guarantees that the yielded callback will be executes with one of the +/// successfully submitted payloads as the input. Otherwise (e.g. if the yield +/// receipt has already timed out) `0` will be returned, indicating that this +/// payload could not be submitted successfully. pub fn promise_yield_resume( data_id_len: u64, data_id_ptr: u64, payload_len: u64, payload_ptr: u64, -) -> (); +) -> u32; ``` ## Reference Implementation From 10920a635728a7b0dd3b1ca89ff5954eb1b99877 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Thu, 7 Mar 2024 18:53:13 +0100 Subject: [PATCH 32/33] some minor improvements to the comments --- neps/nep-519-yield-execution.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 66d33110a..6ea51641a 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -92,18 +92,20 @@ pub fn promise_yield_create( /// /// `data_id_len` and `data_it_ptr`: Used to pass the unique resumption token /// that was returned to the smart contract in the `promise_yield_create()` -/// function. +/// function (via the register). /// /// `payload_len` and `payload_ptr`: the smart contract can provide an /// additional optional blob of arguments that should be passed to the callback /// that will be resumed. These are available via the `promise_result` host /// function. /// -/// The function returns `1` if submitting the payload was successful. This -/// guarantees that the yielded callback will be executes with one of the -/// successfully submitted payloads as the input. Otherwise (e.g. if the yield -/// receipt has already timed out) `0` will be returned, indicating that this -/// payload could not be submitted successfully. +/// This function can be called multiple times. If it is called successfully +/// multiple times, then the implementation guarantees that the yielded callback +/// will execute with one of the successfully submitted payloads. If submission +/// was successful, then `1` is returned. Otherwise (e.g. if the yield receipt +/// has already timed out or the yielded callback has already been executed) `0` +/// will be returned, indicating that this payload could not be submitted +/// successfully. pub fn promise_yield_resume( data_id_len: u64, data_id_ptr: u64, From 77bb83ed99de7129636d5022d47470d7bfe56d37 Mon Sep 17 00:00:00 2001 From: Akhi Singhania Date: Fri, 8 Mar 2024 15:47:46 +0100 Subject: [PATCH 33/33] apply minor nit comment --- neps/nep-519-yield-execution.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/neps/nep-519-yield-execution.md b/neps/nep-519-yield-execution.md index 6ea51641a..e636d0df3 100644 --- a/neps/nep-519-yield-execution.md +++ b/neps/nep-519-yield-execution.md @@ -99,13 +99,13 @@ pub fn promise_yield_create( /// that will be resumed. These are available via the `promise_result` host /// function. /// -/// This function can be called multiple times. If it is called successfully -/// multiple times, then the implementation guarantees that the yielded callback -/// will execute with one of the successfully submitted payloads. If submission -/// was successful, then `1` is returned. Otherwise (e.g. if the yield receipt -/// has already timed out or the yielded callback has already been executed) `0` -/// will be returned, indicating that this payload could not be submitted -/// successfully. +/// This function can be called multiple times with the same data id. If it is +/// called successfully multiple times, then the implementation guarantees that +/// the yielded callback will execute with one of the successfully submitted +/// payloads. If submission was successful, then `1` is returned. Otherwise +/// (e.g. if the yield receipt has already timed out or the yielded callback has +/// already been executed) `0` will be returned, indicating that this payload +/// could not be submitted successfully. pub fn promise_yield_resume( data_id_len: u64, data_id_ptr: u64,