Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for yielded execution #519

Merged
merged 33 commits into from
Jun 18, 2024
Merged
Changes from 7 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
c23b15c
WIP: yield execution
akhi3030 Nov 17, 2023
1f8dbbd
rename file as per instructions
akhi3030 Nov 17, 2023
0931acd
Add a summary and update some metadata
akhi3030 Nov 17, 2023
5b54b47
minor cleanup
akhi3030 Nov 20, 2023
20c993f
add motivation
akhi3030 Nov 20, 2023
b8d882c
add draft of host function calls
akhi3030 Dec 5, 2023
fcb5230
Apply suggestions from code review
akhi3030 Dec 12, 2023
bc6708b
update the API
akhi3030 Feb 14, 2024
aef122a
add alternatives
akhi3030 Feb 14, 2024
5ce8652
address some review comments
akhi3030 Feb 19, 2024
4cc7d88
add discussion of security implications
akhi3030 Feb 19, 2024
a57d2a5
fix lint issue
akhi3030 Feb 19, 2024
ce2a28d
add a reference implementation
akhi3030 Feb 19, 2024
3fa309a
fix some more lints
akhi3030 Feb 19, 2024
4f0bf49
Add Saketh as author
akhi3030 Feb 22, 2024
48757e4
Update neps/nep-519-yield-execution.md
akhi3030 Feb 28, 2024
60da3e1
Update neps/nep-519-yield-execution.md
akhi3030 Feb 28, 2024
48bd2a9
Update neps/nep-519-yield-execution.md
akhi3030 Feb 28, 2024
64b5fb6
Update neps/nep-519-yield-execution.md
akhi3030 Feb 28, 2024
c058614
Update neps/nep-519-yield-execution.md
akhi3030 Feb 28, 2024
7084473
Update neps/nep-519-yield-execution.md
akhi3030 Feb 28, 2024
8acb8da
specify blocks
akhi3030 Feb 28, 2024
7adce0e
specify blocks again
akhi3030 Feb 28, 2024
09cc2e0
cleanup templates that are not needed
akhi3030 Feb 28, 2024
8facd6b
remove ref impl
akhi3030 Feb 28, 2024
43fa6f3
clarify how the args are available and some formatting
akhi3030 Feb 28, 2024
cd9bc85
minor improv english
akhi3030 Feb 28, 2024
812cff3
apply suggestion from review
akhi3030 Feb 28, 2024
2728727
apply suggestion from review
akhi3030 Feb 28, 2024
577970e
Improve specification
akhi3030 Feb 28, 2024
13ebeb6
Update neps/nep-519-yield-execution.md
akhi3030 Mar 7, 2024
10920a6
some minor improvements to the comments
akhi3030 Mar 7, 2024
77bb83e
apply minor nit comment
akhi3030 Mar 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
164 changes: 164 additions & 0 deletions neps/nep-519-yield-execution.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
---
NEP: 519
Title: Yield Execution
Authors: Akhi Singhania <akhi3030@gmail.com>
Status: Draft
DiscussionsTo: https://github.com/near/NEPs/pull/519
Type: Protocol
Version: 0.0.0
Created: 2023-11-17
LastUpdated: 2023-11-20
---

## Summary

Today, when a smart contract is called by a user or another contract, it has no sensible way to delay responding to the caller. There exist use cases where contracts would benefit from being able to delay responding till some arbitrary time in the future. This proposal introduces such possibility into the NEAR protocol.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

till some arbitrary time

This feels potentially misleading to me given that we enforce a timeout. Could it be clearer to say something like "await a future transaction" rather than "delay responding"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully improved now.


## Motivation

There exist some situations where when a smart contract on NEAR is called, it will only be able to provide an answer at some arbitrary time in the future. So the callee needs a way to defer replying to the caller till this time in future.
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved

Examples include when a smart contract (`S`) provides MPC signing capabilities parties external to the NEAR protocol are computing the signature. The rough steps are:
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved
1. Signer contract provides a function `fn sign_payload(Payload, ...)`.

Check failure on line 22 in neps/nep-519-yield-execution.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Lists should be surrounded by blank lines [Context: "1. Signer contract provides a ..."]

neps/nep-519-yield-execution.md:22 MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "1. Signer contract provides a ..."]
2. When called, the contract updates some contract state which is being monitored by external indexers to indicate that a new signing request has been received. It also defers replying to the caller.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice I think the signers will directly index the transactions on the account and observe the receipts generated by yield (no need for any storage in contract state). But perhaps it is easier to explain things with this example.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully improved now. I'll add the example in a subsequent commit.

3. The indexers observe the new signing request, they compute a signature and call another function `fn signature_available(Signature, ...)` on the signer contract.
4. The signer contract validates the signature and if validate, replies to the original caller.
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved

Today, the NEAR protocol has no sensible way to defer replying to the caller in step 2 above. This proposal proposes adding two following new host functions to the NEAR protocol:

- `yield`: this can be called by a contract to indicate to the protocol that it is not ready yet to reply to its caller.
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved
- `resume`: a contract can use this mechanism to indicate to a protocol that it is now ready to reply to a caller that it had deferred earlier.
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved

If these two host functions were available, then `yield` would be used in step 2 above and `resume` would be used in step 4 above.
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved

## Specification

The proposal is to add the following host functions to the NEAR protocol:


```rust
/// Instructs the protocol that the smart contract is not ready yet to respond
/// to its caller yet. The smart contract promises to call `yield_resume()`
/// within `yield_num_blocks` blocks. When `yield_resume()` is called, the
/// protocol will call the method on the smart contract that is identified by
/// `method_name_len` and `method_name_ptr` and this method may respond to the caller.
/// Once the method has responded, `yield_resume` can no longer be called on this promise.
///
/// If the contract fails to call `yield_resume()` within `yield_num_blocks`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// If the contract fails to call `yield_resume()` within `yield_num_blocks`,
/// If the contract fails to, in a `yield_resume()`, call `value_return`, `panic` or `abort` within `yield_num_blocks`,

This makes the API much more flexible, but does it make it harder to implement?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understood the idea properly, then I think we need to think about the semantics some more. Let's say the following has happened:

  • contract A called the signer to get a signature and the signer has called yield create.
  • contract B called the signer to get a signature and the signer has called yield create.
  • now some other contract C calls the signer and the signer panics, which one of the above two outstanding executions should we terminate? I don't think we can really pick between them so the only sensible choice seems to be to terminate both. It doesn't seem like that that is a good choice either.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if contract C calls the signer and that call panics before it calls yield_create then the call from contract C should panic and none of the others.

This refers to a situation where contract D calls yield_create, the signer calls yield_resume and then the resumed call panics, ending contract D's suspended call. It shouldn't effect any other outstanding calls.

/// then the protocol will call the method on the smart contract that is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// then the protocol will call the method on the smart contract that is
/// then the protocol may call the method on the smart contract that is

I don't think this needs to be required in the protocol, rather it's an option the protocol has.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... I would've thought that this must happen at some point otherwise the smart contract might leak memory. If the contract created some state and then called yield_create which it then wanted to free in the callback, that might never happen.

Also I see this as analogous to the case where contract A calls B and B fails to respond to A. Then the protocol generates a response for A.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I agree with this, it makes life easier for contract developers if there is always a response eventually.

/// identified by `method_name_len` and `method_name_ptr` with a timeout error.
///
/// `gas_for_resumed_method` is the prepayment of Gas that will be used to
/// execute the method identified by `method_name_len` and `method_name_ptr`.
///
/// `gas_weight`: as specified in
/// [this](https://github.com/near/NEPs/blob/master/neps/nep-0264.md) NEP, this
/// improves the devX by allowing the developer to specify how to divide up the
/// remaining gas.
///
/// Return value: u64: Similar to the
/// [promise_create](https://github.com/near/nearcore/blob/a908de36ab6f75eb130447a5788007e26d05f93e/runtime/near-vm-runner/src/logic/logic.rs#L1281)
/// host function, this function also create a promise and returns an index to
/// the promise.
pub fn yield_create(
method_name_len: u64,
method_name_ptr: u64,
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved
yield_num_blocks: u64,
gas_for_resumed_method: Gas,
gas_weight: u64,
) -> u64;

/// When a smart contract has postponed replying to its caller earlier, it can
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved
/// use this function to indicate that it may now be ready to reply to it. When
/// this is called, then the protocol will call the method that the smart
/// contract referred to in the earlier `yield_create()` call.
///
/// `promise_index`: the index that was returned from the promise that created
/// from an earlier call to `yield_create()`.
///
/// `arguments_len` and `arguments_ptr`: the smart contract can provide an
/// optional list of arguments that should be passed to the method that will be
/// resumed.
pub fn yield_resume(promise_index: u64, arguments_len: u64, argument_ptr: u64) -> ();
```

## Reference Implementation

[This technical section is required for Protocol proposals but optional for other categories. A draft implementation should demonstrate a minimal implementation that assists in understanding or implementing this proposal. Explain the design in sufficient detail that:

* Its interaction with other features is clear.

Check failure on line 89 in neps/nep-519-yield-execution.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Unordered list style [Expected: dash; Actual: asterisk]

neps/nep-519-yield-execution.md:89:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]
* Where possible, include a Minimum Viable Interface subsection expressing the required behavior and types in a target programming language. (ie. traits and structs for rust, interfaces and classes for javascript, function signatures and structs for c, etc.)

Check failure on line 90 in neps/nep-519-yield-execution.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Unordered list style [Expected: dash; Actual: asterisk]

neps/nep-519-yield-execution.md:90:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]
* It is reasonably clear how the feature would be implemented.

Check failure on line 91 in neps/nep-519-yield-execution.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Unordered list style [Expected: dash; Actual: asterisk]

neps/nep-519-yield-execution.md:91:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]
* Corner cases are dissected by example.

Check failure on line 92 in neps/nep-519-yield-execution.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Unordered list style [Expected: dash; Actual: asterisk]

neps/nep-519-yield-execution.md:92:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]
* For protocol changes: A link to a draft PR on nearcore that shows how it can be integrated in the current code. It should at least solve the key technical challenges.

Check failure on line 93 in neps/nep-519-yield-execution.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Unordered list style [Expected: dash; Actual: asterisk]

neps/nep-519-yield-execution.md:93:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]

The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work.]

## Security Implications

[Explicitly outline any security concerns in relation to the NEP, and potential ways to resolve or mitigate them. At the very least, well-known relevant threats must be covered, e.g. person-in-the-middle, double-spend, XSS, CSRF, etc.]

## Alternatives

[Explain any alternative designs that were considered and the rationale for not choosing them. Why your design is superior?]

## Future possibilities

[Describe any natural extensions and evolutions to the NEP proposal, and how they would impact the project. Use this section as a tool to help fully consider all possible interactions with the project in your proposal. This is also a good place to "dump ideas"; if they are out of scope for the NEP but otherwise related. Note that having something written down in the future-possibilities section is not a reason to accept the current or a future NEP. Such notes should be in the section on motivation or rationale in this or subsequent NEPs. The section merely provides additional information.]
akhi3030 marked this conversation as resolved.
Show resolved Hide resolved

## Consequences

[This section describes the consequences, after applying the decision. All consequences should be summarized here, not just the "positive" ones. Record any concerns raised throughout the NEP discussion.]

### Positive

* p1

Check failure on line 115 in neps/nep-519-yield-execution.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Unordered list style [Expected: dash; Actual: asterisk]

neps/nep-519-yield-execution.md:115:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]

### Neutral

* n1

Check failure on line 119 in neps/nep-519-yield-execution.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Unordered list style [Expected: dash; Actual: asterisk]

neps/nep-519-yield-execution.md:119:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]

### Negative

* n1

Check failure on line 123 in neps/nep-519-yield-execution.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Unordered list style [Expected: dash; Actual: asterisk]

neps/nep-519-yield-execution.md:123:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]

### Backwards Compatibility

[All NEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. Author must explain a proposes to deal with these incompatibilities. Submissions without a sufficient backwards compatibility treatise may be rejected outright.]

## Unresolved Issues (Optional)

[Explain any issues that warrant further discussion. Considerations

* What parts of the design do you expect to resolve through the NEP process before this gets merged?

Check failure on line 133 in neps/nep-519-yield-execution.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Unordered list style [Expected: dash; Actual: asterisk]

neps/nep-519-yield-execution.md:133:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]
* What parts of the design do you expect to resolve through the implementation of this feature before stabilization?
* What related issues do you consider out of scope for this NEP that could be addressed in the future independently of the solution that comes out of this NEP?]

## Changelog

[The changelog section provides historical context for how the NEP developed over time. Initial NEP submission should start with version 1.0.0, and all subsequent NEP extensions must follow [Semantic Versioning](https://semver.org/). Every version should have the benefits and concerns raised during the review. The author does not need to fill out this section for the initial draft. Instead, the assigned reviewers (Subject Matter Experts) should create the first version during the first technical review. After the final public call, the author should then finalize the last version of the decision context.]

### 1.0.0 - Initial Version

> Placeholder for the context about when and who approved this NEP version.

#### Benefits

> List of benefits filled by the Subject Matter Experts while reviewing this version:

* Benefit 1
* Benefit 2

#### Concerns

> Template for Subject Matter Experts review for this version:
> Status: New | Ongoing | Resolved

| # | Concern | Resolution | Status |
| --: | :------ | :--------- | -----: |
| 1 | | | |
| 2 | | | |

## Copyright

Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
Loading