Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement wasip3 interfaces #1

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

implement wasip3 interfaces #1

wants to merge 4 commits into from

Conversation

rvolosatovs
Copy link
Member

@rvolosatovs rvolosatovs commented Feb 3, 2025

@rvolosatovs rvolosatovs force-pushed the feat/wasip3 branch 3 times, most recently from 251c2a1 to b766e9a Compare February 3, 2025 18:17
rvolosatovs pushed a commit that referenced this pull request Feb 3, 2025
This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix clippy warnings and bench/fuzzing errors

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

revert atomic.wit whitespace change

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix build when component-model disabled

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

bless component-macro expected output

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix no-std build error

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix build with --no-default-features --features runtime,component-model

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

partly fix no-std build

It's still broken due to the use of `std::collections::HashMap` in
crates/wasmtime/src/runtime/vm/component.rs.  I'll address that as part of the
work to avoid exposing global task/future/stream/error-context handles to
guests.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

maintain per-instance tables for futures, streams, and error-contexts

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

refactor task/stream/future handle lifting/lowering

This addresses a couple of issues:

- Previously, we were passing task/stream/future/error-context reps directly to
  instances while keeping track of which instance had access to which rep.  That
  worked fine in that there was no way to forge access to inaccessible reps, but
  it leaked information about what other instances were doing.  Now we maintain
  per-instance waitable and error-context tables which map the reps to and from
  the handles which the instance sees.

- The `no_std` build was broken due to use of `HashMap` in
  `runtime::vm::component`, which is now fixed.

Note that we use one single table per instance for all tasks, streams, and
futures.  This is partly necessary because, when async events are delivered to
the guest, it wouldn't have enough context to know which stream or future we're
talking about if each unique stream and future type had its own table.  So at
minimum, we need to use the same table for all streams (regardless of payload
type), and likewise for futures.  Also, per
WebAssembly/component-model#395 (comment),
the plan is to move towards a shared table for all resource types as well, so
this moves us in that direction.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix wave breakage due to new stream/future/error-context types

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

switch wasm-tools to v1.220.0-based branch

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

check `task.return` type at runtime

We can't statically verify a given call to `task.return` corresponds to the
expected core signature appropriate for the currently running task, so we must
do so at runtime.  In order to make that check efficient, we intern the types.

My initial plan was to use `ModuleInternedTypeIndex` and/or `VMSharedTypeIndex`
for interning, but that got hairy with WasmGC considerations, so instead I added
new fields to `ComponentTypes` and `ComponentTypesBuilder`.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add `TypedFunc::call_concurrent` and refine stream/future APIs

This implements what I proposed in https://github.com/dicej/rfcs/blob/component-async/accepted/component-model-async.md#wasmtime.  Specifically, it adds:

- A new `Promise` type, useful for working with concurrent operations that require access to a `Store` to make progress.
- A new `PromisesUnordered` type for `await`ing multiple promises concurrently
-`TypedFunc::call_concurrent` (which returns a `Promise`), allowing multiple host->guest calls to run concurrently on the same instance.
- Updated `{Stream|Future}{Writer|Reader}` APIs which use `Promise`

The upshot is that the embedder can now ergonomically manage arbitrary numbers
of concurrent operations.  Previously, this was a lot more difficult to do
without accidentally starving some of the operations due to another one
monopolizing the `Store`.

Finally, this includes various refactorings and fixes for bugs exposed by the
newer, more versatile APIs.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

clean up verbosity in component/func.rs

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

snapshot

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

implement stream/future read/write cancellation

This required a somewhat viral addition of `Send` and `Sync` bounds for async
host function closure types, unfortunately.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add `Func::call_concurrent` and `LinkerInstance::func_new_concurrent`

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

dynamic API support for streams/futures/error-contexts

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix `call_host` regression

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add component model async end-to-end tests

I've ported these over from https://github.com/dicej/component-async-demo

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix test regressions and clippy warnings

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

satisfy clippy

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix async tests when `component-model-async` enabled

Enabling this feature for all tests revealed various missing pieces in the new
`concurrent.rs` fiber mechanism, which I've addressed.

This adds a bunch of ugly `#[cfg(feature = "component-model-async")]` guards,
but those will all go away once I unify the two async fiber implementations.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add and modify tests to cover concurrent APIs

Primarily, this tests and implements cases where parameters and/or results must
be passed via linear memory instead of the stack.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

`concurrent_{imports|exports}` component macro codegen tests

This enables codegen testing of the `concurrent_imports` and
`concurrent_exports` options to `wasmtime::component::bindgen` and also fixes
code generation for world-level function and resource exports that use the
concurrent call style.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

`concurrent_{imports|exports}` component macro expanded tests

This enables testing of the `concurrent_imports` and `concurrent_exports`
options in `crates/component-macro/tests/expanded.rs`.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add tests/misc_testsuite/component-model-async/*.wast

These only test instantiation of components which use various async options and
built-ins so far.  Next, I'll happy and sad path tests which actually execute
code.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

appease clippy

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add tests/misc_testsuite/component-model-async/fused.wast

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add non-panicking bounds checks where appropriate

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

remove post-return bits from async result lift code

...at least until we've determined whether post-return options even make sense
for async-lifted exports.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix component-model-async/fused.wast test failure

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

use `enum` types to represent status and event codes

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix component-model-async/fused.wast test failure (2nd try)

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

use `gc_types = true` in component-model-async/fused.wast

We use `Instruction::RefFunc` when generating adapters for async lifts and/or
lowers, which Winch doesn't understand, and apparently `gc_types = true` is what
tells the test infra not to use Winch.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

trap if async function finishes without calling `task.return`

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

update wit-bindgen and fix rebase damage

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

call post-return function if any for async->sync fused calls

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix non-component-model-async build; appease clippy

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

bless bindgen output whitespace changes

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

enforce resource borrow requirements for async calls

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

update `wit-bindgen` and simplify `async_borrowing_callee` test

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

call `InstanceFlags::set_may_enter` where appropriate

There's still more work to do to fully implement (and test) the reentrance rules
for concurrent tasks, but this is a start.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

finish implementing reentrance checks

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

feat: implement error-context (#1)

* feat: initial error-context implementation

This commit implements error-context related functions inside the VM,
along with tests to ensure that basic error-context.new and
error-context.debug-message functionality works.

Signed-off-by: Victor Adossi <vadossi@cosmonic.com>

* wip: add test for error context callee/caller transfer

Signed-off-by: Victor Adossi <vadossi@cosmonic.com>

* wip: test for async context transfer

Signed-off-by: Victor Adossi <vadossi@cosmonic.com>

---------

Signed-off-by: Victor Adossi <vadossi@cosmonic.com>

run cargo fmt

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

appease clippy

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

pull in Roman's unit stream work; add world-level export test

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add unit stream tests

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

CI fixes

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix non-component-model-async build

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add round-trip tests with many parameters and results

This adds tests to cover round-tripping values whose flattened form exceed
`MAX_FLAT_PARAMS`.  Although we already had tests to cover this in
tests/all/component_model/func.rs, those tests did not involve any flavor of
composition; these do.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

test: add multiple send as initial context stream test

Signed-off-by: Victor Adossi <vadossi@cosmonic.com>

fix Victor's test case

Turns out this was the first test case where the host calls a component via an
async-lifted export, which in turn calls another component's async-lifted export
via a sync-lowered import, which didn't work.  Now it does!

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

refactor sync->async component call adapters

We no longer use global variables to stash parameters and results due to the
hazard of concurrent calls clobbering them.  Instead, we now stash them in the
`GuestTask` object for the task.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>
@rvolosatovs rvolosatovs force-pushed the feat/wasip3 branch 2 times, most recently from 4bf3a15 to 83ae160 Compare February 3, 2025 18:21
dicej added a commit that referenced this pull request Feb 4, 2025
This adds support for loading, compiling, linking, and running components which
use the [Async
ABI](https://github.com/WebAssembly/component-model/blob/main/design/mvp/Async.md)
along with the [`stream`, `future`, and
`error-context`](WebAssembly/component-model#405) types.
It also adds support for generating host bindings such that multiple host
functions can be run concurrently with guest tasks -- without monopolizing the
`Store`.

See the [implementation RFC](bytecodealliance/rfcs#38)
for details, as well as [this
repo](https://github.com/dicej/component-async-demo) containing end-to-end smoke
tests.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix clippy warnings and bench/fuzzing errors

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

revert atomic.wit whitespace change

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix build when component-model disabled

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

bless component-macro expected output

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix no-std build error

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix build with --no-default-features --features runtime,component-model

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

partly fix no-std build

It's still broken due to the use of `std::collections::HashMap` in
crates/wasmtime/src/runtime/vm/component.rs.  I'll address that as part of the
work to avoid exposing global task/future/stream/error-context handles to
guests.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

maintain per-instance tables for futures, streams, and error-contexts

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

refactor task/stream/future handle lifting/lowering

This addresses a couple of issues:

- Previously, we were passing task/stream/future/error-context reps directly to
  instances while keeping track of which instance had access to which rep.  That
  worked fine in that there was no way to forge access to inaccessible reps, but
  it leaked information about what other instances were doing.  Now we maintain
  per-instance waitable and error-context tables which map the reps to and from
  the handles which the instance sees.

- The `no_std` build was broken due to use of `HashMap` in
  `runtime::vm::component`, which is now fixed.

Note that we use one single table per instance for all tasks, streams, and
futures.  This is partly necessary because, when async events are delivered to
the guest, it wouldn't have enough context to know which stream or future we're
talking about if each unique stream and future type had its own table.  So at
minimum, we need to use the same table for all streams (regardless of payload
type), and likewise for futures.  Also, per
WebAssembly/component-model#395 (comment),
the plan is to move towards a shared table for all resource types as well, so
this moves us in that direction.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix wave breakage due to new stream/future/error-context types

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

switch wasm-tools to v1.220.0-based branch

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

check `task.return` type at runtime

We can't statically verify a given call to `task.return` corresponds to the
expected core signature appropriate for the currently running task, so we must
do so at runtime.  In order to make that check efficient, we intern the types.

My initial plan was to use `ModuleInternedTypeIndex` and/or `VMSharedTypeIndex`
for interning, but that got hairy with WasmGC considerations, so instead I added
new fields to `ComponentTypes` and `ComponentTypesBuilder`.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add `TypedFunc::call_concurrent` and refine stream/future APIs

This implements what I proposed in https://github.com/dicej/rfcs/blob/component-async/accepted/component-model-async.md#wasmtime.  Specifically, it adds:

- A new `Promise` type, useful for working with concurrent operations that require access to a `Store` to make progress.
- A new `PromisesUnordered` type for `await`ing multiple promises concurrently
-`TypedFunc::call_concurrent` (which returns a `Promise`), allowing multiple host->guest calls to run concurrently on the same instance.
- Updated `{Stream|Future}{Writer|Reader}` APIs which use `Promise`

The upshot is that the embedder can now ergonomically manage arbitrary numbers
of concurrent operations.  Previously, this was a lot more difficult to do
without accidentally starving some of the operations due to another one
monopolizing the `Store`.

Finally, this includes various refactorings and fixes for bugs exposed by the
newer, more versatile APIs.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

clean up verbosity in component/func.rs

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

snapshot

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

implement stream/future read/write cancellation

This required a somewhat viral addition of `Send` and `Sync` bounds for async
host function closure types, unfortunately.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add `Func::call_concurrent` and `LinkerInstance::func_new_concurrent`

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

dynamic API support for streams/futures/error-contexts

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

support callback-less (AKA stackful) async lifts

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix `call_host` regression

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add component model async end-to-end tests

I've ported these over from https://github.com/dicej/component-async-demo

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix test regressions and clippy warnings

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

satisfy clippy

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix async tests when `component-model-async` enabled

Enabling this feature for all tests revealed various missing pieces in the new
`concurrent.rs` fiber mechanism, which I've addressed.

This adds a bunch of ugly `#[cfg(feature = "component-model-async")]` guards,
but those will all go away once I unify the two async fiber implementations.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add and modify tests to cover concurrent APIs

Primarily, this tests and implements cases where parameters and/or results must
be passed via linear memory instead of the stack.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

`concurrent_{imports|exports}` component macro codegen tests

This enables codegen testing of the `concurrent_imports` and
`concurrent_exports` options to `wasmtime::component::bindgen` and also fixes
code generation for world-level function and resource exports that use the
concurrent call style.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

`concurrent_{imports|exports}` component macro expanded tests

This enables testing of the `concurrent_imports` and `concurrent_exports`
options in `crates/component-macro/tests/expanded.rs`.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add tests/misc_testsuite/component-model-async/*.wast

These only test instantiation of components which use various async options and
built-ins so far.  Next, I'll happy and sad path tests which actually execute
code.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

appease clippy

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add tests/misc_testsuite/component-model-async/fused.wast

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add non-panicking bounds checks where appropriate

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

remove post-return bits from async result lift code

...at least until we've determined whether post-return options even make sense
for async-lifted exports.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix component-model-async/fused.wast test failure

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

use `enum` types to represent status and event codes

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix component-model-async/fused.wast test failure (2nd try)

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

use `gc_types = true` in component-model-async/fused.wast

We use `Instruction::RefFunc` when generating adapters for async lifts and/or
lowers, which Winch doesn't understand, and apparently `gc_types = true` is what
tells the test infra not to use Winch.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

trap if async function finishes without calling `task.return`

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

update wit-bindgen and fix rebase damage

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

call post-return function if any for async->sync fused calls

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix non-component-model-async build; appease clippy

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

bless bindgen output whitespace changes

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

enforce resource borrow requirements for async calls

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

update `wit-bindgen` and simplify `async_borrowing_callee` test

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

call `InstanceFlags::set_may_enter` where appropriate

There's still more work to do to fully implement (and test) the reentrance rules
for concurrent tasks, but this is a start.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

finish implementing reentrance checks

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

feat: implement error-context (#1)

* feat: initial error-context implementation

This commit implements error-context related functions inside the VM,
along with tests to ensure that basic error-context.new and
error-context.debug-message functionality works.

Signed-off-by: Victor Adossi <vadossi@cosmonic.com>

* wip: add test for error context callee/caller transfer

Signed-off-by: Victor Adossi <vadossi@cosmonic.com>

* wip: test for async context transfer

Signed-off-by: Victor Adossi <vadossi@cosmonic.com>

---------

Signed-off-by: Victor Adossi <vadossi@cosmonic.com>

run cargo fmt

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

appease clippy

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

pull in Roman's unit stream work; add world-level export test

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add unit stream tests

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

CI fixes

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix non-component-model-async build

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

add round-trip tests with many parameters and results

This adds tests to cover round-tripping values whose flattened form exceed
`MAX_FLAT_PARAMS`.  Although we already had tests to cover this in
tests/all/component_model/func.rs, those tests did not involve any flavor of
composition; these do.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

test: add multiple send as initial context stream test

Signed-off-by: Victor Adossi <vadossi@cosmonic.com>

fix Victor's test case

Turns out this was the first test case where the host calls a component via an
async-lifted export, which in turn calls another component's async-lifted export
via a sync-lowered import, which didn't work.  Now it does!

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

refactor sync->async component call adapters

We no longer use global variables to stash parameters and results due to the
hazard of concurrent calls clobbering them.  Instead, we now stash them in the
`GuestTask` object for the task.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>
@dicej
Copy link
Collaborator

dicej commented Feb 4, 2025

FYI, I just pushed my async branch to main in this repo, so I think we can delete the former and switch this PR to point to the latter.

@rvolosatovs rvolosatovs force-pushed the feat/wasip3 branch 2 times, most recently from 78fc752 to 5c4f359 Compare February 5, 2025 14:53
@rvolosatovs rvolosatovs changed the base branch from async to main February 5, 2025 14:53
@rvolosatovs rvolosatovs force-pushed the feat/wasip3 branch 2 times, most recently from aa348fb to a453fee Compare February 5, 2025 18:21
@rvolosatovs rvolosatovs force-pushed the feat/wasip3 branch 3 times, most recently from 034cee9 to 43f2639 Compare February 11, 2025 17:00
dicej and others added 3 commits February 11, 2025 18:32
I had a few goals with this PR:

  1. Improve the ergonomics of concurrent import bindings by supporting
  `async`/`await` sugar and allowing store access to be arbitrarily interspersed
  between `await` points -- while still preventing references to the store
  across `await` points.

  2. Get rid of the `Data` associated types for `Host` traits and support
  `add_to_linker_get_host` where the `Host` impl is not the same as the `T` in
  `Store<T>`.

  3. Allow creating, reading from, writing to, etc. streams and futures without
  exposing `StoreContextMut` directly.

Unfortunately, after a day of intense type tetris I failed to achieve items 2 or
3, so this only covers item 1.

Regarding item 1: I've introduced a new `Accessor` type which wraps a `*mut dyn
VMStore` and provides access to it only via a `with` method that accepts a
synchronous closure which takes a `StoreContextMut<T>` parameter.  The closure
can do what it likes and return an arbitrary value as long as that result has a
`'static` lifetime (i.e. does not borrow from the store).  This ensures that the
host function is able to access the store only between `await`s and not across
them; we prohibit the latter because it would prevent other async-lowered
imports from running concurrently.  Finally, since host function takes a `&mut
Accessor<T>`, it is not possible for the reference to outlive the future
returned by the host function, and since the `with` method takes `&mut self` it
cannot be used recursively.

Regarding items 2 and 3: In order to read from or write to streams/futures, we
need to be able to efficiently lift and lower their payload types, which
requires that both the payload type (of which there could be several for a given
world) and the `T` in `Store<T>` be in scope.  I was unable to find a way to
thread those types through the various layers of closures, futures, and
generated code without adding unwanted `'static` bounds and/or breaking the
blanket `impl`s used for forwarding calls from `&mut X` to `X`.  Also, the usual
tricks of using dyn objects or vtables could only be used ergonomically to erase
one of the two types but not both.  I'd love to revisit this later with the help
of a Rust type wizard to see if there's a strategy I missed.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

2 participants