Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement component model resources in Wasmtime #6691

Merged
merged 49 commits into from
Jul 22, 2023

Conversation

alexcrichton
Copy link
Member

@alexcrichton alexcrichton commented Jul 5, 2023

This PR is an implementation of the resource datatype in the component model as specified upstream. The goal of this PR is to get lots of the low-level infrastructure for resources sorted out, but not 100% of the story as there's still remaining work. Features implemented in this PR are:

  • Validation, translation, and compilation of resources now works
  • Implementation of all resource-related intrinsics such as resource.{rep,new,drop}
  • Updates to the compiled image of components as necessary for intrinsics
  • Implementation of runtime state of resources, aka tables, in component instances
  • Updates to type-checking and type-matching to take resources into account
  • Implementation of an embedding API for host-defined resources
  • Implementation of an embedding API for guest-defined resources
  • An initial suite of test which showcase basic behavior of resources and the capabilities of the embedding API

This is intended to be a solid implementation for all the "internals" of resources throughout Wasmtime. It's expected that all further work will be much easier, less invasive, and not so large a scale. Or at least that's the hope. Note though that this is not a 100% complete story for resources in Wasmtime. For example it's still not possible to take a WIT off the shelf with resources and use that with Wasmtime. Key missing features in Wasmtime related to resources are:

This work is a prerequisite for the above, though, and the hope additionally is that the above can all start to progress in parallel with this work as a basis. Basically getting enough in that everything is no longer bottlenecked on me but it's possible to start building out from here.

Closes #6583

@github-actions github-actions bot added cranelift Issues related to the Cranelift code generator cranelift:wasm wasmtime:api Related to the API of the `wasmtime` crate itself labels Jul 5, 2023
@github-actions
Copy link

github-actions bot commented Jul 5, 2023

Subscribe to Label Action

cc @peterhuene

This issue or pull request has been labeled: "cranelift", "cranelift:wasm", "wasmtime:api"

Thus the following users have been cc'd because of the following labels:

  • peterhuene: wasmtime:api

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

This commit fixes a minor issue in
`FunctionIndices::link_and_append_code` which previously ended up only
filling out the `wasm_to_native_trampolines` field for the first module
rather than all the modules. Additionally the first module might have
too many entries that encompass all modules instead of just its own
entries. The fix in this commit is to refactor this logic to ensure that
the necessary maps are present for all translations.

While technically a bug that can be surfaced through the embedder API
it's pretty obscure. The given test here panics beforehand but succeeds
afterwards, but this is moreso prep for some future resource-related
work where this map will need persisting into the component metadata
side of things.
Lots of bits and pieces squashed into this commit. Much to be done
still.
Also add a test which requires host-defined drop to be called which
isn't working.
No need to check for a null funcref when we already know ahead of time
if it's ever going to be null or not.
Plumb around dynamic information about resource types.
Implemented both in the raw wasm intrinsic as well as the host.
@alexcrichton
Copy link
Member Author

Ok I think this amounts for everything except (borrow $T), fact, bindgen!, and documentation. Otherwise I think all the bits and pieces are there, or at least in basic form. I'm going to spend tomorrow trying to figure out how hard borrow will be and then focus next week on docs, tests, and polish.

crates/cranelift/src/compiler/component.rs Outdated Show resolved Hide resolved
crates/wasmtime/src/component/component.rs Outdated Show resolved Hide resolved
crates/cranelift-shared/src/lib.rs Outdated Show resolved Hide resolved
@github-actions github-actions bot added the fuzzing Issues related to our fuzzing infrastructure label Jul 10, 2023
@alexcrichton
Copy link
Member Author

Thanks again for reviewing this @fitzgen, I know it was a big ask!

I'm gonna go ahead and queue this up for merging, and leave the newtype index for libcalls as a follow-up if that's ok, but otherwise I think I've addressed the other feedback you had

@alexcrichton alexcrichton enabled auto-merge July 21, 2023 19:54
@alexcrichton alexcrichton added this pull request to the merge queue Jul 21, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 21, 2023
@alexcrichton alexcrichton enabled auto-merge July 21, 2023 20:28
@alexcrichton alexcrichton added this pull request to the merge queue Jul 21, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 21, 2023
@alexcrichton alexcrichton enabled auto-merge July 21, 2023 20:58
@alexcrichton alexcrichton added this pull request to the merge queue Jul 21, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 21, 2023
They all involve compilation which takes too long and doesn't currently
work
@alexcrichton alexcrichton enabled auto-merge July 21, 2023 21:59
@alexcrichton alexcrichton added this pull request to the merge queue Jul 21, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 21, 2023
@alexcrichton alexcrichton added this pull request to the merge queue Jul 21, 2023
Merged via the queue into bytecodealliance:main with commit 5a6ed0f Jul 22, 2023
@alexcrichton alexcrichton deleted the resources branch July 22, 2023 00:06
geekbeast pushed a commit to geekbeast/wasmtime that referenced this pull request Aug 1, 2023
* main: (47 commits)
  Add core dump support to the runtime (bytecodealliance#6513)
  Resource table tracks child relationships (bytecodealliance#6779)
  Wasmtime: Move `OnDemandInstanceAllocator` to its own module (bytecodealliance#6790)
  wasi: Test the stdio streams implementation (bytecodealliance#6764)
  Don't generate same-named imports in fact modules (bytecodealliance#6783)
  Wasmtime: Add support for Wasm tail calls (bytecodealliance#6774)
  Cranelift: Fix `ABIMachineSpec::gen_add_imm` for riscv64 (bytecodealliance#6780)
  Update the wasm-tools family of crates, disallow empty component types (bytecodealliance#6777)
  Fix broken link to WASI API documentation (bytecodealliance#6775)
  A bunch of cleanups for cranelift-codegen-meta (bytecodealliance#6772)
  Implement component-to-component calls with resources (bytecodealliance#6769)
  Ignore async_stack_size if async_support is disabled (bytecodealliance#6771)
  A bunch of minor cleanups (bytecodealliance#6767)
  Fix flaky tests in preview2 streams (bytecodealliance#6763)
  Refactor and simplify component trampolines (bytecodealliance#6751)
  Cranelift: Implement tail calls on riscv64 (bytecodealliance#6749)
  WASI Preview 2: rewrite streams and pollable implementation (bytecodealliance#6556)
  cranelift-wasm: Add support for translating Wasm tail calls to CLIF (bytecodealliance#6760)
  Cranelift: Get tail calls working on aarch64 (bytecodealliance#6723)
  Implement component model resources in Wasmtime (bytecodealliance#6691)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cranelift:wasm cranelift Issues related to the Cranelift code generator fuzzing Issues related to our fuzzing infrastructure wasmtime:api Related to the API of the `wasmtime` crate itself
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement resources for the component model
3 participants