Add definitions of tiers-of-support for Wasmtime

[alexcrichton]

This commit adds documentation of a Tiers-based system for classifying
how supported a component is within Wasmtime. This was somewhat
pioneered in the Wasmtime 1.0 RFC but the documentation here is
expanded to include more than just API stability but additionally other
components. Inspiration for this is drawn from Rust's definition of
support tiers as well.

The motivation for this is to help clarify what exactly it means to live
at each tier and what is expected. For example one thing this document
clarifies is the requirements necessary for landing new major changes in
Wasmtime at all. Additionally this helps clarify what it means to have
the highest level of support vs "otherwise well supported".

[alexcrichton]

As-is this document is a work in progress and I would like to gather feedback from others about their thoughts on this classification. My hope is that most of the guidelines here are pretty uncontroversial, but if there are points of contention then I will close this and reopen it as an RFC for a more official venue of discussion.

I also plan on talking about this in the upcoming days at the Cranelift and Wasmtime weekly meetings to cast a wide net for getting interest in this.

[abrown]

I think wasi-nn could arguably be in tier 2.

[alexcrichton]

Given the current list of requirements sketched out I believe that wasi-nn is still largely missing tests. There's one example that is compiled and run but I think that we'd want more of a unit-test-like-suite otherwise.

At least personally this sort of tracks with my gut feeling about wasi-nn and wasi-crypto where they're present but don't feel all-that-well integrated/first-class. WASI itself isn't really all that first-class by requiring an extra wasmtime-wasi crate which hasn't had much time put into its API design for stability too, though. I don't know how best to think about WASI proposals myself.

[jameysharp]

I think this sounds great; I just have a few clarifying edits.

[jameysharp]

I'm happy with this, and I feel it's useful to have these criteria written down.

[cfallin]

This is a great overall idea, as I mentioned in the Wasmtime biweekly -- thanks for putting this together @alexcrichton!

I had a few additional thoughts on the sorts of things that could be tiered, both related to Cranelift. Basically any large feature or piece of functionality could in theory be assigned a tier, and two that come to mind are:

• Support for Cranelift features not exercised by Wasmtime. While we want to maintain wide support as much as possible, in practice the parts of Cranelift that are exercised by Wasmtime get a lot more testing and are more solid as a result. The set of features/instructions/types used by cranelift-wasm is also significantly smaller than the whole theoretical range of programs that can be expressed in CLIF. At least in the following dimensions we can make this distinction:

  • Instructions (CLIF opcodes);
  • Value types (I32/I64/F32/F64/128-bit vectors, vs. everything else -- narrow and wide types, bools, etc.);
  • Other calling conventions (e.g., SpiderMonkey/Baldrdash);
  • Other "code model" settings, such as relocation restrictions, etc;
  • Other Cranelift settings than what is configured by Wasmtime in general.

  I would argue that the above (i.e., non-Wasmtime uses of Cranelift) should be listed as tier 3, in order to capture the de-facto situation we have today. To graduate from tier 3, we would need someone working on Cranelift fulltime whose focus is these other use-cases.

• I also think that debug support is already de-facto tier 3, and should be listed as such. Partly this is because no one currently on the project fully understands the code, and partly this is because there are some lingering issues or shortcomings. We've recently seen debug support as sort of best-effort, in the sense that we try to keep it working for the tests (I put in a few weeks of effort specifically on regalloc2 and the new-backend migrations for this) but we don't have the developer time to make and support strong guarantees. To graduate from tier 3, we would need someone who deeply understands DWARF and debug infrastructure to overhaul the code, improve its robustness, then continue to support it.

And finally to write down something I said at the meeting here: I think it would be really useful to list "what is needed to graduate" alongside features/platforms at lower tiers. This serves both to capture some rationale, and to motivate: if I (as a potential contributor) see that all a given feature needs to be tier-1 and on by default is some more fuzzing, and I want that feature, maybe I will jump in and help. I wrote down some "to graduate" conditions above as examples. In particular I think some of the items in your list are pretty close: e.g. aarch64 really just needs fuzzing on real hardware to be tier-1 supported, if I understand correctly.

[alexcrichton]

Discussion of this in the Wasmtime/Cranelift meetings felt positive about the various bits here, so I'd like to give some more time in case anyone has feedback, otherwise I will merge later this week.

[alexcrichton]

Ok there hasn't been any further discussion of this so I'm going to queue up its merge. PRs and tweaks over time are of course welcome though!