Gate many CLI/Wasmtime features behind Cargo features

[alexcrichton]

This PR is borne out of discussions from last week's CG meetings about the suitability of Wasmtime in embedded scenarios. One of the primary concerns that arose was the binary size of the Wasmtime engine being too large for these scenarios. In my experience binary size is one aspect of Rust where it doesn't come for free so this aligns with my expectations. That being said I've yet to see a case in Rust where a particular binary footprint couldn't be achieved, hence this PR.

One thing I've noticed is that many folks evaluating Wasmtime seem to look at the size of the wasmtime executable itself when performing size comparisons. While this likely doesn't reflect the true size of what a custom embedding would be it's nevertheless an easy comparison that can be made. To that effect I've added a number of Cargo features to the wasmtime-cli crate to disable functionality up to and including wasmtime compile for example. This PR enables building a wasmtime exectuable that is stripped down to the bare bones - all it can do is run precompiled WebAssembly files. That being said it should still be a full-featured runtime at this time where only some optional features such as DWARF processing for addr2line are disabled.

Additionally in this PR I've added a new chapter to the Wasmtime book about building a minimal build of Wasmtime. This explains not only the existence of the --no-default-features build flag but additionally the consequences in terms of functionality and binary size. Furthermore this section additionally directly expands on tips and tricks for building a minimal binary using various Rust compiler flags and nightly features. This section shows how the wasmtime CLI is 130M in debug mode and the smallest build producable after this PR is 2.1M.

The goal of this PR is to get the ball rolling on size-related optimizations, not necessarily be the end-all-be-all. Further optimizations to binary size will likely require more invasive changes about how we develop Wasmtime or similar. For example we may have to lose the relatively rich error information Wasmtime currently has to get the next level of significant win. That being said I'm at least personally lacking a bit in direction for where to go next to optimize. My hope is that by showcasing a much smaller wasmtime CLI executable it shows to folks that Wasmtime for these use cases is viable, even if it's not suitable just yet. For example the CLI uses crates such as clap which won't be present in custom embeddings. Having an actual custom embedding to optimize for will be useful. In lieu of this I plan on continuing to poke to see what we can do.

One option as a result of this PR is to build a wasmtime-lite executable on CI and upload it to GitHub Actions/Releases as well. I'm not totally convinced that'd be too useful so I haven't done that. Another option though is to add a CI builder which builds a minimal binary and verifies that it doesn't go above a particular threshold to catch regressions in binary size (e.g. if a new feature is added ensure that it can be disabled with --no-default-features). I've left this for a future PR as well.

I should also note that my hope is that when optimizing for binary size it would not hinder the development of Wasmtime anywhere else. In that sense it should ideally be easy for anyone to understand how the conditionally compiled code works and it shouldn't appear as a maze of cfgs to wade through to get things working. While it's definitely more #[cfg] than before, I'm particularly interested if folks feel this is too onerous to maintain or burdensome. If so I'd be interested in improving how this is all designed.

[fitzgen]

Looks great to me, the new page in the guide looks great as well.

[bjorn3]

Stripping debuginfo using -Cstrip=debuginfo should also help a fair bit, right?

[alexcrichton]

Oh excellent suggestion I forgot about that! Turns out the majority of the win from -Zbuild-std was stripping debuginfo

[alexcrichton]

I should highlight this as a drive-by change by me (cc @cfallin), but this'll rename -C enable-pcc to -C pcc (most other options omit the "enable" or "disable" to have everything be "if you specify it it's enabled unless you say =n in which case you're asking to disable it")

[github-actions]

Subscribe to Label Action

cc @peterhuene

This issue or pull request has been labeled: "wasmtime:api", "wasmtime:c-api", "wasmtime:config", "wasmtime:docs"

Thus the following users have been cc'd because of the following labels:

• peterhuene: wasmtime:api, wasmtime:c-api

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

[github-actions]

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

• If you added a new Config method, you wrote extensive documentation for
  it.

  Our documentation should be of the following form:

  Short, simple summary sentence.
  
  More details. These details can be multiple paragraphs. There should be
  information about not just the method, but its parameters and results as
  well.
  
  Is this method fallible? If so, when can it return an error?
  
  Can this method panic? If so, when does it panic?
  
  # Example
  
  Optional example here.
  

• If you added a new Config method, or modified an existing one, you
  ensured that this configuration is exercised by the fuzz targets.

  For example, if you expose a new strategy for allocating the next instance
  slot inside the pooling allocator, you should ensure that at least one of our
  fuzz targets exercises that new strategy.

  Often, all that is required of you is to ensure that there is a knob for this
  configuration option in wasmtime_fuzzing::Config (or one
  of its nested structs).

  Rarely, this may require authoring a new fuzz target to specifically test this
  configuration. See our docs on fuzzing for more details.

• If you are enabling a configuration option by default, make sure that it
  has been fuzzed for at least two weeks before turning it on by default.

---

To modify this label's message, edit the .github/label-messager/wasmtime-config.md file.

To add new label messages or remove existing label messages, edit the
.github/label-messager.json configuration file.

Learn more.

[abrown]

Nice work documenting the pruning! I've always sort of wondered what the minimum size would be.

[kayabaNerve]

This merged jitdump and vtune into a new, single feature profiling, adding the ittapi dependency to anyone who solely wants jitdump. It also didn't properly update documentation as https://docs.rs/wasmtime/latest/wasmtime/ still details a jitdump/vtune feature (with no profiling feature).

While I'm not against the existence of a new profiling feature, could the original two features be restored with profiling activating both of them?

[alexcrichton]

Ah I merged those since they were both on-by-default and there didn't at the time appear to be much benefit to sharding further. Out of curiosity is this something where you're concerned about binary size of the ittapi dependency? Or is it otherwise not building for you for one reason or another?

One reason I unified them was I also placed the guest profiler behind the same profiling feature and by having a complete on/off switch it would enable conditionally compiling out even more code as well in theory too. I was hoping to avoid a feature-per-profiler, so if possible I'd prefer to solve the problem without splitting it back up, but that depends on the issue that you're having and if it can only be solved by adding a vtune feature back that seems ok too

[kayabaNerve]

As someone who refers to an upstream which disables default features, then re-enables features as needed, this did introduce the ittapi dependency for me as I updated it in my downstream. To be clear, I agree with that policy.

I don't personally care about ittapi's effect on binary size. I care about the fact I now have ittapi in scope and will have to, eventually, read through and audit ittapi to ensure it's non-malicious (or import the Bytecode Alliance's vet statement for it, which I can assume exists). While I can recognize the library is hosted and maintained by Intel, the actual publishers of the crate are three individuals, any of who could be compromised.

TL;DR It increases my supply chain and complicates the package graph for no benefit to me, making this a downgrade in wasmtime's utility.

Again. I don't mind the profiling feature. I just wish I could still access the original jitdump feature. Accordingly, I believe this can be implemented without hurting the performance/effects/code-gen of the profiling feature.

[alexcrichton]

That makes sense yeah, thanks for explaining! If it's ok with you I'd prefer to stay the current course, however. Wasmtime doesn't guarantee that we won't add new dependencies with new version of Wasmtime even with default features disabled. That being said we do guarantee that all dependencies used by Wasmtime have a vetted version by BA maintainers.

Otherwise I'd personally prefer to avoid sharding the profiling feature into a feature-per-strategy as well as that feels like a bit of an overload of Cargo features (we already have a lot). But again I don't feel super strongly about this, just a slight preference.

[kayabaNerve]

I fully understand dependencies may increase. That does not affect my commentary on needless dependencies having increased.

As for staying the course, I can respect your preference of simplicity. If in the future, you change your mind and would accept a PR re-defining profiling as a superset of both features, please let me know.