wasmtime serve seems to ignore --dir option, directory is not preopened

[javorszky]

We've been trying to create a wasi:http world file to have access to the filesystem. We couldn't get it working.

Test Case

I've put up a repository here: https://github.com/javorszky/wasmtime-fscheck-serve which has the source code, a dockerfile, a makefile to make sure everything is consistent across different machines and host triplets.

Steps to Reproduce

• clone the repository
• build the docker container with make bd
• run the docker container with make dr
• send an empty get request to localhost:8080

Expected Results

I expect the wasm component to respond with a 200 OK with the body being the list of files and directories in the opened directory.

Actual Results

Response is a panicked crash with the following error log in docker:

stderr [3] :: thread '<unnamed>' panicked at src/lib.rs:28:53:
stderr [3] :: called `Result::unwrap()` on an `Err` value: Custom { kind: Uncategorized, error: "failed to find a pre-opened file descriptor through which \"/shenanigans\" could be opened" }
stderr [3] :: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2024-09-03T08:48:29.977839Z ERROR wasmtime_cli::commands::serve: [3] :: Error {
    context: "error while executing at wasm backtrace:\n    0: 0x258f2 - wasmtime_filesystem_debug.wasm!__rust_start_panic\n    1: 0x2546e - wasmtime_filesystem_debug.wasm!rust_panic\n    2: 0x25224 - wasmtime_filesystem_debug.wasm!std::panicking::rust_panic_with_hook::h8bb4f94cf9e68e20\n    3: 0x24646 - wasmtime_filesystem_debug.wasm!std::panicking::begin_panic_handler::{{closure}}::h2c354a3372fcca00\n    4: 0x245ad - wasmtime_filesystem_debug.wasm!std::sys_common::backtrace::__rust_end_short_backtrace::hf791a0fba63af5e9\n    5: 0x24dbc - wasmtime_filesystem_debug.wasm!rust_begin_unwind\n    6: 0x2a66b - wasmtime_filesystem_debug.wasm!core::panicking::panic_fmt::haadcabb094f4c85b\n    7: 0x2c2de - wasmtime_filesystem_debug.wasm!core::result::unwrap_failed::hd09ff05c9c2fb25f\n    8: 0x6a1d - wasmtime_filesystem_debug.wasm!<wasmtime_filesystem_debug::Component as wasi::proxy::exports::wasi::http::incoming_handler::Guest>::handle::hd44cd139d21c109c\n    9: 0x7853 - wasmtime_filesystem_debug.wasm!wasi::proxy::exports::wasi::http::incoming_handler::_export_handle_cabi::he753d69b8503d63c\n   10: 0x7798 - wasmtime_filesystem_debug.wasm!wasi:http/incoming-handler@0.2.1#handle\nnote: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information",
    source: UnreachableCodeReached,
}
error: hyper::Error(User(Service), guest never invoked `response-outparam::set` method: error while executing at wasm backtrace:
    0: 0x258f2 - wasmtime_filesystem_debug.wasm!__rust_start_panic
    1: 0x2546e - wasmtime_filesystem_debug.wasm!rust_panic
    2: 0x25224 - wasmtime_filesystem_debug.wasm!std::panicking::rust_panic_with_hook::h8bb4f94cf9e68e20
    3: 0x24646 - wasmtime_filesystem_debug.wasm!std::panicking::begin_panic_handler::{{closure}}::h2c354a3372fcca00
    4: 0x245ad - wasmtime_filesystem_debug.wasm!std::sys_common::backtrace::__rust_end_short_backtrace::hf791a0fba63af5e9
    5: 0x24dbc - wasmtime_filesystem_debug.wasm!rust_begin_unwind
    6: 0x2a66b - wasmtime_filesystem_debug.wasm!core::panicking::panic_fmt::haadcabb094f4c85b
    7: 0x2c2de - wasmtime_filesystem_debug.wasm!core::result::unwrap_failed::hd09ff05c9c2fb25f
    8: 0x6a1d - wasmtime_filesystem_debug.wasm!<wasmtime_filesystem_debug::Component as wasi::proxy::exports::wasi::http::incoming_handler::Guest>::handle::hd44cd139d21c109c
    9: 0x7853 - wasmtime_filesystem_debug.wasm!wasi::proxy::exports::wasi::http::incoming_handler::_export_handle_cabi::he753d69b8503d63c
   10: 0x7798 - wasmtime_filesystem_debug.wasm!wasi:http/incoming-handler@0.2.1#handle
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information

Caused by:
    wasm trap: wasm `unreachable` instruction executed)

The important error is this one:

failed to find a pre-opened file descriptor through which \"/shenanigans\" could be opened

coming from this source code:

fs::read_dir("/shenanigans").unwrap()

where the command to start it was

/root/.wasmtime/bin/wasmtime serve --dir=/shenanigans ./target/wasm32-wasip1/debug/wasmtime_filesystem_debug.wasm

Versions and Environment

Wasmtime version or commit: wasmtime-cli 24.0.0

Operating system: linux bullseye (docker base image is rust:bullseye)

Architecture: aarch64-unknown-linux-gnu

Extra Info

It looks like preopen doesn't happen with wasmtime serve, as there's no openat syscall to the directory we're trying to preopen. For comparison I also have a cli version of the wasm with a preopened directory where I can see an openat call to the directory I specified in strace, and the component prints out the contents of the directory as expected.

I also tried many variations of passing -Scli=y -Spreview2=y with no fix to directory access.

Adding -Dlogging=y does not yield anything useful.

My assumption that wasmtime serve should honour the --dir flag is based on the following:

• PR 8279: Support dirs/env vars in wasmtime serve #8279
• conversation in issue 8086: Change wasmtime serve to apply -S common by default #8086
• this source code dive:
  1. serve command's handle_request calls new_store
  2. serve command's new_store calls configure_wasip2
  3. common's configure_wasip2 iterates over the directories to open

It looks like common's self.dirs is not getting populated from the command line argument.

[alexcrichton]

Thanks for the detailed report! I think the main issue is with this line where that causes cargo component to use the "proxy adapter" when converting WASIp1 calls to WASIp2. The proxy version hardcodes that the filesystem is inaccessible and will always return an empty filesystem. If you want access to the filesystem I think you probably want to just remove that line. That should cause the "reactor adapter" to be used instead which forwards WASIp1 calls to WASIp2.

After doing that you'll want to pass -Scli to wasmtime serve, and it should work this time. It didn't work before because the proxy adapter meant that WASIp2 functions weren't ever called.

Does that help clarify what's happening?

[javorszky]

@alexcrichton yes, that makes sense. I removed that line and now the component works as expected, thank you so much!

I went through the documentation for wasmtime, and couldn't find any mention of what proxy:true does. I'll keep digging, but is there info about this already published I'm not finding?

Again, much appreciated!

[javorszky]

Solved by removing the proxy = true line from the Cargo.toml under the [package.metadata.component] section

[alexcrichton]

Ah the reason you didn't find that in the wasmtime docs is that it's a feature of cargo component, not Wasmtime. Wasmtime doesn't currently house docs for each guest language that could compile to wasm, although https://component-model.bytecodealliance.org/ is a location where such docs could work. Alas though I don't believe the proxy = true part is documented all that well in either place right now either, but it'd be good to contribute!

[javorszky]

Excellent, thank you! I'll put together some documentation once I figure out what it does in detail 🙂