This gets us another step closer to running the Wasmtime tests.

Add a `result` return type to `command` so that it can indicate success
or failure.

The idea here is that this isn't a full `i32` return value because the
meaning of return values isn't portable across platforms. Also, Typed Main
is a better long-term answer for users that want rich error return
values from commands.

…lliance#45)

* implement `wasi-filesystem::readdir` and related functions

This adds a `directory_list` test and provides the required host implementation.

I've also added a file length check to the `file_read` test, just to cover a bit
more of the API.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

* fix memory corruption in `fd_readdir` polyfill

We were copying `name.len() * 256` bytes instead of just `name.len()` bytes,
which was overwriting other parts of `State` and causing untold havoc.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

* check type of entry in `Table::delete`

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

…iance#46)

…ce#47)

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

Many users won't need these, because they'll just grab one default clock
handle and reuse it for the rest of the program. But it should still be
possible to close a clock handle.

Also, add a simple test for the clock APIs.

* Use `use` imports consistently.

* Rename `flags_from_descriptor_flags` to `descriptor_flags_from_flags`.

* Rename `black_box` to `obscure`.

* Make some comments be doc comments.

* Use a hyphen for compound adjectives.

* Delete an unused variable.

* Update the name of `change-file-permissions-at`.

* Use generated typedefs instead of hard-coding u64.

* Use core::hint::black_box now that it's stabilized.

1. It needed a `config.async_support(true)` to actually work at all.
2. Add `inherit_stdin` and `inherit_stdout` calls when building WasiCtx to make stdio useful.
3. Set stdout descriptor to 1.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

* Implement file append functionality.

 - In the preview1-to-preview2 polyfill, using `append_via_stream`.
 - In the host implementation, using a new system-interface `FileIoExt::append` function.

* Add a basic testcase.

Replace `AsRawHandleOrSocket` with `AsBorrowedHandleOrSocket`, to make
the Windows code work more like the Unix code.

And add CI tests for macOS too.

This deletes the old unmaintained Windows `poll_oneoff` implementation
and replaces it with one based on `WSAPoll`, which rustix is able to
present via a compatible API with Unix `poll`, so it can share the same
implementation.

Use `num_ready_bytes()` on objects which don't contain pollable handles.

When a `ReadPipe` is reading from a memory buffer, there is no host file
descriptor to poll with, but there doesn't need to be because the
contents are immediately available. Teach the `poll_oneoff` function to
use system-interface's `ReadReady` trait, which returns the number of
bytes read to be read immediately, to implement polling for such input
sources.

This also fixes the polyfill to avoid calling `bytes_readable` on
non-socket handles, which is needed to make the test pass.

On Windows, there doesn't appear to be a way to sync a directory, to
ensure that the directory entry for a file is sync'd. So for now, just
silently succeed. I've opened WebAssembly/wasi-filesystem#79 to track
this at the spec level.

* Import the Wasmtime WASI tests.

Copy the Wasmtime WASI tests in crates/test-programs/wasi-tests as of
7b75ba9 into the preview2 prototype
at test-programs/wasi-tests.

This adds TODO lines to tests/runtime.rs for tests which aren't passing
yet.

* Open directories properly on Windows.

* implement some host filesystem operations:

create_directory_at
stat
remove_directory_at
symlink_at
unlink_file_at

* host filesystem tests: some new ones pass, others use EXPECT_FAIL constant

by using `if EXPECT_FAIL` instead of `if true` its now easier to find
which failing tests are actually passing.

The following are now expected to pass (though it may not be the fault
of this branch fixing them):

dangling_fd
file_pread_pwrite
file_unbuffered_write
isatty
path_exists
symlink_create

* dangling_fd fails on windows

* test-programs macros: deduplicate tests

I invoked the test_log::test and tokio::test proc macros incorrectly, so
it created 2 identically named versions of each test

…ance#55)

This corresponds to WebAssembly/wasi-filesystem#76.

…iance#67)

* On Windows, ignore access-denied errors from `sync_all`.

On Windows `sync_all` does `FlushFileBuffers` which fails with an
access-denied error if the file not opened for writing. Ignore this
error, for compatibility with POSIX-style APIs.

Fixes bytecodealliance#66.

* Ignore `ERROR_ACCESS_DENIED` from `sync_data` too.

* Temporarily disable directory syncing for now.

There will be other worlds in WASI, so rename the `wasi-command` world
to accomodate them.

* Update wasmtime and wit-bindgen

This commit updates the `wasmtime` dependency and the
`wit-bindgen-guest-rust` dependencies to namely update the `wit-parser`
and component underlying implementations. This pulls in
bytecodealliance/wasm-tools#867 which is the implementation of `use` for
WIT. This does not currently break apart the one large `wasi.wit` file,
instead just gets tests working.

* Split apart `wasi.wit` into multiple `*.wit` files

This commit refactors the monolithic `wasi.wit` file into multiple
files. I've taken some liberties in naming here so suggestions are
definitely welcome! I've resolved some TODO annotations as well about
`use`-ing types between interfaces. There are two issues remaining,
however:

* The `wasi-command` world still hardcodes `u32` because `world` items
  don't support `use` just yet. That isn't a hard limitation of WIT,
  however, it's just a temporary limitation of the implementation.

* The `wasi-clocks` interface defines the `wasi-future` type. This is
  due to a cyclic dependency where `wasi-future` is defined within
  `wasi-poll` which depends on `wasi-clocks` for types. I've left this
  to a future refactoring to probably have a `types` interface of some
  form somewhere.

* Update patch for wasm-tools

* Switch to wasmtime upstream

* Update CI for build

* Remove patch overrides

* Update names of uploaded files

* Rename `wasi-future` to `waitable`.

You can wait on `waitable`s multiple times, which differs from proper
futures, which you can only wait on once.

* Rename `waitable` to `pollable`.

Because the function they're passed to is `poll-oneoff`.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

use main module's `cabi_realloc` instead of `memory.grow`

* Add basic TCP socket APIs.

This is based on @npmccullum's [wasi-snapshot-preview2 draft] which is
in turn based on the [wasi-sockets proposal], though for simplify for
now it omits UDP sockets and some other features.

It's also based on the [pseudo-streams PR]; so that should proceed
first before this.

[draft wasi-snapshot-preview2]: https://github.com/npmccallum/wasi-snapshot-preview2
[wasi-sockets proposal]: https://github.com/WebAssembly/wasi-sockets
[pseudo-streams PR]: bytecodealliance/preview2-prototyping#29

* bump wasm-tools to latest releases

* wit-component includes upstream fixes, plus wasm-metadata
* update wasm-encoder to match

* test-programs build.rs: use `cargo +nightly` flag to ensure nightly

on CI, rustup has the default set to nightly, but this may not be the
case on developer machines

* invoke cargo nightly through rustup

…codealliance#76)

When no timeout is requested, pass `-1` to the host `poll` to disable
the timeout.

…odealliance#73)

* Split wasi-stream into separate input and output stream types.

This syncs the prototype's streams with the upstream wasi-io repo.

Streams are unidirectional, so this allows us to statically describe
whether something is an input stream or an output stream in an
interface.

This differs a little from the component model async streams, which
don't have separate input and output streams, but it does partially
reflect how the component model async design differentiates between
input streams in type signatures, which are passed in as arguments,
and output streams, which are returned as return values.

* Fix compilation on Windows.

…ance#81)

this is broken at the moment and is getting in the way of testing more
interesting bits of functionality. we'll re-enable it once other stuff
is working.

update the whole cap-std family in the process

…ytecodealliance#69)

* filesystem: fill in remaining todo'd methods which have wasi-common implementations

* turn on fd_filestat_set, path_rename_dir_trailing_slashes, and symlink_filestat tests, which now pass

* change expect_fail to a function

* filesystem set flags: return Notsup instead of panic

…lliance#83)

* short-circuit reentrance when allocating stack and `State`

Per bytecodealliance/wasm-tools#919,
`wit-component` needs to lazily allocate the adapter stack to avoid
premature or infinite reentrance from the main module to the adapter.
This means adding an additional `allocation_state` global variable and
using it to determine when to return early from an exported function,
e.g. because we're in the process of either allocating the stack or
allocating `State`.

This requires an updated `wit-component` dependency once
bytecodealliance/wasm-tools#919 has been
merged.

Fixes bytecodealliance#78

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

* remove redundant unsafe blocks

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

* update dependencies

This brings us up-to-date with wasi-tools, wit-bindgen, and the latest
component ABI.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

---------

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

* Use precompiled `wit-bindgen` binaries from CI
* Use a release branch of Wasmtime for 6.0.0
* Use `wit-bindgen`-the-crate from crates.io

This updates to the latest versions of the wit files in the proposal
repos, and updates the polyfill and host implementations accordingly.
The changes here are just some renames and reorganizations.

…ode/update-wasi

Update wasi-filesystem, wasi-clocks, wasi-io, and wasi-poll.

…rt_preopens_environment_2

Import functions provide environment variables and preopens

…ce#89)

Preview1's `fd_readdir` includes `.` and `..`, but Preview2 is changing
to avoid this. Update the Preview2 host implementation to omit these
entries, and add code the polyfill to re-add them.

…ce#93)

The order of operations for handling the seconds/nanoseconds part of the
structure were a bit off which meant that a nonzero nanosecond field
would have an unexpected effect on the result.

Closes bytecodealliance#92

The hope is that Wasmtime from crates.io is, from now on, sufficient to
run all the component tests and such.

* Rebase on the new wasi-sockets.

This switches to using the wasi-sockets wit files from
WebAssembly/wasi-sockets#16. Many things are still stubbed out with
`todo!()` for now.

* Fix compilation on Windows.

* Allow preopen file descriptors to be closed.

This ports the test updates from bytecodealliance#5828 to the
preview2-prototype and implements the ability to close preopened file
descriptors.

* Fix compilation on Windows.

* Make `read-via-stream` etc. infallible.

Wrap `File`'s contents in an `Arc` so that file handles and stream
handles can share the underlying file descriptor, so that we don't have
to call `try_clone()`. This avoids the possibility of `try_clone()`
failing.

This also makes `File` similar to `Dir`, which already has an `Arc`, in
preparation for merging `File` and `Dir` now that WASI has decided that
file and directory shouldn't be separate types.

* Fix build on Windows.

* Rebase on the wasi-cli world.

Rebase on the wasi-cli wit files. This incorporates some major renamings
and reorganizations, and updates to use wasi-sockets from the repo rather
than our old prototype files.

At present this depends on several bugfixes in wasmtime and wit-bindgen
so this is using git dependencies for now.

This also temporarily preserves the dedicated stderr interface, which is
useful for debugging the file descriptor table code, which needs to work
before the regular stderr works.

* Update the verify program for the new module names.

* Disable debug-assertions in the adapter build.

This is roughly the same as, the previous `unchecked` option in the
bindings, with the changes in the latest wit-bindgen.

* Update CI to use the new file names.

…wasi-common-linking

Co-habitate with wasi-common from wasmtime

* rename host runtime tests to command tests

* add a test for using wasi in a reactor

* commit cargo.lock

* reactor tests: fix wit deps

* test-programs: reactor-tests can build on stable

note that this fails because it exposes wasi-libc ctors calling import
functions from inside cabi_realloc

* test-programs: show that ctors fix in wit-bindgen fixes bug

* ci: install wasm32 targets for stable as well as nightly

* wit-bindgen: use 0.4.0

* ci: use wit-bindgen 0.4.0

* Co-habitate with wasi-common from wasmtime

* adapter: code motion in cargo feature & artifact names to cli-command, cli-reactor

there will shortly be a third type of reactor (non-cli, idk what to call it)

---------

Co-authored-by: Trevor Elliott <telliott@fastly.com>

This updates to the latest wasi-filesystem, wasi-sockets, wasi-io, and
wasi-cli wit changes, except for two things:
 - `filesystem.types` is temporarily still named `filesystem.filesystem`, to
    work around bytecodealliance#5961
 - wasi-stderr is temporarily still present, for debugging

…code/update-wit

Update to the latest wit files.

…tures_drop_cli

drop cli- prefix from reactor, command features & artifact names

…ngs-crate

feat: add wasi bindings crate

This adds a basic scaffolding for WASI HTTP proposal as defined [here](https://github.com/WebAssembly/wasi-http/tree/1185b0a0224bc3e11274ad6298fd065cae472651/wit).

rather than the freestanding function which existed before we added that
to the code generator in wasmtime#5589

)

* Add the proxy world, and remove HTTP from command.

Add the proxy world, from the wasi-http repository, and add a
`--world` flag to the `host` command for selecting which world to run
them in. Current options are `command` and `proxy`.

Also, remove HTTP APIs from the command world, because command is meant
for CLI commands that run with a `main` function with command-line
arguments, so it doesn't include programs that run with incoming HTTP
requests

This also removes HTTP APIs from the reactor world. This world doesn't
yet have a standards proposal and doesn't yet have a clear purpose, but
it's otherwise currently in sync with the command world, and this PR
restores it to that.

* Add poll and streams to the proxy world implementation.

To facilitate this, factor out all the code into helper functions.
In the future bindings tools may give us better option for this, but for
now just manually factor things out.

* Initial TCP socket host implementation.

Initial TCP socket host implementation. This is not yet usable, as it
doesn't yet populate the network resource with any networks, but it's
a start with the major TCP socket functions implemented.

* Update dependencies.

* Populate the default instance-network pool.

* Fix an unbounded buffer allocation.

* Add the tcp-create-socket and udp-create-socket implementations.

* Allow things like getsockname to work on listening sockets.

* Wire up command-line arguments.

* Enable rustix with the "net" feature on all platforms.

…time

Update to Wasmtime 7

…ppable_for_real_this_time

Use trappable errors in host bindings for wasi-filesystem

…preopens to main

stdio never gets initialized for the reactor, tha

These are useful for giving guests read-only access to a directory and
its contents.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

fix nanosecond->millisecond conversion in `poll_oneoff`

add `ReadOnlyDir` and `ReadOnlyFile`

Redesign stdio to be available via an import function; eliminate special stderr interface