Introduce strongly-typed system primitives (#1561)

* Introduce strongly-typed system primitives

This commit does a lot of reshuffling and even some more. It introduces
strongly-typed system primitives which are: `OsFile`, `OsDir`, `Stdio`,
and `OsOther`. Those primitives are separate structs now, each implementing
a subset of `Handle` methods, rather than all being an enumeration of some
supertype such as `OsHandle`. To summarise the structs:

* `OsFile` represents a regular file, and implements fd-ops
  of `Handle` trait
* `OsDir` represents a directory, and primarily implements path-ops, plus
  `readdir` and some common fd-ops such as `fdstat`, etc.
* `Stdio` represents a stdio handle, and implements a subset of fd-ops
  such as `fdstat` _and_ `read_` and `write_vectored` calls
* `OsOther` currently represents anything else and implements a set similar
  to that implemented by `Stdio`

This commit is effectively an experiment and an excercise into better
understanding what's going on for each OS resource/type under-the-hood.
It's meant to give us some intuition in order to move on with the idea
of having strongly-typed handles in WASI both in the syscall impl as well
as at the libc level.

Some more minor changes include making `OsHandle` represent an OS-specific
wrapper for a raw OS handle (Unix fd or Windows handle). Also, since `OsDir`
is tricky across OSes, we also have a supertype of `OsHandle` called
`OsDirHandle` which may store a `DIR*` stream pointer (mainly BSD). Last but not
least, the `Filetype` and `Rights` are now computed when the resource is created,
rather than every time we call `Handle::get_file_type` and `Handle::get_rights`.
Finally, in order to facilitate the latter, I've converted `EntryRights` into
`HandleRights` and pushed them into each `Handle` implementor.

* Do not adjust rights on Stdio

* Clean up testing for TTY and escaping writes

* Implement AsFile for dyn Handle

This cleans up a lot of repeating boilerplate code todo with
dynamic dispatch.

* Delegate definition of OsDir to OS-specific modules

Delegates defining `OsDir` struct to OS-specific modules (BSD, Linux,
Emscripten, Windows). This way, `OsDir` can safely re-use `OsHandle`
for raw OS handle storage, and can store some aux data such as an
initialized stream ptr in case of BSD. As a result, we can safely
get rid of `OsDirHandle` which IMHO was causing unnecessary noise and
overcomplicating the design. On the other hand, delegating definition
of `OsDir` to OS-specific modules isn't super clean in and of itself
either. Perhaps there's a better way of handling this?

* Check if filetype of OS handle matches WASI filetype when creating

It seems prudent to check if the passed in `File` instance is of
type matching that of the requested WASI filetype. In other words,
we'd like to avoid situations where `OsFile` is created from a
pipe.

* Make AsFile fallible

Return `EBADF` in `AsFile` in case a `Handle` cannot be made into
a `std::fs::File`.

* Remove unnecessary as_file conversion

* Remove unnecessary check for TTY for Stdio handle type

* Fix incorrect stdio ctors on Unix

* Split Stdio into three separate types: Stdin, Stdout, Stderr

* Rename PendingEntry::File to PendingEntry::OsHandle to avoid confusion

* Rename OsHandle to RawOsHandle

Also, since `RawOsHandle` on *nix doesn't need interior mutability
wrt the inner raw file descriptor, we can safely swap the `RawFd`
for `File` instance.

* Add docs explaining what OsOther is

* Allow for stdio to be non-character-device (e.g., piped)

* Return error on bad preopen rather than panic

crates/wasi-common/src/ctx.rs

@@ -1,13 +1,16 @@
 use crate::entry::{Entry, EntryHandle};
 use crate::fdpool::FdPool;
 use crate::handle::Handle;
-use crate::sys::oshandle::{OsHandle, OsHandleExt};
+use crate::sys::osdir::OsDir;
+use crate::sys::osother::{OsOther, OsOtherExt};
+use crate::sys::stdio::{Stderr, StderrExt, Stdin, StdinExt, Stdout, StdoutExt};
 use crate::virtfs::{VirtualDir, VirtualDirEntry};
 use crate::wasi::types;
 use crate::wasi::{Errno, Result};
 use std::borrow::Borrow;
 use std::cell::RefCell;
 use std::collections::HashMap;
+use std::convert::TryFrom;
 use std::ffi::{self, CString, OsString};
 use std::fs::File;
 use std::path::{Path, PathBuf};
@@ -32,9 +35,9 @@ pub enum WasiCtxBuilderError {
     /// Provided sequence of bytes contained an unexpected NUL byte.
     #[error("provided sequence contained an unexpected NUL byte")]
     UnexpectedNul(#[from] ffi::NulError),
-    /// Provided `File` is not a directory.
-    #[error("preopened directory path {} is not a directory", .0.display())]
-    NotADirectory(PathBuf),
+    /// The root of a VirtualDirEntry tree must be a VirtualDirEntry::Directory.
+    #[error("the root of a VirtualDirEntry tree at {} must be a VirtualDirEntry::Directory", .0.display())]
+    VirtualDirEntryRootNotADirectory(PathBuf),
     /// `WasiCtx` has too many opened files.
     #[error("context object has too many opened files")]
     TooManyFilesOpen,
@@ -43,8 +46,8 @@ pub enum WasiCtxBuilderError {
 type WasiCtxBuilderResult<T> = std::result::Result<T, WasiCtxBuilderError>;
 
 enum PendingEntry {
-    Thunk(fn() -> io::Result<OsHandle>),
-    File(File),
+    Thunk(fn() -> io::Result<Box<dyn Handle>>),
+    OsHandle(File),
 }
 
 impl std::fmt::Debug for PendingEntry {
@@ -53,9 +56,9 @@ impl std::fmt::Debug for PendingEntry {
             Self::Thunk(f) => write!(
                 fmt,
                 "PendingEntry::Thunk({:p})",
-                f as *const fn() -> io::Result<OsHandle>
+                f as *const fn() -> io::Result<Box<dyn Handle>>
             ),
-            Self::File(f) => write!(fmt, "PendingEntry::File({:?})", f),
+            Self::OsHandle(f) => write!(fmt, "PendingEntry::OsHandle({:?})", f),
         }
     }
 }
@@ -105,22 +108,37 @@ impl PendingCString {
     }
 }
 
+struct PendingPreopen(Box<dyn FnOnce() -> WasiCtxBuilderResult<Box<dyn Handle>>>);
+
+impl PendingPreopen {
+    fn new<F>(f: F) -> Self
+    where
+        F: FnOnce() -> WasiCtxBuilderResult<Box<dyn Handle>> + 'static,
+    {
+        Self(Box::new(f))
+    }
+
+    fn into(self) -> WasiCtxBuilderResult<Box<dyn Handle>> {
+        self.0()
+    }
+}
+
 /// A builder allowing customizable construction of `WasiCtx` instances.
 pub struct WasiCtxBuilder {
     stdin: Option<PendingEntry>,
     stdout: Option<PendingEntry>,
     stderr: Option<PendingEntry>,
-    preopens: Option<Vec<(PathBuf, Box<dyn Handle>)>>,
+    preopens: Option<Vec<(PathBuf, PendingPreopen)>>,
     args: Option<Vec<PendingCString>>,
     env: Option<HashMap<PendingCString, PendingCString>>,
 }
 
 impl WasiCtxBuilder {
     /// Builder for a new `WasiCtx`.
     pub fn new() -> Self {
-        let stdin = Some(PendingEntry::Thunk(OsHandle::from_null));
-        let stdout = Some(PendingEntry::Thunk(OsHandle::from_null));
-        let stderr = Some(PendingEntry::Thunk(OsHandle::from_null));
+        let stdin = Some(PendingEntry::Thunk(OsOther::from_null));
+        let stdout = Some(PendingEntry::Thunk(OsOther::from_null));
+        let stderr = Some(PendingEntry::Thunk(OsOther::from_null));
 
         Self {
             stdin,
@@ -167,27 +185,27 @@ impl WasiCtxBuilder {
 
     /// Inherit stdin from the host process.
     pub fn inherit_stdin(&mut self) -> &mut Self {
-        self.stdin = Some(PendingEntry::Thunk(|| Ok(OsHandle::stdin())));
+        self.stdin = Some(PendingEntry::Thunk(Stdin::stdin));
         self
     }
 
     /// Inherit stdout from the host process.
     pub fn inherit_stdout(&mut self) -> &mut Self {
-        self.stdout = Some(PendingEntry::Thunk(|| Ok(OsHandle::stdout())));
+        self.stdout = Some(PendingEntry::Thunk(Stdout::stdout));
         self
     }
 
-    /// Inherit stdout from the host process.
+    /// Inherit stderr from the host process.
     pub fn inherit_stderr(&mut self) -> &mut Self {
-        self.stderr = Some(PendingEntry::Thunk(|| Ok(OsHandle::stderr())));
+        self.stderr = Some(PendingEntry::Thunk(Stderr::stderr));
         self
     }
 
     /// Inherit the stdin, stdout, and stderr streams from the host process.
     pub fn inherit_stdio(&mut self) -> &mut Self {
-        self.stdin = Some(PendingEntry::Thunk(|| Ok(OsHandle::stdin())));
-        self.stdout = Some(PendingEntry::Thunk(|| Ok(OsHandle::stdout())));
-        self.stderr = Some(PendingEntry::Thunk(|| Ok(OsHandle::stderr())));
+        self.stdin = Some(PendingEntry::Thunk(Stdin::stdin));
+        self.stdout = Some(PendingEntry::Thunk(Stdout::stdout));
+        self.stderr = Some(PendingEntry::Thunk(Stderr::stderr));
         self
     }
 
@@ -231,28 +249,32 @@ impl WasiCtxBuilder {
 
     /// Provide a File to use as stdin
     pub fn stdin(&mut self, file: File) -> &mut Self {
-        self.stdin = Some(PendingEntry::File(file));
+        self.stdin = Some(PendingEntry::OsHandle(file));
         self
     }
 
     /// Provide a File to use as stdout
     pub fn stdout(&mut self, file: File) -> &mut Self {
-        self.stdout = Some(PendingEntry::File(file));
+        self.stdout = Some(PendingEntry::OsHandle(file));
         self
     }
 
     /// Provide a File to use as stderr
     pub fn stderr(&mut self, file: File) -> &mut Self {
-        self.stderr = Some(PendingEntry::File(file));
+        self.stderr = Some(PendingEntry::OsHandle(file));
         self
     }
 
     /// Add a preopened directory.
     pub fn preopened_dir<P: AsRef<Path>>(&mut self, dir: File, guest_path: P) -> &mut Self {
-        self.preopens.as_mut().unwrap().push((
-            guest_path.as_ref().to_owned(),
-            Box::new(OsHandle::from(dir)),
-        ));
+        let preopen = PendingPreopen::new(move || {
+            let dir = OsDir::try_from(dir).map_err(WasiCtxBuilderError::from)?;
+            Ok(Box::new(dir))
+        });
+        self.preopens
+            .as_mut()
+            .unwrap()
+            .push((guest_path.as_ref().to_owned(), preopen));
         self
     }
 
@@ -277,18 +299,22 @@ impl WasiCtxBuilder {
             }
         }
 
-        let dir = if let VirtualDirEntry::Directory(entries) = dir {
-            let mut dir = VirtualDir::new(true);
-            populate_directory(entries, &mut dir);
-            Box::new(dir)
-        } else {
-            panic!("the root of a VirtualDirEntry tree must be a VirtualDirEntry::Directory");
-        };
-
+        let guest_path_owned = guest_path.as_ref().to_owned();
+        let preopen = PendingPreopen::new(move || {
+            if let VirtualDirEntry::Directory(entries) = dir {
+                let mut dir = VirtualDir::new(true);
+                populate_directory(entries, &mut dir);
+                Ok(Box::new(dir))
+            } else {
+                Err(WasiCtxBuilderError::VirtualDirEntryRootNotADirectory(
+                    guest_path_owned,
+                ))
+            }
+        });
         self.preopens
             .as_mut()
             .unwrap()
-            .push((guest_path.as_ref().to_owned(), dir));
+            .push((guest_path.as_ref().to_owned(), preopen));
         self
     }
 
@@ -336,15 +362,16 @@ impl WasiCtxBuilder {
             log::debug!("WasiCtx inserting entry {:?}", pending);
             let fd = match pending {
                 PendingEntry::Thunk(f) => {
-                    let handle = EntryHandle::new(f()?);
-                    let entry = Entry::from(handle)?;
+                    let handle = EntryHandle::from(f()?);
+                    let entry = Entry::new(handle);
                     entries
                         .insert(entry)
                         .ok_or(WasiCtxBuilderError::TooManyFilesOpen)?
                 }
-                PendingEntry::File(f) => {
-                    let handle = EntryHandle::new(OsHandle::from(f));
-                    let entry = Entry::from(handle)?;
+                PendingEntry::OsHandle(f) => {
+                    let handle = OsOther::try_from(f)?;
+                    let handle = EntryHandle::new(handle);
+                    let entry = Entry::new(handle);
                     entries
                         .insert(entry)
                         .ok_or(WasiCtxBuilderError::TooManyFilesOpen)?
@@ -353,13 +380,9 @@ impl WasiCtxBuilder {
             log::debug!("WasiCtx inserted at {:?}", fd);
         }
         // Then add the preopen entries.
-        for (guest_path, dir) in self.preopens.take().unwrap() {
-            if !dir.is_directory() {
-                return Err(WasiCtxBuilderError::NotADirectory(guest_path));
-            }
-
-            let handle = EntryHandle::from(dir);
-            let mut entry = Entry::from(handle)?;
+        for (guest_path, preopen) in self.preopens.take().unwrap() {
+            let handle = EntryHandle::from(preopen.into()?);
+            let mut entry = Entry::new(handle);
             entry.preopen_path = Some(guest_path);
             let fd = entries
                 .insert(entry)

crates/wasi-common/src/entry.rs

@@ -1,11 +1,9 @@
-use crate::handle::Handle;
-use crate::wasi::types::{Filetype, Rights};
+use crate::handle::{Handle, HandleRights};
+use crate::wasi::types::Filetype;
 use crate::wasi::{Errno, Result};
-use std::cell::Cell;
 use std::ops::Deref;
 use std::path::PathBuf;
 use std::rc::Rc;
-use std::{fmt, io};
 
 pub(crate) struct EntryHandle(Rc<dyn Handle>);
 
@@ -33,118 +31,67 @@ impl Deref for EntryHandle {
     }
 }
 
-/// An abstraction struct serving as a wrapper for a host `Descriptor` object which requires
-/// certain rights `rights` in order to be accessed correctly.
+/// An abstraction struct serving as a wrapper for a `Handle` object.
 ///
-/// Here, the `descriptor` field stores the host `Descriptor` object (such as a file descriptor, or
-/// stdin handle), and accessing it can only be done via the provided `Entry::as_descriptor` method
+/// Here, the `handle` field stores an instance of `Handle` type (such as a file descriptor, or
+/// stdin handle), and accessing it can only be done via the provided `Entry::as_handle` method
 /// which require a set of base and inheriting rights to be specified, verifying whether the stored
-/// `Descriptor` object is valid for the rights specified.
+/// `Handle` object is valid for the rights specified.
 pub(crate) struct Entry {
-    pub(crate) file_type: Filetype,
     handle: EntryHandle,
-    pub(crate) rights: Cell<EntryRights>,
     pub(crate) preopen_path: Option<PathBuf>,
     // TODO: directories
 }
 
-/// Represents rights of an `Entry` entity, either already held or
-/// required.
-#[derive(Debug, Copy, Clone)]
-pub(crate) struct EntryRights {
-    pub(crate) base: Rights,
-    pub(crate) inheriting: Rights,
-}
-
-impl EntryRights {
-    pub(crate) fn new(base: Rights, inheriting: Rights) -> Self {
-        Self { base, inheriting }
-    }
-
-    /// Create new `EntryRights` instance from `base` rights only, keeping
-    /// `inheriting` set to none.
-    pub(crate) fn from_base(base: Rights) -> Self {
-        Self {
-            base,
-            inheriting: Rights::empty(),
-        }
-    }
-
-    /// Create new `EntryRights` instance with both `base` and `inheriting`
-    /// rights set to none.
-    pub(crate) fn empty() -> Self {
+impl Entry {
+    pub(crate) fn new(handle: EntryHandle) -> Self {
+        let preopen_path = None;
         Self {
-            base: Rights::empty(),
-            inheriting: Rights::empty(),
+            handle,
+            preopen_path,
         }
     }
 
-    /// Check if `other` is a subset of those rights.
-    pub(crate) fn contains(&self, other: &Self) -> bool {
-        self.base.contains(&other.base) && self.inheriting.contains(&other.inheriting)
+    pub(crate) fn get_file_type(&self) -> Filetype {
+        self.handle.get_file_type()
     }
-}
 
-impl fmt::Display for EntryRights {
-    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
-        write!(
-            f,
-            "EntryRights {{ base: {}, inheriting: {} }}",
-            self.base, self.inheriting
-        )
+    pub(crate) fn get_rights(&self) -> HandleRights {
+        self.handle.get_rights()
     }
-}
 
-impl Entry {
-    pub(crate) fn from(handle: EntryHandle) -> io::Result<Self> {
-        let file_type = handle.get_file_type()?;
-        let rights = handle.get_rights()?;
-        Ok(Self {
-            file_type,
-            handle,
-            rights: Cell::new(rights),
-            preopen_path: None,
-        })
+    pub(crate) fn set_rights(&self, rights: HandleRights) {
+        self.handle.set_rights(rights)
     }
 
-    /// Convert this `Entry` into a host `Descriptor` object provided the specified
+    /// Convert this `Entry` into a `Handle` object provided the specified
     /// `rights` rights are set on this `Entry` object.
     ///
-    /// The `Entry` can only be converted into a valid `Descriptor` object if
+    /// The `Entry` can only be converted into a valid `Handle` object if
     /// the specified set of base rights, and inheriting rights encapsulated within `rights`
-    /// `EntryRights` structure is a subset of rights attached to this `Entry`. The check is
+    /// `HandleRights` structure is a subset of rights attached to this `Entry`. The check is
     /// performed using `Entry::validate_rights` method. If the check fails, `Errno::Notcapable`
     /// is returned.
-    pub(crate) fn as_handle(&self, rights: &EntryRights) -> Result<EntryHandle> {
+    pub(crate) fn as_handle(&self, rights: &HandleRights) -> Result<EntryHandle> {
         self.validate_rights(rights)?;
         Ok(self.handle.get())
     }
 
-    /// Check if this `Entry` object satisfies the specified `EntryRights`; i.e., if
+    /// Check if this `Entry` object satisfies the specified `HandleRights`; i.e., if
     /// rights attached to this `Entry` object are a superset.
     ///
     /// Upon unsuccessful check, `Errno::Notcapable` is returned.
-    pub(crate) fn validate_rights(&self, rights: &EntryRights) -> Result<()> {
-        if self.rights.get().contains(rights) {
+    pub(crate) fn validate_rights(&self, rights: &HandleRights) -> Result<()> {
+        let this_rights = self.handle.get_rights();
+        if this_rights.contains(rights) {
             Ok(())
         } else {
             log::trace!(
                 "     | validate_rights failed: required rights = {}; actual rights = {}",
                 rights,
-                self.rights.get(),
+                this_rights,
             );
             Err(Errno::Notcapable)
         }
     }
-
-    /// Test whether this descriptor is considered a tty within WASI.
-    /// Note that since WASI itself lacks an `isatty` syscall and relies
-    /// on a conservative approximation, we use the same approximation here.
-    pub(crate) fn isatty(&self) -> bool {
-        self.file_type == Filetype::CharacterDevice
-            && self
-                .rights
-                .get()
-                .contains(&EntryRights::from_base(Rights::FD_SEEK | Rights::FD_TELL))
-    }
 }