Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serializable errors proof-of-concept #328

Closed
wants to merge 5 commits into from
Closed

Conversation

cycraig
Copy link
Contributor

@cycraig cycraig commented Jul 21, 2021

Description of change

Exploring various implementations of serializable errors for actor handlers and bindings.

Introduces a FlatEnum procedural macro to derive serializable C-style enums for errors, which hopefully reduces maintenance required at the cost of obscuring the actual types in the source code. The generated code is no_std compatible, even though the flat-enum crate itself requires std.

Essentially this changes the JsValue errors emitted from the wasm bindings to include the enum variant code. E.g.

Error::DecodeBitmap(std::io::Error::new(std::io::ErrorKind::Other, "something went wrong!"))
  • before, JS received only a string description of the error:
    "Failed to decode roaring bitmap: something went wrong!"

  • after, JS can match on the type of error too (as a string):
    {"code":"DecodeBitmap","description":"Failed to decode roaring bitmap: something went wrong!"}

Edit: the above behaviour has changed, the code is now accessible as a field on the error exported with #[wasm_bindgen], so it's no longer a string map/dictionary.

Note that specifying macro attributes such as #[wasm_bindgen] in derive proc-macros is an unstable feature only available on nightly for now (tracking issue), so we still have to serialize errors to and from JsValue manually with this approach. The alternative is ditching the FlatEnum proc-macro and writing out all the flat error structs and enum codes manually. Edit: this is incorrect, see my comment below. We still can't use #[wasm_bindgen] right now but that's due to conflicting exported symbols, which can be circumvented if we think it's worth it.

Without the #[wasm_bindgen] attribute we cannot generate proper error types with Typescript. However, until wasm-bindgen switches to a more useful JsError alternative to JsValue for handling errors in Result<T, JsValue>, we wouldn't be able to use those types anyway without manually casting the returned JsValue in Typescript code (possibly). There are several tracking issues for better error handling in wasm-bindgen (rustwasm/wasm-bindgen#1004, rustwasm/wasm-bindgen#1017, rustwasm/wasm-bindgen#1742, rustwasm/wasm-bindgen#2463). Edit: this is inaccurate, returning an Err(SomeError) where SomeError is annotated with #[wasm_bindgen] converts it to the correct type on the Javascript side when catching it.

Another downside of the proc_macro approach to deriving FlatEnum is that it needs to be called in the libraries where those errors are defined, and also adds the wasm-bindgen dependency to all the libraries which export an error. The dependency is optional and feature-gated but it's still not ideal. There may be an alternative, cleaner approach using a declarative macro which can be called in the bindings code instead, but iterating over the enum variants is more difficult with that approach and needs more work.

Incomplete and just a proof-of-concept, do no merge.

Links to any relevant issues

Serializable errors #321

Type of change

Add an x to the boxes that are relevant to your changes.

  • Bug fix (a non-breaking change which fixes an issue)
  • Enhancement (a non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Fix

How the change has been tested

Describe the tests that you ran to verify your changes.
Make sure to provide instructions for the maintainer as well as any relevant configurations.

Change checklist

Add an x to the boxes that are relevant to your changes.

  • I have followed the contribution guidelines for this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

cycraig added 3 commits July 21, 2021 16:41
Implement a custom proc-macro to derive the flat/C-style enum variants of
errors automatically, to reduce maintenance.
Derive From for conveting internal errors to JsValue with Into::into.
@cycraig
Copy link
Contributor Author

cycraig commented Jul 27, 2021

I think this approach---deriving FlatEnum for errors exported to Wasm bindings---can be considered complete as a proof-of-concept now. It's not perfect but it's useful in that it confirms some fundamental issues and things to look out for going forward.

Advantages:

  • informative: the derived FlatEnum exposes the enum variant to JS code, compared to just a string error message currently, potentially allowing certain errors to be handled differently in user code (e.g. retrying on a timeout).
  • easily maintainable: using #[derive(FlatEnum), any updates to the error enums are propagated, so the definitions only need to be updated once in the Rust code.
  • simplifies error handling: by implementing Into<JsValue> (through the From trait), errors in Wasm bindings can be short-circuited using the ? operator, without an intermediate .map_err(<function that converts error to JsValue>) step, as a minor convenience.
  • typed: with proper #[wasm_bindgen] usage the exported errors can now be matched by type. Edit: this was added after the initial comment.

Disadvantages:

  • "clunky": the derive approach requires the wasm-bindgen dependency to be added to all crates which export errors to bindings. While this is feature-gated, it's still a disadvantage until it's decided whether or not to use the #[wasm_bindgen] attribute throughout the identity.rs crates, which would require the dependency regardless.
  • wasm-specific: these flat errors are only really useful for the Wasm bindings, as other language bindings throw errors as exceptions automatically (e.g. uniffi-rs) or have their own requirements for error handling.
  • opaque: the FlatEnum derived structs are generated at compile time, so as a developer it's difficult to access its structure or get IDE assistance if trying to use them when coding. The flat errors also lose context and backtrace information.
    Edit: Javascript/Typescript developers can access the types from the exported wasm package now, so this isn't a concern.
  • stringly-typed: the enum variant code is serialized to JsValue as a string, and so cannot be exhaustively matched in a switch/case statement in e.g. Typescript. As mentioned in this PR and issue [Task] Implement a serializable error #321, this seems to be a fundamental problem with wasm-bindgen requiring all result errors to be converted to JsValue.
    Edit: this was corrected with #[wasm_bindgen]

I'm not a massive fan of this approach but wasm-bindgen only allowing JsValue for result errors is really the core problem preventing an ideal, strongly-typed solution. I can't imagine any solution from our side would be perfect until that is addressed in wasm-bindgen (along with the ability to export rich enums automatically using #[wasm_bindgen]).
Edit: this has been addressed, errors caught in Javascript/Typescript have proper types now.

An alternative approach would be to define the C-style enum errors in the wasm bindings to avoid polluting the identity.rs crates. That would use a declarative macro though, which might have problems enumerating the enum variants without extra dependencies.

Edit: this isn't particularly useful for serializable errors for the actor. I would still go with manually-defined serializable errors that wrap the errors in Rust in that case.

@PhilippGackstatter
Copy link
Contributor

PhilippGackstatter commented Jul 28, 2021

Very nice PoC!

I am primarily wondering whether JS developers would actually match on the code property. I'm very unfamiliar with JS error handling, but at least according to this SO answer, the accepted default is both messy and inaccurate, i.e. catching all or nothing. So, essentially, is the trouble with the proc-macro and generating all those errors worth it? If our error handling approach with the codes is actually unidiomatic for JS, then it might not end up being used and most of the work might just go to waste 😕.

It would be interesting to have a JS developer's view. @abdulmth do you think having

{"code":"DecodeBitmap","description":"Failed to decode roaring bitmap: something went wrong!"}

thrown instead of just

"Failed to decode roaring bitmap: something went wrong!"

would be a worthwhile improvement? Would JS devs actually match on the code if they have the possibility to do so? Or are they generally used to match on strings if they want to catch exception A, but not exception B?

@cycraig as you mentioned in #321, the C-style enums aren't actually a fundamental binding requirement, contrary to my initial belief. If we end up going with the current approach anyway, because it has value for the JS devs, I think I would still prefer regular Rust enums (option 2) for the actor, since that would be more idiomatic for matching. Do you think we could allow deriving both C-style enums for bindings, and Rust enums for the actor?

Depending on whether we go forward with the current JS error approach with codes, we might then also want to be able to convert an actor error implemented as a Rust style enum to a C style enum for exposure to JS bindings. But I think we can leave that conversation for later.

@abdulmth
Copy link
Contributor

abdulmth commented Jul 28, 2021

Exceptions are usually done by throwing an Error object (see Error). Expected are the name and message properties. But as Phillip said, you can throw anything in JavaScript (a string or an arbitrary object). But usually I expect an object to be thrown that has name and message properties.

@cycraig
Copy link
Contributor Author

cycraig commented Jul 28, 2021

@cycraig as you mentioned in #321, the C-style enums aren't actually a fundamental binding requirement, contrary to my initial belief. If we end up going with the current approach anyway, because it has value for the JS devs, I think I would still prefer regular Rust enums (option 2) for the actor, since that would be more idiomatic for matching. Do you think we could allow deriving both C-style enums for bindings, and Rust enums for the actor?

Indeed, the intermediate C-style are basically useless for the wasm-bindings because the enum variants can't be matched on in JavaScript except as a string and the Rust code exported to Wasm can just use the rich error enums directly. In my opinion the actor should use Rust enums that wrap any errors it needs as a string, unless it is a useful subvariant e.g.

#[derive(serde::Serialize, serde::Deserialize)]
enum SomeActorError {
  StrongholdPasswordNotSet,
  AccountError(string),
  ClientError(string),
  ...
}

Then implement From manually for each of the errors as-needed:

impl From<AccountError> for SomeActorError {
  fn from(error: AccountError) -> SomeActorError {
    match error {
      StrongholdPasswordNotSet => SomeActorError::StrongholdPasswordNotSet,
      err => SomeActorError::AccountError(err.to_string()),
    }
  }
}

If we need to send things like backtrace information (which seems to be a somewhat complex subject), then we could use an intermediate struct (that is still serializable) to hold it, e.g.

#[derive(serde::Serialize, serde::Deserialize)]
struct SerdeError {
  error: String,
  backtrace: Backtrace/String/something serializable,
}


#[derive(serde::Serialize, serde::Deserialize)]
enum SomeActorError {
  StrongholdPasswordNotSet(SerdeError),
  AccountError(SerdeError ),
  ClientError(SerdeError),
  ...
}

The above examples are just for demonstration. One could derive those From implementations to embed the variant code using a proc-macro, as done in this PR but in my opinion the actor errors and wasm binding errors should be considered completely separately.

Exceptions are usually done by throwing an Error object (see Error)

This changes things a bit. The current implementation only returns arbitrary objects/strings, never a proper JavaScript Error. I.e. when using try-catch for errors in the following code example, e instanceof Error will always return false for anything we return, even with the current changes in this PR:

let messageId;
try {
    messageId = await client.publishDocument(doc.toJSON());
} catch (e) {
    if (e instanceof Error) {
        console.log(`e is an Error! ${e}`)
    } else {
        console.log(`not an Error... ${e}`)
    }
}

This will always print "not an Error..."

We can convert the error we return into a js_sys::Error, as suggested by rustwasm/wasm-bindgen#1735 (comment). This would make e instanceof Error return true. However, this approach may not be ideal since the only two fields of Error (name and message) are both simple strings, so while we could pass the serialised JSON error as the message I think it may need to be parsed as JSON inside JavaScript to access custom fields like code and description?

Based on the points raised already, I'll work on another proof-of-concept. Essentially it will convert the Rust errors directly to some WasmError struct without C-style enums as an intermediary (since those are useless and unused).

@cycraig cycraig self-assigned this Jul 29, 2021
@PhilippGackstatter
Copy link
Contributor

I think it may need to be parsed as JSON inside JavaScript to access custom fields like code and description?

I'm not sure I've understood yet why we can't map the code into name and description into message? If code is the name of the enum variant, you can match on that even if it's a string.

I've taken a quick look at aws-sdk-js to see their approach, and it seems they frequently use throw new Error("some error description"). If we overwrite error.name with the name of the variant, then devs can use that to implement branching logic. I understand that we have not the same level of expressiveness as in Rust, because of nesting, but I thought we had accepted this as a trade-off. This seems like the best and simplest thing we can do with the JS Error type. If someone wants branching logic within a variant, they can still do some (ugly of course) error.name.contains("error string i want to match"), which is probably what most JS devs need to do anyway with most exceptions.

@cycraig
Copy link
Contributor Author

cycraig commented Jul 29, 2021

I'm not sure I've understood yet why we can't map the code into name and description into message? If code is the name of the enum variant, you can match on that even if it's a string.

We absolutely can do that and that will likely be the final implementation. I'm just checking whether we can do any better, by e.g. adding more fields to the returned error apart from the description, such as stacktraces to aid in debugging.

On a related note: I have to correct my previous statement that we can't use #[wasm_bindgen] in the proc-macro, we absolutely can (as long as it's placed above/before the #[derive(...)] attributes).

This enables us to generate something like the following:

#[wasm_bindgen]
#[derive(Clone, Debug)]
pub struct WasmError {
  code: WasmErrorCode,
  description: String,
}

#[wasm_bindgen]
impl WasmError {
  #[wasm_bindgen(constructor)]
  pub fn new(code: WasmErrorCode, description: String) -> Self {
    Self {
      code,
      description,
    }
  }

  #[wasm_bindgen(getter)]
  pub fn code(&self) -> WasmErrorCode {
    self.code
  }

  #[wasm_bindgen(getter)]
  pub fn description(&self) -> String {
    self.description.clone()
  }
}

#[wasm_bindgen]
#[derive(Clone, Copy, Debug)]
pub enum WasmErrorCode {
  CryptoError,
  ...
}

In Javascript, this would enable the user to check the type of code too:

let messageId;
try {
    messageId = await client.publishDocument(doc.toJSON());
} catch (e) {
    if (e instanceof Error) {
        console.log(`e is an Error! ${e}`)          // WasmError is still NOT a Javascript Error
    } else if (e instanceof WasmError) {            // WasmError is exported by identity-wasm
        if (e.code === WasmErrorCode.CryptoError) { // WasmErrorCode enum variants can be compared directly
            console.log(`CryptoError: ${e}`)
        } else {
            console.log(`Unknown WasmError: ${e}`)
        }
    } else {
        console.log(`not an Error: ${e}`)
    }
}

However, doing that in practice for all the different error enums identity.rs exports, the linker fails due to "duplicate symbols":

 = note: rust-lld: error: duplicate symbol: flaterror_new
          >>> defined in C:\Work\iota\identity.rs\bindings\wasm\target\wasm32-unknown-unknown\release\deps\libidentity_credential-61734c946a494da3.rlib(identity_credential-61734c946a494da3.identity_credential.23b4pm7p-cgu.1.rcgu.o)
          >>> defined in C:\Work\iota\identity.rs\bindings\wasm\target\wasm32-unknown-unknown\release\deps\libidentity_did-9b8e16a2407ef1de.rlib(identity_did-9b8e16a2407ef1de.identity_did.9s0pxrfz-cgu.0.rcgu.o)

          rust-lld: error: duplicate symbol: __wbindgen_describe_flaterror_new
          >>> defined in C:\Work\iota\identity.rs\bindings\wasm\target\wasm32-unknown-unknown\release\deps\libidentity_credential-61734c946a494da3.rlib(identity_credential-61734c946a494da3.identity_credential.23b4pm7p-cgu.1.rcgu.o)
          >>> defined in C:\Work\iota\identity.rs\bindings\wasm\target\wasm32-unknown-unknown\release\deps\libidentity_did-9b8e16a2407ef1de.rlib(identity_did-9b8e16a2407ef1de.identity_did.9s0pxrfz-cgu.0.rcgu.o)

          rust-lld: error: duplicate symbol: flaterror_code
          >>> defined in C:\Work\iota\identity.rs\bindings\wasm\target\wasm32-unknown-unknown\release\deps\libidentity_credential-61734c946a494da3.rlib(identity_credential-61734c946a494da3.identity_credential.23b4pm7p-cgu.1.rcgu.o)
          >>> defined in C:\Work\iota\identity.rs\bindings\wasm\target\wasm32-unknown-unknown\release\deps\libidentity_did-9b8e16a2407ef1de.rlib(identity_did-9b8e16a2407ef1de.identity_did.9s0pxrfz-cgu.0.rcgu.o)

          rust-lld: error: duplicate symbol: __wbindgen_describe_flaterror_code
          >>> defined in C:\Work\iota\identity.rs\bindings\wasm\target\wasm32-unknown-unknown\release\deps\libidentity_credential-61734c946a494da3.rlib(identity_credential-61734c946a494da3.identity_credential.23b4pm7p-cgu.1.rcgu.o)
          >>> defined in C:\Work\iota\identity.rs\bindings\wasm\target\wasm32-unknown-unknown\release\deps\libidentity_did-9b8e16a2407ef1de.rlib(identity_did-9b8e16a2407ef1de.identity_did.9s0pxrfz-cgu.0.rcgu.o)

          rust-lld: error: duplicate symbol: flaterror_description
          >>> defined in C:\Work\iota\identity.rs\bindings\wasm\target\wasm32-unknown-unknown\release\deps\libidentity_credential-61734c946a494da3.rlib(identity_credential-61734c946a494da3.identity_credential.23b4pm7p-cgu.1.rcgu.o)
          >>> defined in C:\Work\iota\identity.rs\bindings\wasm\target\wasm32-unknown-unknown\release\deps\libidentity_did-9b8e16a2407ef1de.rlib(identity_did-9b8e16a2407ef1de.identity_did.9s0pxrfz-cgu.0.rcgu.o)

[and so on...]

We could try and make the derived struct names hygienic (it's really a problem with the code generated by wasm_bindgen that's unhygienic and conflicting in the same namespace though). The fix would be to give unique names to all Error enums in the identity.rs sub-crates, e.g. AccountError, DiffError etc. Not too bad but ultimately it doesn't seem worth it if Javascript developers aren't going to use it. Update: renaming the errors worked.

I've taken a quick look at aws-sdk-js to see their approach, and it seems they frequently use throw new Error("some error description").

With the norm being generic error strings in Javascript, I think simply returning a Javascript Error with name = code and message = description is "fine". I've been spoiled by the enums and error handling in Rust but the reality is probably that it's not especially useful nor idiomatic in Javascript.

@cycraig
Copy link
Contributor Author

cycraig commented Jul 29, 2021

After some yak shaving, I've updated the code with proper usage of #[wasm_bindgen] which renames the exported generated types from e.g. WasmIotaError back to IotaError in Javascript code, so it elides the conversion on their side.

This means proper type comparisons may now be used as per the original proposal in the issue (instead of string comparisons).
E.g.

let messageId;
try {
    messageId = await client.publishDocument(doc.toJSON());
} catch (e) {
    if (e instanceof Error) {                 // exported errors are still NOT subclasses of Javascript Errors
        console.log(`e is an Error! ${e}`)
    } else if (e instanceof CoreError) {      // CoreError is exported by identity-core
        switch (e.code) {                     // CoreErrorCode enum variants can be compared directly
            case CoreErrorCode.Crypto: {  
                console.log(`CryptoError: ${e}`); break;
            }
            case CoreErrorCode.InvalidUrl: {
                console.log(`InvalidUrl: ${e}`); break;
            }
            default: {
                console.log(`Unhandled CoreError: ${e}`)
            }
        }
    } else {
        console.log(`not an Error: ${e}`)
    }
}

Most of the original disadvantages of this approach have been addressed now (at the cost of re-naming the errors to avoid namespace conflicts and including wasm-bindgen as a feature-gated dependency in most of the identity crates) so it's not too bad anymore. It's perhaps overkill and unlikely to be used though.

I'll still push the alternative implementation based on js_sys::Error soon.

@cycraig cycraig mentioned this pull request Aug 2, 2021
10 tasks
@cycraig
Copy link
Contributor Author

cycraig commented Aug 25, 2021

Closed in favour of #344

@cycraig cycraig closed this Aug 25, 2021
@JelleMillenaar JelleMillenaar deleted the feat/serde-errors-poc branch November 16, 2021 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants