Create the `web-sys` crate mechanically from WebIDL #409

fitzgen · 2018-07-06T22:52:24Z

I vendored all the WebIDL from mozilla-central and then used an apache-style available/enabled symlinking thing to choose which ones we actually generate bindings for. Right now the only thing we actually generate bindings for is Event.

See the changes to the guide for details on the expected workflow for adding support for more Web APIs.

This is still WIP because I'm running into issues getting the project builder working for different crates. The way it wants to link package.json and friends is a bit funky. Nothing too hard to fix, but I am not going to come back to this for a couple days and figured folks might want to provide some early feedback.

This will eventually contain all the WebIDL-generated bindings to Web APIs.

This is necessary for the WebIDL frontend, which can't translate many WebIDL constructs into equivalent wasm-bindgen AST things yet. It lets us make incremental progress: we can generate bindings to methods we can support right now even though there might be methods on the same interface that we can't support yet.

It was only `pub` so that we could test it, but we ended up moving towards integration tests rather than unit tests that assert particular ASTs are parsed from WebIDL files.

Instead of going into an infinite loop, detect when webpack-dev-server fails to start up and early exit the test.

Instead of only a relative path.

This helps for debugging failing WebIDL-related tests.

Still need to fix and finish the test.

alexcrichton

This looks fantastic to me!

alexcrichton · 2018-07-07T01:49:18Z

.travis.yml

+      install:
+        - npm install
+      script:
+        - (cd crates/web-sys && cargo test)


I'm personally also a fan of cargo test --manifest-path crates/web-sys/Cargo.toml

That way if we don't fill it in the error message doesn't look quite so bizarre

Otherwise we don't see any output!

Make sure these are looked up in the git project root rather than the crate root

This was meant for debugging and is otherwise pretty noisy

alexcrichton · 2018-07-07T02:42:19Z

guide/src/web-sys/overview.md

+These are the WebIDL interfaces that we will actually generate bindings for (or
+at least bindings for *some* of the things defined in these files).
+
+These are all symlinks into the `webidls/available` directory.


Given that windows doesn't support symlinks very well, could we instead perhaps move files from available to enabled?

…ral)]` Fixes rustwasm#432

…urposes Helps debug bad WebIDL bindings generation inside tests.

This helps when debugging things that need to become structural.

fitzgen · 2018-07-09T20:12:23Z

Looks like appveyor headless testing is failing with this now:

errors:
    exception: __wbg_f_values_values_Array_target is undefined
    stack: __wbg_f_values_values_Array@http://localhost:8080/1.bundle.js:695:5
    __wbg_f_values_values_Array@http://localhost:8080/bundle.js:59:23
    wasm_bindgen::js::Array::values::h31b91cbdcb83beef@http://localhost:8080/bundle.js:2676:1
    test80::get_values::h95694f03c9da97e7@http://localhost:8080/bundle.js:1544:1
    get_values@http://localhost:8080/bundle.js:1740:1
    get_values@http://localhost:8080/1.bundle.js:312:27
    test@http://localhost:8080/0.bundle.js:22:36
    run@http://localhost:8080/bundle.js:9489:23
    ./run.js/

fitzgen · 2018-07-09T20:24:09Z

Looks like appveyor headless testing is failing with this now:

I think this is failing on master as well.

We only use it to serialize headless tests so that we don't try to bind the port concurrently. Its OK to run another headless test if an earlier one panicked.

Allows dynamically casting values to `js::Object` and `js::Function`.

alexcrichton · 2018-07-09T22:10:22Z

Bah that's probably myself botching a merge, sorry about that! Looks like AppVeyor is running Firefox 59 but Array.prototype.values was added in Firefox 60, which'd explain the failing test.

If we create bindings to a method that doesn't exist in this implementation, then it shouldn't fail until if/when we actually try and invoke that missing method.

alexcrichton · 2018-07-09T22:22:43Z

src/js.rs

@@ -443,6 +443,16 @@ extern "C" {
    pub fn to_string(this: &Function) -> JsString;
 }

+impl JsValue {
+    pub fn as_function(&self) -> Option<Function> {


Should this perhaps be &Function via a transmute? (same w/ as_object below)

And document all the cast/convert/check methods for js value.

fitzgen · 2018-07-09T23:35:11Z

Ok, finally got CI green! Going to merge!

This commit fixes instantiation of the wasm module even if some of the improted APIs don't exist. This extends the functionality initially added in rustwasm#409 to attempt to gracefully allow importing values from the environment which don't actually exist in all contexts. In addition to nonexistent methods being handled now entire nonexistent types are now also handled. I suspect that eventually we'll add a CLI flag to `wasm-bindgen` to say "I assert everything exists, don't check it" to trim out the extra JS glue generated here. In the meantime though this'll pave the way for a wasm-bindgen shim to be instantiated in both a web worker and the main thread, while using DOM-like APIs only on the main thread.

fitzgen added 14 commits July 6, 2018 09:17

Create a new web-sys crate

0b3c2c3

This will eventually contain all the WebIDL-generated bindings to Web APIs.

ci: Test the new web-sys crate in CI

248d233

web-sys: Add a small README

f591478

web-sys: Vendor all the WebIDL files from mozilla-central

40239d3

webidl: Add a bunch of missing semicolons

8005a89

webidl: Make parsing private

c55e1c5

It was only `pub` so that we could test it, but we ended up moving towards integration tests rather than unit tests that assert particular ASTs are parsed from WebIDL files.

webidl: Remove uses of undefined import types

2d552a1

test-project-builder: Add more profiling timers

d432b5b

test-project-builder: Detect when webpack-dev-server fails

43a740a

Instead of going into an infinite loop, detect when webpack-dev-server fails to start up and early exit the test.

webidl: Specify version for dev-dependency on wasm-bindgen-backend

af2aa4f

Instead of only a relative path.

test-project-builder: Build projects in "very verbose" mode

71ec407

This helps for debugging failing WebIDL-related tests.

guide: Add section about contributing to web-sys

efee676

WIP enable Event.webidl

13c93c4

Still need to fix and finish the test.

fitzgen added the frontend:webidl Issues related to the WebIDL frontend to wasm-bindgen label Jul 6, 2018

fitzgen requested review from alexcrichton and ohanar July 6, 2018 22:52

alexcrichton reviewed Jul 7, 2018

View reviewed changes

alexcrichton added 5 commits July 6, 2018 19:35

Update expected webidl output

6c3ed64

Start out a test's status as incomplete

5f671f8

That way if we don't fill it in the error message doesn't look quite so bizarre

Fix onerror function in headless mode

455c102

Otherwise we don't see any output!

Fix package.json/node_modules handling in project generation

a79dd3c

Make sure these are looked up in the git project root rather than the crate root

Avoid logging body text

94529ae

This was meant for debugging and is otherwise pretty noisy

alexcrichton reviewed Jul 7, 2018

View reviewed changes

alexcrichton and others added 6 commits July 6, 2018 22:55

Fix a relative path

b4f0fb5

More expected test fixes

cff9559

Fix a typo

ecf6bdd

Merge remote-tracking branch 'origin/master' into web-sys

384a983

Merge remote-tracking branch 'rustwasm/master' into web-sys

c43d740

test-project-builder: Allow asynchronous tests

3266ced

fitzgen mentioned this pull request Jul 9, 2018

Unforgeable attributes need to be #[wasm_bindgen(structural)] #432

Closed

fitzgen added 5 commits July 9, 2018 12:35

webidl: Convert [Unforgeable] attributes into `#[wasm_bindgen(structu…

16ea94c

…ral)]` Fixes rustwasm#432

test-project-builder: Print generated WebIDL bindings for debugging p…

4040020

…urposes Helps debug bad WebIDL bindings generation inside tests.

When we can't find a descriptor, say which one can't be found

882183b

This helps when debugging things that need to become structural.

web-sys: Test bindings for Event

126207d

ci: Use --manifest-path dir instead of cd dir && ...

c992343

fitzgen added 2 commits July 9, 2018 13:13

web-sys: Just move .webidl files isntead of symlinking to enable them

68478c6

Merge remote-tracking branch 'rustwasm/master' into web-sys

5d7fdea

tests: Polyfill Array.prototype.values for older browsers in CI

6bfeab5

fitzgen changed the title ~~[WIP] Create the web-sys crate mechanically from WebIDL~~ Create the web-sys crate mechanically from WebIDL Jul 9, 2018

fitzgen added 3 commits July 9, 2018 14:02

test-project-builder: Don't panic on poisoned headless test mutex

bc06137

We only use it to serialize headless tests so that we don't try to bind the port concurrently. Its OK to run another headless test if an earlier one panicked.

JsValue: Add {is,as}_{object,function} methods

f7bb74d

Allows dynamically casting values to `js::Object` and `js::Function`.

tidy: Fix whitespace and missing semicolons

ae3b610

fitzgen added 3 commits July 9, 2018 15:11

Allow for dynamic feature detection of methods

05d9eb8

If we create bindings to a method that doesn't exist in this implementation, then it shouldn't fail until if/when we actually try and invoke that missing method.

tests: Do feature detection in Array.prototype.values test

834bfad

Merge remote-tracking branch 'rustwasm/master' into web-sys

8bc01f4

alexcrichton reviewed Jul 9, 2018

View reviewed changes

fitzgen added 3 commits July 9, 2018 15:51

Add JsValue::{is_string, as_js_string} methods

4076f4f

And document all the cast/convert/check methods for js value.

eslint: allow backtick string literals

eeebca9

Only generate a fallback import function for non-structural imports

0a5a50b

fitzgen merged commit f2f2d72 into rustwasm:master Jul 9, 2018

alexcrichton mentioned this pull request Oct 11, 2018

Fix generated shims if APIs don't exist #959

Merged

grovesNL mentioned this pull request Dec 4, 2019

Update WebGPU WebIDL #1896

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create the `web-sys` crate mechanically from WebIDL #409

Create the `web-sys` crate mechanically from WebIDL #409

fitzgen commented Jul 6, 2018

alexcrichton left a comment

alexcrichton Jul 7, 2018

alexcrichton Jul 7, 2018

fitzgen commented Jul 9, 2018

fitzgen commented Jul 9, 2018

alexcrichton commented Jul 9, 2018

alexcrichton Jul 9, 2018

fitzgen Jul 9, 2018

fitzgen commented Jul 9, 2018

Create the web-sys crate mechanically from WebIDL #409

Create the web-sys crate mechanically from WebIDL #409

Conversation

fitzgen commented Jul 6, 2018

alexcrichton left a comment

Choose a reason for hiding this comment

alexcrichton Jul 7, 2018

Choose a reason for hiding this comment

alexcrichton Jul 7, 2018

Choose a reason for hiding this comment

fitzgen commented Jul 9, 2018

fitzgen commented Jul 9, 2018

alexcrichton commented Jul 9, 2018

alexcrichton Jul 9, 2018

Choose a reason for hiding this comment

fitzgen Jul 9, 2018

Choose a reason for hiding this comment

fitzgen commented Jul 9, 2018

Create the `web-sys` crate mechanically from WebIDL #409

Create the `web-sys` crate mechanically from WebIDL #409