Document events on window

[cvan]

Issue by cvan
Monday Feb 29, 2016 at 00:32 GMT
Originally opened as MozillaReality/webvr-spec#15

---

• vrdisplayconnected
• vrdisplaydisconnected
• vrdisplaypresentchange

[cvan]

Comment by cvan
Wednesday Mar 02, 2016 at 03:33 GMT

---

The spec doesn't yet go into depth yet about the events. But the Web IDL is here.

It was brought up earlier (and in this comment by @ashconnell), I actually like the idea of passing detail about the VR display in the event. That way you don't have to look things up on navigator or keep globals references around to the VRDisplays.

I'd like to suggest adding a vrdisplay attribute on the VRDisplay events.

Let's take a look at two examples of other Web APIs that have similar events:

Fullscreen API

FWIW, the Fullscreen API spec doesn't describe any custom event data to be passed.

In the vendor-prefixed, non-Promise-based versions of the Fullscreen API implemented in Firefox and Chrome today, the fullscreenchange and fullscreenerror events emitted do not contain any info about the state change of the Fullscreen mode and on which element. You have to actually look at document.fullscreenElement and compare it to the element whose state you expected to change. It's not the worst thing, but it's not a pattern to emulate IMO.

Gamepad API

The Gamepad API spec defines two events gamepadconnected and gamepaddisconnected, both of which to have a gamepad attribute. It's implemented in Firefox only (not yet in Chrome), and I have to say I very much enjoy the convenience of these events (and the gamepad attributes for easy identification).

[cvan]

Comment by mkeblx
Saturday Apr 02, 2016 at 02:13 GMT

---

When would the most appropriate time to trigger vrdisplayconnected be called in scenario where VRDisplayCapabilities:hasExternalDisplay is going to always be false, like in Cardboard scenario (and currently our in-VR browser)?

[cvan]

Comment by cvan
Sunday Apr 03, 2016 at 20:23 GMT

---

IMO immediately, before DOMContentLoaded and window - I'll let @kearwood and @toji weigh in though

[cvan]

Comment by toji
Sunday Apr 03, 2016 at 21:52 GMT

---

We need to get some more explicit text into the spec about this, but I think in general the pattern seen in the Gamepad API is applicable here. In that API gamepadconnected is fired as soon as a device receives a user gesture, even if it was already technically connected beforehand. Applying that to WebVR, since we're not gating device access on any sort of user action, it seems logical that if a VRDisplay is available when the "vrdisplayconnected" listener is registered it should immediately fire, and thereafter only fire when new devices are connected. Not sure if it should fire every time a listener is added, or only once per page, though. Once per page seems more sane, but also means that libraries you use can "steal" the event from you without you knowing it, simply by registering first.

I'd love @kearwood's input on this as well!

[cvan]

Comment by toji
Sunday Apr 03, 2016 at 21:56 GMT

---

Whoops! Wrong button! Don't mind me!

[cvan]

Comment by cvan
Monday Apr 04, 2016 at 01:12 GMT

---

@toji I like this. I know in Chrome the Gamepad wouldn't show up in navigator.getGamepads() until a user physically presses/changes a button/axis on the gamepad. Firefox, iirc, used to fire gamepadconnected immediately, but I think we changed it to follow Chrome's behaviour of gating on a user gesture. and, from my tests on Windows and Mac, you are correct.

For WebVR, per this discussion in issue #22, if we plan on adding a permission check (using the Permissions API) for querying VR devices, for navigator.getVRDisplays, then that changes things. The first time a user loads the page and the permission hasn't been granted before, the vrdisplayconnected would be emitted only after the permission is granted by the user. Any subsequent visits to that page (or origin?), if the user had previously granted permission to query the VR displays, the vrdisplayconnected event could be emitted immediately.

I like this behaviour because it means I can rely on vrdisplayconnected to handle detecting whether the user has a VR device or the user has given me access to use the VR device (i.e., query the available VR displays). This also makes hot-swapping super easy: I no longer have to call navigator.getVRDisplays() on page load or set up some silly check in a setInterval or requestAnimationFrame loop that handles adding/removing connected/disconnected VR displays based on their properties. That's a lot of boilerplate work.

[S]ince we're not gating device access on any sort of user action, it seems logical that if a VRDisplay is available when the "vrdisplayconnected" listener is registered it should immediately fire, and thereafter only fire when new devices are connected.

@toji: what do you mean "if a VRDisplay is available when the 'vrdisplayconnected' listener is registered"?

Do you literally mean if the developer called navigator.getVRDisplays() and the VRDisplay was in the array that got resolved? I guess I don't understand what that prevents. If the developer calls navigator.getVRDisplays() and we continue to not gate on any user permission or user gesture to call navigator.getVRDisplays(), why not just immediately fire vrdisplayconnected events when the browser detects a VR display?

IMO, connected/disconnected events offer a very nice option for developers to handle VR support and state. It also means developers wouldn't need to call navigator.getVRDisplays() just to usually get the first item from the array resolved (as the single VR display to be used).

In conclusion, I recommend either introducing a permission to the Permission API for "querying VR displays" (i.e., navigator.getVRDisplays, the events, etc.) -or- immediately emitting the vrdisplayconnected events when the browser detects VR displays.

Thoughts?

[cvan]

Comment by toji
Monday Apr 04, 2016 at 15:37 GMT

---

@cvan: What I meant was that the event should fire when the event listener is added (as in: window.addEventListener('vrdisplayconnected', callback, false);) if a VRDisplay is actually connected. Tracking when these event listeners are added will be important anyway for Firefox and Chrome (not so much for the GearVR browser) in that we want defer initializing the VR systems at all until the page indicates it wants to use them. navigator.getDisplays() is one way that the page can indicate it's using VR, adding listeners for these events is another.

[cvan]

Comment by cvan
Monday Apr 04, 2016 at 22:26 GMT

---

Oh, that's what you mean. Gotcha, I like that. I just looked at the Chromium source; I didn't know the gamepadconnected and gamepaddiscconnected events were added back in May of 2014. Sweet!

So, as you know, the Gamepad API implementations in Firefox and Chromium were also slightly different in several ways.

I did some testing, and I found a few more inconsistencies I wasn't already aware of.

If a gamepad is connected and there's a gamepadconnected event listener …

1. Firefox emits the gamepadconnected event immediately on page load
   • notice this this happens without ever calling navigator.getGamepads()
   • notice this is not gated on a user gesture from the user such as pressing a button or moving an axis on the gamepad
2. Chrome does not consistently emit the gamepadconnected event
   • notice this happens likely because navigator.getGamepads() is not called
   • notice even when a user gesture occurs, the event still never (consistently) fires (perhaps worth filing a Chromium bug to address this); if I close the tab and reopen the tab and try again, it sometimes seems to fire - something's off

If a gamepad is connected, navigator.getGamepads() is called, and there's a gamepadconnected event listener …

1. Firefox returns the correct Gamepad objects upon calling navigator.getGamepads() (without any user gesture) and emits the gamepadconnected event immediately on page load (also without any user gesture)
   • same as above
2. Chrome populates the Gamepad objects upon calling navigator.getGamepads() and emits the gamepadconnected event when the first user gesture occurs
   • notice this event is likely emitted because navigator.getGamepads() is also called
   • notice even when a user gesture occurs
3. Upon reloading the page, Chrome populates the Gamepad objects upon calling navigator.getGamepads() but does not emit the gamepadconnected event upon any user gesture
   • notice this event is not emitted even with navigator.getGamepads() and a user gesture (this certainly seems like something worth filing a Chromium bug to address)
   • notice the snapshot of the Gamepad object is returned without an initial user gesture (is this intentional?)
   • notice closing the tab and reopening the tab is necessary for the gamepadconnected event to work again (also seems like a bug)

Anyway, I know there were discussions on the Gamepad mailing list, Bugzilla + Chromium bugs, and a couple spec issues filed for related issues (e.g., w3c/gamepad/issues#4, w3c/gamepad/issues#8, w3c/gamepad/issues#10).

I should probably move this discussion to the w3c/gamepad repo and file Chromium and Firefox bugs. But since we need to follow a similar pattern for the WebVR APIs (navigator.getVRDisplays in particular), we should probably align on a correct way of handling this. Some portions (such as gating on user action) could possibly be non-normative text in the spec, but I'd like for us to at least have a discussion so we don't have implementations that are slightly different causing web developers to have to special case for the different browsers'.

[cvan]

Comment by mkeblx
Wednesday Apr 06, 2016 at 00:13 GMT

---

How big an issue is a library or previous registered handling stealing? Is it important to ensure that all vrdisplayconnected listeners will always be called or is it maybe sufficient that developer shall only have one listener per page for any handling they want to do?

[cvan]

Comment by cvan
Wednesday Apr 06, 2016 at 05:13 GMT

---

Perhaps I'm misunderstanding the issues at hand here, but IMO the browser should engage querying for VR devices when at least one of the following criteria is satisfied:

• a vrdisplayconnected event listener is registered
• a vrdisplaydisconnected event listener is registered
• a meta[name="viewmode"][content*="stereo"] element exists
• navigator.getVRDisplays() is called at any time

[cvan]

Comment by toji
Wednesday Apr 06, 2016 at 23:42 GMT

---

@mkeblx: The case I'm concerned about is a page which relies on "vrdisplayconnected" to show things like an "Enter VR" button. If they register that even fairly late in the page load but some third party library (Modernizr seems like a logical one?) has registered the same event will the library get the event and the users script never see it? In that case it sounds like the right solution is to always call getVRDisplays on startup to get the initial list and then install the listener to see updates. Not a terrible pattern, but I doubt developers will follow it rigorously.

[cvan]

Comment by cvan
Thursday Apr 07, 2016 at 00:45 GMT

---

First, do we ever plan on adding a permission to call navigator.getVRDisplays()? I'd prefer no, at least for this first version.

@mkeblx: The case I'm concerned about is a page which relies on "vrdisplayconnected" to show things like an "Enter VR" button. If they register that even fairly late in the page load but some third party library (Modernizr seems like a logical one?) has registered the same event will the library get the event and the users script never see it?

Is that an issue though? You can have multiple event listeners for vrdisplayconnected, and it'll be fine.

Maybe you aren't talking about exactly a "Modernizr" detection script for WebVR (which I'll file an issue for, btw). But, fwiw, this is how the Gamepad API is detected.

How is this different from having THREE.VREffect or webvr-polyfill listening for vrdisplaypresentchange - and adding a listener myself for vrdisplaypresentchange in my WebVR app too (e.g., A-Frame).

And what if the third-party script calls navigator.getVRDisplays()? How's that any different?

In that case it sounds like the right solution is to always call getVRDisplays on startup to get the initial list and then install the listener to see updates. Not a terrible pattern, but I doubt developers will follow it rigorously.

I don't know - it seems like a roundabout way of identifying intent IMO.

[cvan]

Comment by mkeblx
Thursday Apr 07, 2016 at 01:02 GMT

---

The issue is when the vrdisplayconnected event fires if VR display already connected prior to page load no? If have two listeners registered ideally would have event fire after both but could fire inbetween, not an issue of capture but missing event.

[cvan]

Comment by cvan
Thursday Apr 07, 2016 at 02:22 GMT

---

The issue is when the vrdisplayconnected event fires if VR display already connected prior to page load no?

Per my comment above, Firefox fires the gamepadconnected event when the document loads - even if there are multiple event listeners. I haven't tested all different scenarios, such as having a event listener in the <script> in the <head> and another in the <body>, etc.

Interestingly, the Gamepad API implementation on Windows (xinput) in Firefox actually requires a gamepad gesture (button press, axis move), and only then, will it emit a gamepadconnected event. As mentioned above, the behaviour on Mac doesn't require a gamepad gesture. Heh, weird stuff.

If we do figure out a sane signal for identifying user intent for interacting with the WebVR APIs, we should do the same with the Gamepad API.

I understand gating it on navigator.getVRDisplays(), but it feels dirty from a developer ergonomics POV IMO. The Battery Status API does something interesting. navigator.getVRDisplays() is different in that it returns an array of devices, and we fire the events on window.

I like firing events on window because it enables hot-swapping which saves developers a ton of headache. Does anyone have any other ideas? @luser, perhaps?

[cvan]

Comment by luser
Thursday Apr 07, 2016 at 02:32 GMT

---

So it's kind of late but a couple of points:

1. Chrome does implement the gamepadconnected / gamepaddisconnected events (and has for a while).
2. Firefox has always had the "user interaction to expose a Gamepad to a page" behavior, from the very first prototype patches I wrote.
3. If this isn't working properly on Firefox on Mac that's a bug.

The events are very useful from an API ergonomics perspective--handling them is much simpler than polling to detect new devices. Polling for new data is OK, although for the Gamepad API we would like to spec events for input data as well (and Firefox has non-standard GamepadButton{Down,Up} and GamepadAxisMove events behind a pref).

[cvan]

@toji can we spec for these events to fire detail of the corresponding VRDisplay (à la the webvr-polyfill)?

first mentioned here - but wanted to bring it up again here.

[toji]

I don't see why not.

[cvan]

We should spec when users can expect these events (e.g., vrdisplayactivate) to be fired (e.g., before/after DOMContentLoaded, load, etc.)?

[ddorwin]

One way to do this is using algorithms (rather than prose), including modification of existing algorithms. This can also help identify potential implementation or usability issues. However, the spec doesn't appear to follow this model currently.

[cvan]

I'm going to file a separate issue for the timing, as the events have been documented (nit issue: #83): https://w3c.github.io/webvr/#interface-window