Cross-origin iframes works on Safari/iOS, but not on Chrome/Android

[RSpace]

According to the readme, Chrome for Android should support cross-origin iframes when allow="gyroscope; accelerometer" is set on the iframe. That's not the case, though, it appears the iframe is not able to access the features it needs.

In Safari on iOS, postMessage is used to send the motion events, so here it works fine in a cross-origin iframe.

Here's an example - it's simply the example from here put on a separate domain to get cross-origin:
https://cimmerse-iframe-tester.glitch.me/

Since devicemotion is only used on iOS, and Chrome uses the RelativeOrientationSensor API, I can't fix the problem by simply also passing the motion events via postMessage in Chrome also. The documentation on https://developers.google.com/web/updates/2017/09/sensors-for-the-web doesn't specifically mention what features/permisssions need to be set on the iframe for the RelativeOrientationSensor API cross-origin, but it appears that allow="gyroscope; accelerometer" is not enough.

What's the fix?

[jsantell]

This works for me for both Sensors and devicemotion (when I disable Sensors in chrome://flags) in Chrome 67 on Android. What Chrome version are you using?

[RSpace]

I'm also in Chrome 67 for Android, but I've realized I'm seeing quite inconsistent behavior. Sometimes it works as expected, sometimes there is no motion detected and the scene is just still regardless of how I rotate the phone. If I restart Chrome, it usually fixes the problem. So I guess that's unlikely to be due to a bug in this code base?

One thing I don't understand is; how can this work in Chrome even with Sensors disabled, since there's all of these explicit iOS checks in the code?

[jsantell]

When Generic Sensors API are disabled, the logic falls back to devicemotion -- not sure what issue you're running into, if you could dig into some of the events you're seeing that could narrow it down

[RSpace]

Let me clarify: In the example, in Chrome 67 on Android, the motion works fine until I go into fullscreen. Then it stops working, and I have to restart Chrome to get it working again.

In my own, much more complex code base (A-Frame based), the motion doesn't work in Chrome on Android at all when embedded into a cross origin iframe, not even before going to full screen. Also here, it works fine on iOS.

[RSpace]

In my own code, I am occasionally able to get a single reading before I stop receiving additional readings. There seems to be certain events, full screen one of them, that stops the RelativeOrientationSensor from receiving additional readings in a cross-origin iframe. Since this is a very new API, I suppose this could be a bug in Chrome?

[jsantell]

Currently seeing this fail with sensors on in Chrome 67.0.3396.8, in magic window and VR mode.

[jsantell]

Seems that the Sensor stops reporting updating the values once you interact with the scene. Guessing it has to do with focus and permissions. I'll ask around.

[jsantell]

Related cross origin iframe issues with sensors: https://bugs.chromium.org/p/chromium/issues/detail?id=849501, and focused-area definition of the permission model: https://www.w3.org/TR/generic-sensor/#focused-area

[billorr-google]

On Chrome 69, if the iframe is focused, it should get data. If the iframe is not focused it won't get data despite allow flags unless it is the same origin as the outer page. Some kind of a postmessage could be used to plumb data into the frame if the outer page is focused, but we should clarify what the webxr spec behavior is and match that.

On Chrome 67, the logic was wrong, so the iframe wouldn't get poses when focused if cross-origin - only if the outer frame had focus. I believe this regression is in Chrome 68 as well.

[jsantell]

As per the spec, iframes must have focus in order to receive sensor data. This logic was incorrectly reversed in Chrome 67, 68 (newer versions of 68 may have this fix?). As of Chrome 69, it works as intended upon selecting the iframe

@billorr-google beat me to it

[RSpace]

@jsantell @billorr-google Thanks for the clarification. Since Chrome 69 is a few months out, is this something we should work around by removing the isIOS check in https://github.com/immersive-web/cardboard-vr-display/blob/master/src/sensor-fusion/fusion-pose-sensor.js#L292?

Personally, I'd like to see this as a permanent change, so the outer page can always choose to post data into the frame, and so we can provide similar experiences on both Android and iOS even when the iframe is not focused.

I can do a PR to remove the isIOS check if you agree?

[jsantell]

@RSpace that's for the fusion pose sensor which uses devicemotion -- the newer component that handles this uses the Generic Sensor, which doesn't have the postMessage hook, so that's where it fails: https://github.com/immersive-web/cardboard-vr-display/blob/master/src/pose-sensor.js

that being said, I think a faster, temporary solution for this regression may be forcing the devicemotion codepath, which doesn't have this iframe issue. You can call it by:

vrDisplay.poseSensor_.useDeviceMotion()

If this works for you, I'll update the iframe docs on the README as well as the example with context. Let me know!

[jsantell]

Just tested this out with the local iframe example (and ensuring both frames are different origins) -- updating the caveats in the README with this bug and workaround

[RSpace]

Thank you @jsantell and sorry for not getting back with my own test results. Been away a little while.

[jsantell]

@RSpace no worries, let me know if this doesn't solve the issue for you