Support scripting complex sets of mock sensor readings

[Honry]

This thread is separated from #369 (Introduce WebDriver Extension API) to discuss about supporting multiple HTTP requests for the use case of simulating a series of updating mock sensor readings. Thanks @jugglinmike for proposing a great idea!

Posted @jugglinmike's comments:
(#369 (review))

Readings Speaking as a web developer, think this API may not be powerful enough to thoroughly test my applications. Unless I am mistaken, the API requires a POST request for each new sensor reading I want to simulate. In many cases, I may not be able (or willing) to issue HTTP requests at an acceptable rate, especially if the remote end is running over the Internet (maybe via a third party like Sauce Labs).

Instead of sending individual readings in a series, what if I could specify a JavaScript function capable of creating those readings? The implementation could call this at the device's sampling frequency, and if it provided a timestamp, I would be able to derive an expected value in a stateless way. Alternatively, we could accept an ES2015 generator function. A generator could yield readings and receive timestamps from the result (or I could maintain the timestamp myself using the performance timing API). In either case, we could use WebDriver's "execute script" command as a guide.

(#369 (comment))

An application developer might write some code to respond to a rapid increase
in brightness. To test their application logic using this new WebDriver API,
they'd need to simulate the increase via a series of POST requests:

POST /session/c0ffee/sensor/ambient-light
{"illuminance": 0}
POST /session/c0ffee/sensor/ambient-light
{"illuminance": 0.2}
POST /session/c0ffee/sensor/ambient-light
{"illuminance": 0.4}
POST /session/c0ffee/sensor/ambient-light
{"illuminance": 0.6}
POST /session/c0ffee/sensor/ambient-light
{"illuminance": 0.8}
POST /session/c0ffee/sensor/ambient-light
{"illuminance": 1}

However, they won't have any control over when those values get applied.
Depending on network effects, this could be not very rapid at all (and not very
smooth as a result). It will also get more cumbersome for more expressive
sensors (e.g. gyroscope) and for more complex readings (e.g. a "twist"
movement). And since all WebDriver commands execute in series, developers won't
be able to interact with the browser while this is happening.

If instead, the automation API allowed developers to specify a function, they
could express an entire series of sensor readings that was arbitrarily complex
and continuous, and they could do it with a single POST request.

Here's how it might look with a "classic" function body that received the time
offset as its first argument:

POST /session/c0ffee/sensor/ambient-light
{
  "body": "const t = arguments[0]; return {illuminance: Math.min(t / 1000, 1)};"
}

And here's a fancier version using an ES2015 generator function that tracks the
time with its own internal state:

POST /session/c0ffee/sensor/ambient-light
{
  "body": "const start = performance.now(); while (true) { const t = performance.now() - start; yield {illuminance: Math.min(t / 1000, 1)}; }"
}

The latter is more verbose for this example, but the ability to maintain state
could make it much easier to script complex sets of readings.

(#369 (comment))

How about keep single set of readings as current solution, and extend this
section by using WebDriver's "execute script" command to script complex sets
of readings?

That could be a nice simplification, but I don't think the current text
supports it. There doesn't seem to be an JavaScript API to control this, so
the code that developers provide via "Execute Script" doesn't have an interface
to specify mock values.

We could define such a JavaScript API, but that might not be necessary.
Developers may stub out the JavaScript constructor of the device they wish to
mock. Their stub constructor could generate instances which emitted events in
whatever sequence they liked. This just requires injecting code prior to the
evaluation of their application. For some configurations, that could be a
little cumbersome but not impossible. I've filed an issue against WebDriver to
discuss making this easier.

If that is a tenable solution, then maybe web developers aren't an audience for
this feature, after all! What do you think?

[foolip]

This is a similar problem as with pointer device input, which in https://w3c.github.io/webdriver/#actions is solved by having a notion of ticks and pausing to send a whole series of updates in one command.

Something similar might work for sensor readings, given how similar the data flow is.

[foolip]

A potential problem with having JS generate sensor readings is where that script will execute. To not make assumptions about where the sensor readings are being inserted, it may be necessary to isolate that script similar to how worklets are defined: https://drafts.css-houdini.org/worklets/#worklet

https://webaudio.github.io/web-audio-api/#audioworklet is a bit similar to what might be needed, although that's a web-exposed API.

[jugglinmike]

@foolip could you comment on the need to address this in WebDriver? I'm starting to think that the "browser implementer" use case is addressed by @Honry's patch and that the "web app developer" use case is addressed by pure JavaScript mocks. Are there any situations where one of those two approaches will be insufficient?

[foolip]

Since Sensor doesn't have a constructor it won't be possible to perfectly mock the API for a web developer, but I believe it will be possible to mock out sensors well enough to run any tests that are verifying the behavior of a web app as opposed to the browser itself.

Whenever possible I think it'd be good to serve the testing needs of web developers with WebDriver as well, but this is a case where doing so would add significant complexity. So, my inclination would be to not tackle that unless requests from real web developers attempting to use the existing proposed API come in.