consider Client behavior for windows where initial about:blank is replaced with a loaded document

[wanderview]

Our current plan is to expose the environment creation URL as Client.url when the environment is marked execution ready. This is a bit confusing for the case where the environment has multiple documents; like when an initial about:blank is replaced with a loaded document. See:

whatwg/html#2456

My read of the spec and my initial implementation so far expose the about:blank URL briefly as Client.url. In addition, the environment is marked execution ready twice. Once as about:blank and once as the final loaded URL. The second time its marked execution ready this basically amounts to updating the creation URL.

Is this intended behavior for the clients API? I suspect perhaps the intent was not to mark the Client as non-reserved until after the final loaded document is execution ready.

[wanderview]

I suspect perhaps the intent was not to mark the Client as non-reserved until after the final loaded document is execution ready.

Talking with @bzbarsky, it may be very hard to determine if an initial about:blank will indeed be followed up another document. And I don't think we can just ignore about:blank completely because programmatically constructed iframes are supposed to inherit their parent's service worker controller, etc. We need to represent them as Client objects.

[jakearchibald]

In this iframe / window.open / _blank with empty url case where replacement can happen:

• client: the about:blank doc
• reserved client: null
• target client: null

Following the navigation, the client will be the same as the about:blank document.

[wanderview]

@jakearchibald, I'm kind of confused by #1091 (comment). I don't recall this being the outcome from the f2f meeting.

I thought we said we were going to do:

• client: the client initiating the load just like we do for non-replacement cases
• reserved client: the client currently attached to the about:blank window
• target client: I don't remember, but the about:blank window would make sense to me.

If you don't want the existing about:blank to be "reserved" since its already execution ready, we could also do:

• client: the client initiating the load just like we do for non-replacement cases
• reserved client: null
• target client: I don't remember, but the about:blank window would make sense to me.

So in:

// foo.html
let f = document.createElement('iframe');
document.body.appendChild(f);

f.contentWindow.someGlobalStatus = 'yuck';

f.src = 'frame.html';

I guess I would always expect FetchEvent.clientId to reference the foo.html parent window here.

It also seems to make sense to make FetchEvent.targetClientId reference the current about:blank iframe client.

I can go either way about whether FetchEvent.reservedClientId should be null or the iframe client. I guess from an implementation point of view null would be slightly more convenient for me.

@smaug---- also was wondering what we do if replacement load is for a download. The example he gave was:

win = window.open(); win.document.body.innerHTML = "<a href='http://download.me.com/foo.zip'>zip</a>"

In this case, if someone clicked the link I would expect:

• the .clientId to be the about:blank window (since its an anchor tag),
• the reserved client to be null (if we decide to go that way above)
• the target client to be the about:blank window since we don't know yet if it will download

If it was <a download> then I would expect the target client to be null.

[wanderview]

Although, I still wonder how devs can use this API if we don't provide a consistent place for them to find "the client that will exist if this navigation succeeds". If we don't provide the about:blank initial window as the reserved client, then devs have to do some complicated logic to determine they are in this weird replacement case.

[wanderview]

I read through the meeting minutes and it seems we did not really come to conclusion here.

I need to do something for our implementation, though, so I'd like to make a sketch proposal we can discuss at the next meeting. Hopefully I will have something implemented behind a pref so we can try it out.

My idea is to be more explicit about the replacement case. Instead of putting the about:blank client ID in the FetchEvent.reservedClientId we expose it in a new FetchEvent.initialClientId.

I took the term "initial" here from the "initial about:blank document" used in the HTML spec. I also considered replacementClientId, but I think that's confusing because in other navigations the targetClientId is usually being replaced by the reserved client. When there is an initial about:blank we are actually not replacing the client, but reusing it. The document is getting replaced (but a client is not the document).

So for this code snippet:

// foo.html
let f = document.createElement('iframe');
document.body.appendChild(f);

f.contentWindow.someGlobalState = 'yuck';

f.src = 'frame.html';

The FetchEvent would have:

• clientId = foo.html window
• reservedClientId = null
• initialClientId = about:blank iframe window
• targetClientId = about:blank iframe window

After frame.html loads it has the same client id as the initial about:blank iframe.

[wanderview]

By the way, about:blank replacement windows are in theory already exposed, even without reserved Client ID. Consider this page:

<iframe src="frame.html" id="frame"></iframe>
<script>
var f = document.getElementById('frame');
f.contentWindow.foo = 'foo'
</script>

here the .foo global is set on the initial about:blank window. This window is then reused when "frame.html" loads. The "frame.html" page will see .foo on its global.

Consider if "frame.html" is delayed in loading. Perhaps a service worker purposefully does this. In this case:

1. A clients.matchAll() should theoretically return a Client for the initial about:blank before the load completes because it is same origin.
2. A later clients.matchAll() should then return the same Client for the "frame.html" window (because it got reused).

I'm intending to write a test for this, but I'm unsure if I should upstream it to WPT yet or not. What do people think?

[jungkees]

I'm intending to write a test for this, but I'm unsure if I should upstream it to WPT yet or not. What do people think?

IMO, we might want to put a 'do not merget yet' tag on the test until we come to a conclusion. Or, you can just share the test through the firefox code search if that's more convenient?

[wanderview]

I'll post the test once I have it written. Its revealing more problems for me that I'm trying to work through.

One thing that I have noticed is that service workers may have exposed some browser optimizations that were not observable before.

Consider loading a simply <iframe src="foo.html"></iframe>. Per the spec this should create an initial "about:blank". At least in gecko, though, we don't actually do this since nothing tries to access .contentWindow. This allows us to save some work when nothing has interacted with the frame before the load completes.

With service workers, though, we can do this:

1. Intercept foo.html and delay the response
2. Do a clients.matchAll() to observe the initial about:blank

In this case our optimization to avoid creating the initial about:blank is exposed because step (2) won't find a Client. At least with our current Clients implementation anyway.

I'm trying to make this work without forcing a de-opt in the general case. I'm trying to make our code create the initial about:blank on demand when clients.matchAll() would have observed it.

[jungkees]

@wanderview, I think I should catch up a bit but in #1091 (comment):

clientId = foo.html window

clientId for the snippet should be the about:blank client. clientId in my mental model is the request's client's id. When the src attribute of the iframe's set, foo.html's nested browsing context is navigated instead of the top-level browsing context. See https://html.spec.whatwg.org/#otherwise-steps-for-iframe-or-frame-elements step 4. That is, the request passed to fetch will use the iframe's settings object as the client.

I'm a bit concerned about introducing initialClientId. It seems it goes too much complex for corner cases. Coming back to the definition we originally thought:

• client: request's client
• reserved client: literally reserved client for main resource
• target client: to-be-replaced client

For normal case (i.e. navigate from non-initial document), they seem to be apparent.

For replacement case, I think they should be:

• client: about:blank
• reserved client: reserved environment for frame.html request (this is different from the f2f decision)
• target client: null (or event about blank?)

As this is the case where the existing global (and the environment settings object) are legitimately reused, the reserved client being removed and the client being the final client for the resource sounds good to me.

FYI, whatwg/html#2080 is the related PR that I should revisit now.

[jungkees]

FYI. I tested the snippet in #1091 (comment) on Chrome. It doesn't seem to reuse the existing window of about:blank document. The state set to the global object is lost after the navigation.

[wanderview]

FYI. I tested the snippet in #1091 (comment) on Chrome. It doesn't seem to reuse the existing window of about:blank document. The state set to the global object is lost after the navigation.

Yes, I did some compat testing and @bzbarsky pointed out the only real use where browsers maintain compat is this:

<!doctype html>
<!-- top.html -->
<html>
<body>
<!-- declaratively defined iframe -->
<iframe src="nested.html" id="nested"></iframe>

<script>
// Modify the iframe's global synchronously before it loads.  The resulting nested.html
// iframe will have its `window.foo` property set even though we are applying it to
// initial about:blank document here.
let frame = document.getElementById("nested");
frame.contentWindow.foo = "foo";
</script>

</body>
</html>

What do you think the various properties on FetchEvent should be for this situation?

[jungkees]

That's right. I confirmed by testing Chrome reuses the global in the case shown in your previous comment.

What do you think the various properties on FetchEvent should be for this situation?

To my understanding, the declaratively-defined-iframe case and the snippet in #1091 (comment) go through the same flow according to the spec. I think the properties on FetchEvent should be the same. That is:

• client: about:blank
• reserved client: reserved environment for nested.html request (this is different from the f2f decision)
• target client: null (or even about:blank?)

So, devs will see two different clients (about:blank and reserved env) exist during the fetch, which is what's happening indeed. And when the resulting document is installed, the about:blank client is reused and the reserved client is abandoned. (If the navigate found a matching active service worker, that property should be overwritten to the client.)

[wanderview]

reserved client: reserved environment for nested.html request (this is different from the f2f decision)

Do you mean a new reserved client that is different from the about:blank initial document client? Which client does the final document end up with?

Either way you are suggesting that the client ID for a global change during its lifetime. That seems like a bad invariant to break.

And all of this is observable to the service worker. Consider this:

<!doctype html>
<!-- top.html -->
<html>
<body>
<!-- declaratively defined iframe -->
<iframe src="nested.html" id="nested"></iframe>

<script>
let frame = document.getElementById("nested");
frame.contentWindow.navigator.serviceWorker.postMessage(evt => {
  // do stuff and post back to the service worker
  evt.source.postMessage(someMsg);
});
</script>

</body>
</html>

A clients.matchAll() can get and postMessage() to the initial about:blank. Of course the final window is also accessible as well. I think the service worker could get confused if the MessageEvent.source.id changes over time for the same global.

I understand this is complex, but this is the cost of exposing the browser's underlying window/global model via a primitive API like Clients. The about:blank replacement complexity is already there. The primitive needs to reflect that to be honest about what is really happening.

[jungkees]

The primitive needs to reflect that to be honest about what is really happening.

I agree. Just want to make sure it doesn't get more complex than needed.

Do you mean a new reserved client that is different from the about:blank initial document client?

Yes.

Which client does the final document end up with?

The about:blank client.

Either way you are suggesting that the client ID for a global change during its lifetime. That seems like a bad invariant to break.

The client IDs aren't changing. In the fetch event, devs will see both the client and the reserved client, but the reserved client will be gone away when the navigate completes in this case. I mean only the about:blank client with its initial ID will be alive. I think it isn't odd as that's what happens.

In this case, the reserved client is just used during the main resource fetch for matching a service worker, queueing postMessages, etc. I think the messages sent to the reserved client should be requeued to the about:blank client when the document is installed.

[wanderview]

The client IDs aren't changing. In the fetch event, devs will see both the client and the reserved client, but the reserved client will be gone away when the navigate completes in this case. I mean only the about:blank client with its initial ID will be alive. I think it isn't odd as that's what happens.

How can the service worker tell whats going to happen here? Does it have to execute code later to test to see which Client ended up being the final window? This does not seem simpler to me.

In this case, the reserved client is just used during the main resource fetch for matching a service worker, queueing postMessages, etc. I think the messages sent to the reserved client should be requeued to the about:blank client when the document is installed.

I'm sorry, but this seems a lot more complicated than just providing the correct Client in the first place.

As an implementer I'd like to keep the Client behavior as close to actual window creation behavior as possible. It reduces special cases in an already complicated part of the system.

I'm not a web developer, but it seems to me that they would probably like to have a property they can reference that they can be certain will in fact become the final Client.

I'd really like to just put the initial Client ID in the FetchEvent.reservedClient property if we're in this about:blank replacement case, but early postMessage() is a problem. Maybe we need some kind of "delay this message until the current load completes" API on postMessage.

[wanderview]

Or maybe this all suggests we should avoid exposing reserved Clients at all. Its pretty complicated and perhaps not worth the hassle.

@jakearchibald, can you refresh my memory as to why we needed reserved Client ID? Can we get the same thing another way without actually referencing not-quite-there-yet Clients?

[wanderview]

Here is my test case for initial about:blank handling in today's service worker spec:

web-platform-tests/wpt#6304

None of this requires reserved client ID, etc.

[jungkees]

@wanderview , thanks for sharing the tests! Both Firefox and Chrome respond with 'failure: could not find about:blank client' for all the cases. As you pointed earlier, the about:blank window either isn't created yet or not exposed to matchAll in some reason at least.

@jakearchibald, can you refresh my memory as to why we needed reserved Client ID? Can we get the same thing another way without actually referencing not-quite-there-yet Clients?

In my memory, the major use case was to open up an ability to manage caches per-client basis. A reservedClientId and a targetClientId can give a clue to create and delete caches, respectively. Another use case was messaging. I don't recall anything else.

Let's start with what we agree on first. The final client should be the about:blank client. Right? That client is created when the nested browsing context is created and later reused for the fetched document.

In #1091 (comment), you said clientId should be the top-level client (foo.html in the example.) But I think it should be that of the about:blank client. The reason is the fetch request's client is the about:blank client. Do you agree?

Beyond that, what reservedClientId should represent here and whether we'll introduce initialClientId are the further discussion points.

/cc @jakearchibald

[wanderview]

@wanderview , thanks for sharing the tests! Both Firefox and Chrome respond with 'failure: could not find about:blank client' for all the cases. As you pointed earlier, the about:blank window either isn't created yet or not exposed to matchAll in some reason at least.

Correct, but I think this behavior is what we've agreed on in the spec so far, right? This test passes in firefox with the patches I have in progress at:

https://bugzilla.mozilla.org/show_bug.cgi?id=1293277

Let's start with what we agree on first. The final client should be the about:blank client. Right? That client is created when the nested browsing context is created and later reused for the fetched document.

Agreed.

In #1091 (comment), you said clientId should be the top-level client (foo.html in the example.) But I think it should be that of the about:blank client. The reason is the fetch request's client is the about:blank client. Do you agree?

Honestly I'm not sure any more. I thought it would have used the client for the global of the script that made the change, but maybe this has changed recently with @domenic's changes.

In any case, I agree it should be some existing Client which we determined to have "caused" the navigation to start.

Beyond that, what reservedClientId should represent here and whether we'll introduce initialClientId are the further discussion points.

Right.

And the main reason I see for not just putting an initial about:blank client in reservedClientId is the issue with postMessage() not being queued until the final load like you get with a real reserved client.

I guess if we made real reserved clients not queue message events, then this problem would go away.

[jungkees]

Correct, but I think this behavior is what we've agreed on in the spec so far, right?

Yes. I think so.

Honestly I'm not sure any more. I thought it would have used the client for the global of the script that made the change, but maybe this has changed recently with @domenic's changes.

IIRC, it wasn't changed.

In any case, I agree it should be some existing Client which we determined to have "caused" the navigation to start.

I think the initial about:blank case isn't different from other cases. Setting src (both declarartively or from script) triggers a navigate of the nested browsing context. The nested browsing context's active document's client being the request's client seems reasonable.

And the main reason I see for not just putting an initial about:blank client in reservedClientId is the issue with postMessage() not being queued until the final load like you get with a real reserved client.

I'd like to clarify if we were on the same page about this. I didn't mean to put an initial about:blank client in reservedClientId. It was about whether to expose the reserved environment made anyway for the http fetch of the navigate request (because all the normal navigate fetch would expose the reserved environment.)

It seems we're back to the f2f decision considering the about:blank is to be the final client and exposing the reserved client is too complex. For this, devs should be able to detect it's the initial about:blank case. I now better understand why you proposed initialClientId. I was thinking about using clientId instead of adding a new property like:

clients.get(fetchEvent.clientId).then(client => { /* client.url == 'about:blank' && it's a navigate of nested browsing context */ });

Hmm.. I think we still can't distinguish it from a non-initial about:blank case if the session history has other documents in the list. E.g. [document1, document2, about:blank]. So, should we add initialClientId?

[jakearchibald]

Finally caught up with this, sorry for not pitching in sooner.

In the case of:

1. New iframe (initially about:blank).
2. Navigate iframe to same origin URL (by changing the location or src).

From reading https://html.spec.whatwg.org/multipage/iframe-embed-object.html#process-the-iframe-attributes, it seems clientId should be the client that represents the about:blank iframe, even if it's the src attribute being changed.

I don't like using reservedClientId to represent a client that already exists, but my objection is only in the naming. I'm happy to drop reservedClientId and replace it with @wanderview's initialClientId, which (unless I'm mistaken) represents "The client ID of the resulting navigation/worker", which may already exist like in the about:blank case. I'm still not convinced of the naming though. resultingClientId anyone?

targetClientId should be the about:blank client. I was wrong to suggest it should be null. In case it matters: I was stuck on the idea that targetClientId represents a client that would go away if reservedClientId comes to life, but I don't think this needs to be the case.

[jakearchibald]

In summary:

• clientId - the client that initiates the request.
• resultingClientId - the potentially-reserved client that will house the resulting document/worker.
• targetClientId - an existing client that will be replaced, or have its document replaced.

Btw, I can't get the replacement thing to happen in any browser other than Firefox.

• Iframes http://output.jsbin.com/bifelu/
• window.open http://output.jsbin.com/rofecit/quiet

If other browsers are replacing the global when navigating about:blank to a same-origin URL, then that difference is going to show up here too right? As in, Firefox will have the same ID in resultingClientId and targetClientId, but in Chrome they'll be different (unless/until Chrome changes replacement behaviour).

We maybe need to stress in the tests that it's more important that the client IDs reflect what the browser does, rather than simply pass the tests. As in, resultingClientId shouldn't equal targetClientId if the navigation is going to create a new global.

[wanderview]

Btw, I can't get the replacement thing to happen in any browser other than Firefox.

We talked about this in IRC, but for those following along at home, browsers do have some compat here. Its narrowly confined to the particular case where an iframe is statically defined in the document and then an inline script modifies the global before it loads.

Cases where you try to dynamically create the iframe have a variety of compat issues:

1. Firefox marks documents as no longer initial after their load event fires. So if you wait for load before modifying the global, then the global won't be reused.
2. Chrome only seems to reuse the global if the .src attribute is set before the iframe is added to the document.

See the scrollback in this IRC conversation with @bzbarsky for some more discussion:

http://logs.glob.uno/?c=mozilla%23content&s=14+Jun+2017&e=14+Jun+2017#c447305

[wanderview]

I think we need to figure out our postMessage() plan if we're going to store initial and reserved in the same property.

[jakearchibald]

More details from IRC:

As @jungkees said, one of the main use-cases is messaging. The developer should be able to send a message to the resulting page as part of a request of mode navigation.

We handle this with reserved clients by buffering messages until the client is created & the port is started (either explicitly by calling startMessages(), implicitly by assigning to onmessage, or after DOMContentLoaded).

However, this won't work with iframes (or window.open) because it has already loaded with about:blank, and the new document will load into that global. The port has started way before the document loads.

Even if we make initial & reserved different things, we need to solve this for the initial case.

[jakearchibald]

https://html.spec.whatwg.org/multipage/parsing.html#the-end

We currently open the client message queue after DOMContentLoaded. What if we avoided doing that for these initial windows?

For initial windows, we could open the client message queue after the load event (that would fit with Firefox's behaviour right @wanderview?) or expect developers to do it manually using startMessges or assigning to onmessage.

[bzbarsky]

Note that the Firefox behavior is not HTML-spec-compliant (nor is anyone's else's, in various different and incompatible ways), and if/when browsers update to follow the current spec that approach will break.

So you either need to come up with a design here that works with the HTML spec behavior or get the HTML spec changed in some way or something.

[jakearchibald]

@bzbarsky which bit breaks the spec? Opening the queue on the load event? Yeah, I'm happy to ditch that idea and expect developers to open the queue via other means if they want it to work for initial about:blank pages.

[wanderview]

For initial about:blank documents, don't open the client message queue unless startMessages() is called, or there's an assignment to navigator.serviceWorker.onmessage. For non-initial documents, open the client message queue after DOMContentLoaded (as is spec'd now) unless it's already open.

@jakearchibald, would this apply only to Client.postMessage() or are you talking about changing window.postMessage() for initial about:blank also?

Changing Client.postMessage() only seems ok to me I guess.

[jakearchibald]

@wanderview yeah, I'm only talking about client.postMessage(). Don't think we can change window.postMessage.

[jakearchibald]

I've updated web-platform-tests/wpt#6343 will the new names.

I guess the next step is to add the iframe edge cases.

Is there any point adding window.open tests given they'd need to be manual?

[wanderview]

Is there any point adding window.open tests given they'd need to be manual?

Does this really require manual tests? I have a window.open() test in web-platform-tests/wpt#6304 and it seems to work.

[mfalken]

Chromium's test runner seems to allow window.open() to work by default, too.

[jungkees]

@annevk, I'm trying to find a way during the Handle Fetch to see whether the navigate request is from the initial about:blank client. We want to set .resultingClientId to the initial client's id in this case. (Please read from #1091 (comment) for background.)

I couldn't come up with any good way other than thinking about something similar to https://html.spec.whatwg.org/#initialise-the-document-object step 1:

If browsingContext's only entry in its session history is the about:blank Document that was added when browsingContext was created

But I don't think that's a proper way as it'd reference the request's client's browsing context from a task in the SW's event loop. Would that be okay?

Adding a state to an environment settings object to tell it's tied to the initial about:blank Document seems to be an option. But I know we don't want to add one if that's avoidable. Do you have any idea?

[annevk]

If you have a handle on an environment settings object you can get to the "corresponding" document, no? I do think we want to add state there about initial about:blank as currently the definition is a little lacking, but that still requires a lot of refactoring to happen first (which I'm working on, but it's taking a long time).

[jungkees]

If you have a handle on an environment settings object you can get to the "corresponding" document, no?

Yes, but without a state on the environment settings object, I should still cross the process boundary to check if that document is the only entry in the client's browsing context's session history.

Also, after reading whatwg/html#2774 (comment), I'm concerned about accessing such a state defined in the environment settings object from within parallel steps. We should be able to make sure that the state hasn't been mutated. When the navigate flow initiated from the initial about:blank client got to Handle Fetch, isn't there any possibility the browsing context's session history had changed? I believe the navigate algorithm has guards against multiple navigate attempts, but want to clarify we won't have a race condition.

[annevk]

Yeah, if you're in parallel you can't access environment settings objects either. You'd need to propagate that state or message pass it across. (We really need "threads" rather than "in parallel" to describe these things better.)

[jungkees]

I think introducing the thread concept wouldn't solve this particular case where we want to reference the environment settings object's item concurrently from the main thread and the SW thread. (I absolutely agree to the idea of spec'ing the thread concept to solve the ambiguity around designating parallel execution context.)

I believe the navigate algorithm has guards against multiple navigate attempts

I think if we add something like the environment settings object's initial client flag, we wouldn't have a problem accessing the flag from Handle Fetch as https://html.spec.whatwg.org/#navigate step 3, 4, and 6 don't allow two navigate instances to run concurrently.

[jakearchibald]

F2F:

• clientId - made the request
• resultingClientId - becomes the client
• replacesClientId - replaces some client

Implementation priority is resultingClientId. Action: Check that returning null for the others doesn't land us with problems in future.

Action: Look at worklets – can a single client fetch create multiple clients. Pretty sure we decided they were subresources.

[jungkees]

#1245 will track the work decided in this thread.