More precise control over app window features

[mgiuca]

This is an umbrella issue that keeps coming up over and over over the past few years. It's been discussed in small pockets but I don't think there is a dedicated Manifest issue tracking it, so here is one.

The problem

Broadly speaking, the problem is this: display mode and its associated fallback chain does not give developers enough control over the app window, and it's too inflexible to allow us to build new features on top of. This has manifested in a number of separate issues:

1. When we want to add a new feature, that "looks like" a new display mode, it's unclear how to fit it into the fallback chain. This came up for tabbed application mode (Add a "tabbed application" mode #737) in 2018 (still WIP), and now title bar customization ([Title Bar Customization] Should "caption_controls_overlay" be swapped for a new "display" mode? MicrosoftEdge/MSEdgeExplainers#206). By adding new display modes for new features, it's impossible for sites to mix and match these features.
2. Each display mode is poorly specified and inconsistent across implementations (Display mode "standalone" inconsistently handled by browsers #673). Developers have no guarantee about what controls they're actually going to get. It's nice in theory to give control over window UI to the user agent, but in practice, it means developers will build with the most popular browser in mind and create de facto standards (for example, if popular browsers add a back button to display: standalone, sites will assume it exists). Because of this, we've needed (and have not yet succeeded) to provide ways to query whether these controls are being provided by the user agent (Add a way to query whether there is a UA-provided back button #693).
3. The fallback chain is inflexible. Some sites may want minimal-ui falling back to a browser tab. But other sites may only have a slight preference for minimal UI, and be happy to fall "up" to standalone if the user agent doesn't support minimal.

On the first issue, a very similar discussion happened in #737 and MicrosoftEdge/MSEdgeExplainers#206. Do we add the new display mode and slot it into the linear fallback chain (e.g., between fullscreen and standalone?) Or do we break the linearity of the fallback chain, so fullscreen still falls back to standalone whilst my_new_feature also falls back to standalone? Or do we add the new feature as a separate Boolean member alongside display?

Having fullscreen fall back to tabbed mode, or even title bar customization mode, is very problematic, since both of those modes are somewhat special and really should require explicit opt-in from developers. The non-linear fallback chain solves this, but it means you can't mix and match (for example, there would be no way to have a tabbed application with a customizable title bar).

On the third issue, I believe (maybe I'm wrong [Edit: I was]) that Chrome just recently became the first major browser to actually support the long-time-specced display: minimal-ui. [Actually, Firefox supports it.] Historically, minimal-ui in Chrome has basically been equivalent to browser. Encouraging devs to adopt minimal-ui is difficult, since you're basically foregoing "being a PWA" in other browsers that don't yet support this mode.

Proposed solutions

Several solutions have been proposed. I'll try to aggregate them here.

Add new features as separate members

This is the most straightforward. We keep the existing display modes, but don't add any new ones. All new features are "add-ons" which have their own member. For example:

{
  "display": "standalone",
  "tabbed_mode": true,
  "caption_controls_overlay": true
}

This solves issue 1, but not 2 or 3, and could result in a lot of new top-level members.

Allow sites to specify an explicit fallback chain

Something that @dmurph proposed in a private chat with me this morning. Allow display to be either a string or a list of strings; the new list form allows developers to explicitly specify their fallback chain. So now, I can have "display": "minimal-ui" if I want minimal, falling back to browser, or "display": ["minimal-ui", "standalone"] if I have a slight preference for minimal, but definitely want some form of standalone window.

My suggestion was that new features can only be specified in the list form, and the list must include at least one of the four legacy modes. So "display": "tabbed" would be illegal, as would "display": "tabbed", but you could have "display": ["tabbed", "standalone"] (if you want tabs in your app window but don't mind if it's not supported, or "display": ["tabbed", "browser"] (if you absolutely must have a tabbed UI, and if not supported, let the user use their browser tabs).

This partially solves issue 1 (but doesn't allow mixing and matching new features), and solves issue 3, but not 2.

Media query for individual elements like back button

Proposed by me in #693 and explainer written by @fallaciousreasoning here. This doesn't outright solve any of the above issues, but at least gives developers a way to detect which elements are being shown to the user, partially solving #2 above.

This solution is probably necessary in addition to other measures, in order to feature detect browsers that have / have not implemented other measures.

Add a display-modifiers member

This was proposed by @amandabaker on MicrosoftEdge/MSEdgeExplainers#206, and riffed on in comments by @aarongustafson. Keep the existing display-mode, but add a new list or dictionary member, display-modifiers, which lets developers add or remove things piecemeal.

For example, Aaron's proposal presents the new title bar customization feature as simply a "removal" of the "titlebar" feature:

{
  "display": "standalone",
  "display_modifiers": {
    "titlebar": false
  }
}

But this also allows you to explicitly add or remove individual elements like back button, refresh button, etc. I think these would still have to be hints to the UA (we can't mandate that the UA show a refresh button, for instance), but they could be SHOULD requirements.

This would solve 1 because you could choose any of the four existing display modes (or at least choose between standalone and minimal-ui, and then add or remove new features).

This would partially solve 2, because even though display would still be poorly specified (and up to UAs which features they imply), the developer could precisely ask to have, or not have, the things they care about.

This would solve 3 because you can essentially make your own fallback chain. If you want minimal UI but if unsupported, fall back to a standalone window, then you should do display: standalone with a bunch of modifiers to make a bespoke minimal-ui.

I like this solution. One nice thing about it is that it's backwards compatible: browsers that don't implement display_modifiers will still make a best effort to give you what you asked for.

I've proposed a similar solution before (can't find a source), but my solution has a slightly different slant: instead of display "modifiers", we design as if display didn't exist: create a new bag of window_controls (or whatever), which defines all the windowing properties, and then respecify display as simply a preset setting for window_controls. This would fully solve problem 2, because now display is precisely specified (and while it would still be up to the UA how to interpret each individual window control, it would at least finally be clear what developers are asking for when they say display: standalone or display: minimal-ui.

Edit: Credit @amandabaker with the display_modifiers concept.

[benfrancis]

Just to note a historical precedent for this kind of feature, the features argument to window.open() which is a comma-separate list of requested features of the new window.

MDN lists possible values like:

• menubar (whether to render the menu bar)
• toolbar (whether to render toolbar buttons like back, forward, reload, stop)
• location (whether to render the location bar)
• status (whether to render the status bar)
• resizable (whether the window is resizable)
• scrollbars (whether to show horizontal and/or vertical scrollbars)

As I understand it, these values were never properly standardised and were supported inconsistently between browsers and are now mostly ignored by modern browsers where they are just used as a hint to determine whether to open a popup, new tab or new window.

So a couple of options might be:

1. Add a top level features member to the manifest which allows the developer to request enabling/disabling a similar list of UI features for a given application context
2. Heed the warning of history that this may not work in practice and stick to simple display modes

[NotWoods]

For what it's worth, Firefox for Android/Fenix also implements minimal-ui. I suspect based on historical precedent the fine-grained display modifiers won't have great support. I personally prefer the explicit fallback chain, which has similarities to font-family.

[aarongustafson]

Thanks for putting this together @mgiuca. It’s nice to see all of the various pieces in one discussion thread.

One clarification (which I’d like reflected in the original issue): @amandabaker came up with the display modifiers proposal in that initial issue description as an alternate approach, not me. I think it’s a great idea, but I can’t accept credit for it. Personally, I think that approach has the most potential for expansion and future development as well.

Per @benfrancis’ comment, I would not have any issue with calling the member display_features either. I do think "display" needs to be a part of the name, regardless of the suffix we ultimately choose to run with.

[dmurph]

A modification on the "Allow sites to specify an explicit fallback chain" solution could be (to make it even simpler):

When specifying a display option list, if none are supported, then the user agent defaults to 'browser'. All new display options will correspond to a new display mode string, but will only be available in the 'list' version.

For example, displayMode: 'standalone' is valid, but displayMode: 'tabbed' would not. displayMode: ['tabbed', 'minimal-ui'] would be valid, and if the user agent doesn't support any of those, it would fall back to 'browser'

Not necessarily voicing a preference for this solution - just thinking how it could be simpler.

[alancutter]

Just to note a historical precedent for this kind of feature, the features argument to window.open() which is a comma-separate list of requested features of the new window.

Related: Intent to ship: Restrict window.open features parameter

[marcoscaceres]

I'm personally not a fan of changing the display_mode to an array, as that will break backwards compat.

I do find the "display_modifiers" proposal compelling, so hoping @amandabaker will expand on it a bit. As @benfrancis said, there is some precedence on naming some of these UI components, so I'm hoping we can avoid adding new names that can be somewhat confusing (e.g., "caption_controls_overlay" can hopefully match something from the old "window.open()" list).

And as @mgiuca pointed out, I also agree that these should only serve as hints: ultimately, it should be the user who controls the UI components they want around the app (e.g., the ability to zoom text, access reader mode, etc), not the developer app.

[mgiuca]

@aarongustafson :

One clarification (which I’d like reflected in the original issue): @amandabaker came up with the display modifiers proposal in that initial issue description as an alternate approach, not me. I think it’s a great idea, but I can’t accept credit for it. Personally, I think that approach has the most potential for expansion and future development as well.

Apologies, @amandabaker. I was looking at Aaron's comment that expanded on display_modifiers and didn't notice you had proposed it in the original issue report. Updated the text on this bug.

@NotWoods :

For what it's worth, Firefox for Android/Fenix also implements minimal-ui.

Oh OK, thanks for that context. Edited the post.

@marcoscaceres :

I'm personally not a fan of changing the display_mode to an array, as that will break backwards compat.

I proposed it as either a string or an array; it doesn't break backwards compat because the string version would still work.

Re historical window.open modes not being well supported: I think that precedent doesn't necessarily apply here, for a number of reasons:

1. That proposal had a different context, which was giving web pages (in a standalone window) control over browser UI. It's understandable that browser manufacturers were hesitant to actually give sites this level of control. We're in a totally different context here, which is that we're creating an apps platform, with explicitly user-installed apps, and it makes a lot more sense to give those apps control over their user experience.
2. Generally, those window.open settings were about removing browser features. In this context, almost everything is removed by default, and we want to give sites the ability to ask for features added back.
3. window.open simultaneously gave too much and not enough control. It's very tied to the layout of a standard browser window at the time: "menu bar", "tool bar", "status bar" --- these things typically don't even exist on mobile. You're giving sites control over large chunks of browser UI that may or may not make sense in different browsers. At the same time, you're still not giving developers actual control over what controls are visible. If I ask for "toolbar", how do I know if it's got a back button? A refresh button? I don't want to (necessarily) expose control over scrollbars, status bar, toolbars. I want developers to be able to ask for specific actionable controls, like "back button", "refresh button", (maybe) "URL bar".
4. I think we should consider and measure the value of each proposed modifier on its own merits. For example, maybe "back button" is useful enough to be controllable, but "forward button" isn't considered necessary in an app context. I don't want to just throw a dozen controls into the mix. So we'll be a lot more careful, basically, to only expose useful options for developers.

[marcoscaceres]

If I ask for "toolbar", how do I know if it's got a back button? A refresh button? I don't want to (necessarily) expose control over scrollbars, status bar, toolbars. I want developers to be able to ask for specific actionable controls, like "back button", "refresh button", (maybe) "URL bar".

Agree... we need more generalized things like: "navigation controls", etc.

[alancutter]

I'm in favour of the display_modifers proposal as a sibling to the display field.

I think it's inevitable that we will need to make room for more precise control over these modifiers. To future proof this each display modifier should be able to accept either a bool or a bag of further customisations e.g.

{
  "display": "standalone",
  "display_modifiers": {
    "tabs": true
  }
}

or

{
  "display": "standalone",
  "display_modifiers": {
    "tabs": {
      "active_color": "white",
      "inactive_color": "transparent"
    }
  }
}

"tabs": true and "tabs": {} are behaviourally equivalent; all sub customisations are decided by the user agent.

[marcoscaceres]

    "tabs": {
      "active_color": "white",
      "inactive_color": "transparent"
    }
  }

If we start getting to this level of detail, then we are going to be in trouble. I don't want us to get into a situation where we start defining specific styling for things or get overly specific. If we can agree on a teeny tiny subset of abstract things, that would already be a huge step forward.

[alancutter]

I mainly just don't want us to be painted into a corner and have to resort to something like "display_modifiers": ["home_button", "home_button_url=/new_document.html"] should we find sub customisation in high demand.

[dmurph]

Hey folks,

I did a lot of brainstorming and I came up with the following proposal:
https://github.com/dmurph/display-mode/blob/master/explainer.md

I call it display-override, but I'm open to other names. I think it really neatly solves at least one of the problems here (fallbacks), and leaves room for future improvements like display-modifier if necessary, with the added benefit of fallback logic.

I propose this API change as a first step, allowing new display modes like tabbed and customized (or whatever that one is be called) in a way that gives developers control of the fallback chain. If only a few more display modes are needed, then this design should work out just fine, and if demand increases for (exponentially different) display customization as more options are introduced, I outline some ideas in the "Future Considerations" section that show how we can accomplish this as well.

Can you all please take a look and tell me what you think? I would love feedback here :)

[sunggook]

Have you consider about controlling menu (...) as well as context menu with this manifest property?
These are quite browser related menu and some PWA developers don't want user to know that their app's dependency or background engine, particularly when app is downloaded from the Store.

I don't see any modern platform to provide this kind of big menu (in Desktop) in their native app.

[dmurph]

I think that's an interesting problem! Just to be clear, you're asking if this property could help control whether the 'user agent menu' button shows up?

something like a display property like 'standalone-no-useragent' or something like that?

[amandabaker]

@sunggook That sounds like it will primarily apply to store apps since, from what I've heard, we don't want to allow any arbitrary web app to hide that menu (or any of the other icons/origin text that appears next to the three-dot menu) for spoofing/security reasons. Also, would the developer want to hide the menu button on a standalone/minimal-ui/window-controls-overlay/tabbed app? If more than one should be supported, then we would need to introduce one display_override for each.

Maybe there should be a way for developers to request to hide the three-dot menu when putting their app into the store instead of making it a display_override in the manifest?

[mgiuca]

Yeah, I don't think we'll want to provide developers with that option in Chrome. Even though it does make a point of differentiation between native windows and PWA windows (which we generally consider Bad), it adds so much utility, as well as security (giving the user access to the page URL, security info and permissions, for example). I don't think it should be up to the site whether that stuff is available.

@amandabaker Interesting that it might be a property of the store (since then the analogy is giving the developer full control as we would a native app). I'd still be hesitant to expose it there, since it provides so much utility, but it's an option.

[sunggook]

@mgiuca , I don't think I have seen any native app model that provide default menu like PWA so far, the reason is that developers don't like default menu (They want their users to immersive into their app whether it is a client or non client area). I'm not sure if Android PWA has default menu if security is a real concern for it?

@amandabaker , see comment above for Android, and there is also WebView that is almost same boat of PWA in terms of showing web contents as it is, but it doesn't have mandatory UI like the current Desktop PWA's.

@mgiuca , btw, the other suggestion I have besides this is to support platform based UI. Desktop, phone are totally different form factor so it is quite difficult to have universal UI that fit both platforms, and every platform's UI has its own philosophy so single UI view won't work for all . It would be great if developer handles these by themselves instead of individual browser, for example, ('windows': [backbutton=true, 'no_menu', ], 'android'=['no_contex_menu', ].

[dmurph]

I think the 'no browser menu' idea could either be tackled by the first suggestion in my 'future ideas' section:
https://github.com/dmurph/display-mode/blob/master/explainer.md#custom-display-mode-names-with-display-modifiers-style-specification

or to treat it as a separate manifest feature. Either way,I think it's probably worth it's own explainer & discussion. Maybe open a new bug?

[mgiuca]

Update for anyone following along: Daniel's proposal is getting a bunch of discussion from Chromium folks (mainly from Google and Microsoft, and @kenchris). I want to make sure other parties are aware of the discussions.

Interesting discussions are:

• #2: Proposal to reuse display instead of creating a new member
• #3: Proposal to allow display-override to be either a single value or an array
• #10: Proposal to deprecate display rather than adding an override

[mgiuca]

@mgiuca , I don't think I have seen any native app model that provide default menu like PWA so far, the reason is that developers don't like default menu (They want their users to immersive into their app whether it is a client or non client area). I'm not sure if Android PWA has default menu if security is a real concern for it?

Android PWAs expose an equivalent of that menu through a low-priority notification (you can drag from the top of the screen while using a PWA), which exposes the basic features of copy URL, share and open in Chrome.

I don't think it's because "developers don't like it"; I think the value of such a menu is limited in a native app platform. The menu essentially exposes "browser-like features" that apply to the current page: access to encryption information, the URL underlying the current page, being able to move the page back into a browser, HTML-based zoom, find-in-page, print, etc. Those things are traditionally available for web pages, but they have no real native equivalent.

Whether or not developers like it, I see it as a compromise between the "purity" of the app experience where the developer has total control (which is arguably the developer standpoint) and the utility of providing those web-based controls, to ensure that when a user installs a website as an app, we aren't removing a whole bunch of features (security, and otherwise). It's our job as the user agent to represent the user; yes, we're providing a platform for developers, but the web platform doesn't generally give the developer the ability to remove utility from the user (e.g., no way for a website to request that a browser hides the "view source" button).

I think we could see this as a user-agent specific feature which perhaps Edge could do for apps installed in the store.

@mgiuca , btw, the other suggestion I have besides this is to support platform based UI. Desktop, phone are totally different form factor so it is quite difficult to have universal UI that fit both platforms, and every platform's UI has its own philosophy so single UI view won't work for all . It would be great if developer handles these by themselves instead of individual browser, for example, ('windows': [backbutton=true, 'no_menu', ], 'android'=['no_contex_menu', ].

I'm generally opposed to speccing user-agent-specific or OS-specific customization options. That tends to lead to websites that are designed to work really well on a handful of popular operating systems, and with a poor experience on OSes that the site developer hasn't thought about. We're trying to build a platform where you specify your requirements semantically, then the user agent does its best to interpret that data and tailor it for the host OS; that way, a future OS can pop up and interpret the existing data, rather than having to wait for sites to update with specific metadata for the new OS. (An example of this philosophy is the maskable icons design, in which we resisted the temptation to have a field where you provide a specialized icon for each OS; see this discussion and search for "Option C".) Therefore, we should not have an option for each OS; if we want to give developers control over this, we should have a setting like "browser_menu" (which is up to the user agent by default, but can be set to true or false as a hint to the user agent).

[sunggook]

@dmurph , I will investigate further if menu property is worth for an explainer, thanks.
@mgiuca , there is opened bug, https://bugs.chromium.org/p/chromium/issues/detail?id=1093086&q=PWA%20os%3DAndroid&can=2, which is about imperfect icons (compare to Native) in Android. it isn't just for Android, the current universal icons in the PWA doesn't look fit well with Windows either (even if developer provides perfect icons for Android, it won't be guaranteed that it will fit for Windows/Mac, etc). I agree most users and PWA developers wouldn't mind of icons quality, or would be fine with user agent's decision on any display options, but real serious vendors like Facebook, Slack, etc would because they usually have their own design scheme, which isn't controlled by the user agent and (I believe) PWA needs those killer apps.

[mgiuca]

@sunggook : I don't think we should get into a debate here about icons in particular. I was using it as an example of past work we've done on the manifest that makes a good compromise between supplying metadata that can be tailored for individual host OSes, and the universality of the web platform. FYI, crbug.com/1093086 is in support of this work: it is a feature request from @NotWoods (who wrote the Monochrome spec) for Chrome to support Maskable and Monochrome, not asking for platform-specific icons. Meanwhile, #913 has removed the platform field for icons, because it was decided that maskable and monochrome are better (non-platform-specific) ways of supplying that metadata.

real serious vendors like Facebook, Slack, etc would because they usually have their own design scheme, which isn't controlled by the user agent and (I believe) PWA needs those killer apps

If that's the case, I would like to hear from such a vendor what they believe they are unable to do using a combination of any, maskable and monochrome that they could only do by supplying platform-specific icons. In most cases, we should be able to solve the problem by either or both: a) helping them design an appropriate suite of icons using the platform-neutral purposes we've designed, and/or b) adapting the way user agents interpret platform-neutral purposes on certain platforms (e.g., on Windows, it might look better to adapt the monochrome icon into a Windows-style icon tile, rather than the maskable). If all else fails, vendors can have the server dynamically change the manifest based on the user-agent string, which is my preferred option for doing this, since it rightly implies it is a last resort and not a built-in feature of the manifest.

The above all serves as an analogy for other manifest fields such as window controls.

[sunggook]

@mgiuca , yes the discussion here isn't just about icons, it is general manifest properties. btw, thanks for sharing ManifestImageResource recent decision.

It looks like the current manifest allows these 3 for platform awareness.

1. use 'platform', eg. related_applications and previous ManifestImageResource. (or @aarongustafson suggests 'platform' member to App Information/screenshots).
2. use universal solution like maskable and monochrome
3. use dynamic manifest by use user agents based code like 'lang'.

It looks like it is per property decision what scheme it will use. I'm not sure if we continue this way or it's a time for single voice.
(Maybe it is okay so far, but this feature of 'precise control over app window' could be something developer should be ware of platform and the only solution they have is 3 now.)

[mgiuca]

@sunggook That's a useful taxonomy, thanks for sharing.

Yes, in general I am advocating that we move away from 1, recommend 2 wherever possible and 3 as a fallback.

2 offers the least direct control over how things will behave on each specific platform, but that's exactly the point. Web developers are not supposed to be deciding "on Windows, it should do this, on macOS it should do that." The entire point of the Web is to be a fairly tight abstraction over the host OS. The browser manufacturer should decide what to do on Windows or on macOS; the web developer should supply enough generic metadata that the browser can do something reasonable.

We aren't aiming to provide developers with 100% control over the application, just as we don't give developers 100% capabilities they would expect from a native binary. We provide a reasonable trade-off between control and portability.