feat(v2): Plugin for Offline/PWA support #2205

codemonkey800 · 2020-01-11T12:12:26Z

Changes

Implements @docusaurus/plugin-pwa
Add popup for notifying users of updated website content
Adds @docusaurus/plugin-pwa to v2 website, adding offline/PWA support

Motivation

Implements offline support as requested in #458. Having offline support is beneficial for a dev that needs documentation to work on a project where internet is slow or inaccessible (i.e. airplanes, trains, etc.)

Have you read the Contributing Guidelines on pull requests?

Yes 😄

Test Plan

I've manually tested offline support in the browser and as an installed app. I also ran a lighthouse audit.

Loading Offline

Installing as a Chrome App

Open app in `chrome://apps`

Lighthouse Audit

Redirect HTTP to HTTPs is server dependent so it's checked off in the audit.

Update Popup

Related PRs

facebook-github-bot · 2020-01-11T12:12:56Z

Hi codemonkey800! Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file.In order for us to review and merge your code, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

docusaurus-bot · 2020-01-11T12:16:59Z

Deploy preview for docusaurus-2 ready!

Built with commit 0e0b219

https://deploy-preview-2205--docusaurus-2.netlify.app

facebook-github-bot · 2020-01-11T12:20:57Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

bravo-kernel · 2020-01-11T17:32:16Z

This is pretty amazing, I recently tested oflline and noticed the site went down so this is ❤️

nebrelbug · 2020-01-12T01:11:24Z

@codemonkey800 wow, I love this! One possible idea is to implement something like VuePress's plugin (https://vuepress.vuejs.org/plugin/official/plugin-pwa.html) that creates a popup when content has been refreshed.

yangshun · 2020-01-12T14:27:49Z

This is such amazing and great work from the community. We're shorthanded now and I'll only have time to review next weekend though. Thank you from the bottom of my heart ❤️

Maybe @lex111 can take a stab at reviewing it too.

codemonkey800 · 2020-01-13T20:58:45Z

@nebrelbug great suggestion :) I'd love to add something like this, I'll see if it's something I can get around to implementing. I mostly wanted to get a foundation for offline/PWA functionality first

lex111 · 2020-01-17T22:38:32Z

@codemonkey800 This is HUGE and the long-awaited feature! Thanks a lot!

I have no experience in PWA with Workbox, but how does the mobile application update when the documentation is updated? I also noticed that pictures from FB are not showed in offline mode, is this the way it should be?
Can we also fix Lighthouse audit errors?

codemonkey800 · 2020-01-17T23:25:48Z

@lex111 thank you!

Updating

I need to verify this myself, but updating shouldn't be a problem when the documentation is updated. First off, the service worker URL should never change between application builds. You can read more about it here, but basically changing the URL will cause the outdated service worker to load if you cache your index.html.

Now since the service worker URL will always point to the up-to-date service worker, whenever we update the documentation site, a new precache manifest will be generated at the top of the service worker for each build using the InjectManifest plugin:

importScripts('/precache-manifest-server.[hash].js');
importScripts(
  '/precache-manifest-client.[hash].js',
  '/workbox-v[version]/workbox-sw.js',
);

The [hash] will be unique between each build, so we can guarantee that the website will be updated by the service worker using the updated precache manifest. What I do need to check though is when the precache manifest is updated, does the service worker:

Show the old content and update the cache or
Update the cache and then show the new content.

FB Images

The FB images (or more generally images from Markdown posts using authorImageUrl) are not precached like the Webpack assets because we're not processing them through Webpack. Precaching only works through Webpack because of the precache list.

A requirement for the precache list to work is that the resource URL has a way to identify different revisions (like a hash in the URL). Since the FB images aren't processed through Webpack and generated with a unique hash, we can't add them to the precache list.

So instead, I added FB images as Workbox routes so that when you load it the first time over the network, it'll be cached by the service worker. But if you install the app offline before loading the FB images, they won't show up.

I think one way we can fix this is by scanning all authorImageUrls, attach a revision (maybe overall build hash) to it, and add it to the precache manifest.

Lighthouse Errors

Yeah 😆

So these lighthouse errors are because some tags are missing for iOS and Android. Normally webpack-pwa-manifest would have generated all those tags for us, but because we don't use HtmlWebpackPlugin, they don't get injected in production.

I think what I can do here instead is add those tags manually instead of having webpack-pwa-manifest do it for me.

codemonkey800 · 2020-01-17T23:32:47Z

Just wanted to let y'all know I'll be a bit slow to responding to feedback and addressing it in my PR due to studying + work. I'm hoping we can get this merged eventually though 😄

lex111 · 2020-02-24T00:33:05Z

@codemonkey800 how goes it? I am wondering if we can refuse to use external dependency (Workbox) for SW? For example, how is this done in VuePress?

codemonkey800 · 2020-02-25T01:32:44Z

@lex111 i've been swamped with work but I'll push my remaining changes either tonight or tomorrow. It includes the PWA popup to reload the page and processes the service worker in Webpack.

It is definitely possible to make this work without Workbox. The main things we use Workbox for is:

Generating service worker manifest
Using Workbox strategies to cache things on the fly
Handling when to show the popup

VuePress also uses workbox, specifically the generateSW function: https://git.io/JvEPw

packages/docusaurus-plugin-pwa/src/index.js

lex111 · 2020-02-26T08:54:37Z

@codemonkey800 I will check it soon, will I get recommendations on how to correctly check PWA? What could be the negative consequences of this?

codemonkey800 · 2020-02-26T09:24:09Z

@lex111 You could run PWA Lighthouse audits in your dev tools on the preview website. You can also refer to the PWA checklist: https://web.dev/pwa-checklist/

To check the update popup, you'll need to:

Run yarn build in website/ and run an http server. This is to register the initial service worker
Make changes to website/, like update a markdown file somewhere
Run yarn build again and run an http server. If you look in Chrome Dev Tools on the Application tab, you should see a second service worker being installed, eventually moving in the waiting stage, and after that the popup should show

Some possible negatives I can think of are:

Normal browser reloads will not update the website anymore. Either a hard refresh or clicking Refresh on the popup will update the contents.
That said, if a developer introduces a bug that prevents rendering of the popup dialog, the user may not be able to update the website without doing a hard refresh. For example, consider this scenario:

User visits docusuaurs site for the first time and the service worker gets registered
Developer introduces bug that causes renderPopup() to fail
User visits docusuaurs site again. Because their current cached renderPopup() works, it'll show the popup indicating that there's a new version of the website
Developer updates markdown or something
The update popup never shows because the renderPopup() function is buggy, so the user stays stuck with the cached version of the docusuaurs site until they hard refresh the page

There are definitely tradeoffs when making a PWA, but I think it should be up to the discretion of the user if they want to add it. However, whether we add it to the docusuaurs site itself is up for debate 😅

lex111 · 2020-02-26T09:39:22Z

@codemonkey800 thanks for detail input. Hmm, maybe it makes sense to activate PWA only in certain cases? For example, if the user explicitly install the site as an application? If the user visited the site with mobile?

I described these cases in this pseudocode:

let forceOffline = location.search.includes('offline');

function initSW() {
 navigator.serviceWorker.register('/sw.js')
}

function isSaveDate() {
  let saveData = navigator.connection && navigator.connection.saveData
  return window.innerWidth <= 940 || saveData
}

if ((forceOffline || !isSaveDate()) && navigator.serviceWorker) {
  // init SW if the user turned on save data mode or they just went to a page from a mobile OR `force` option exists (for testing purposes)
  // See https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/save-data/ for details
} else if (localStorage.appinstalled) {
  // init SW if user installed site as an app and refresh a page
  } else {
  window.addEventListener('appinstalled', () => {
    // init SW if installed site as an app
    
    localStorage.appinstalled = true;
  });
}

What do you think?

codemonkey800 · 2020-02-26T09:52:35Z

@lex111 For sure, I think that sounds like a good idea. How about we make this the default behavior and add an option like hybrid that defaults to true. If hybrid is false, then make the app entirely PWA without the above cases for users who want a fully PWA doc site

I will need to research and see if it's possible to install an app and run the service worker after because usually it's the other way around

lex111 · 2020-02-26T10:09:22Z

@codemonkey800 I’m not sure exactly whether this option is needed, because in my view PWA + Offline is only needed for certain cases (mobile devices, or if the site is implicitly using it as an application). I updated example above to show the init SW, isn’t this the case in reality?

codemonkey800 · 2020-02-26T10:20:28Z

@lex111 Oh okay never mind I think I understand now. One thing I'm wondering is how should we let the user force offline mode? Aside from adding ~~/#offline~~ /?offline directly to the URL?

lex111 · 2020-02-26T10:25:12Z

@codemonkey800 yep, something like "https://v2.docusaurus.io/?offline" (location.search.includes('offline')). This will be needed for testing, and besides this, in the future I want to add Lighthouse check to CI.

nebrelbug · 2020-02-26T14:08:22Z

@codemonkey800 will there be a way to specify the Workbox caching strategy in the docusaurus.config.js? Ex. strategy: 'stale-while-revalidate' or strategy: 'network-only'

https://developers.google.com/web/tools/workbox/modules/workbox-strategies

codemonkey800 · 2020-02-27T20:13:14Z

@nebrelbug There's currently no way to specify the strategy for the precache manifest unfortunately. See GoogleChrome/workbox#1767

However, you can set up your own runtime strategies using the custom service worker but it won't be added to the cache at the very start like resources in the precache manifest

codemonkey800 · 2020-02-27T23:30:45Z

@lex111 In order for the app install button to show, there needs to be a running service worker, so we can't do registration after 😢

An app will also not be installable unless it's available offline:

lex111 · 2020-02-28T08:44:12Z

@codemonkey800 too bad, are there any options to get around this? For example, check the conditions for the use of PWA in the service worker?

codemonkey800 · 2020-03-06T09:33:50Z

@lex111 We could possibly pass an empty array into precacheAndRoute() to make the browser think it has offline support. Since the array is empty, every resource will be downloaded over the network.

Then when the user installs the app, we can emit a CACHE_FILES message to the service worker so it can precache the actual manifest.

An issue I see here is if the user uninstalls the app, I don't think there's currently a way to detect that, so the service worker will continue to precache files. Is this okay?

- debug logs - better SW params system - offline mode activation strategies - docs

slorber · 2020-07-07T16:07:55Z

Hey @codemonkey800 , I think this PR is ready. If you have some time to review my changes, so that I can merge it asap, that would be great.

Otherwise I'll merge it tomorrow. As it's a plugin, and we are in alpha, we think it's better to get it merged, and see if we have some feedbacks and bug reports to improve it.

Major changes:

custom babel/webpack setup more adapted for SW code
debug mode (custom logs + workbox logs)
configurable offline mode / precaching activation strategies + sensible defaults
better QS params passing between registerSW and SW
use default browser ui for app installation
minor changes (refactors, renamings, docs...)

Overall your code worked great, thanks for your work.

Doc: https://deploy-preview-2205--docusaurus-2.netlify.app/docs/next/using-plugins/#docusaurusplugin-pwa

codemonkey800 · 2020-07-07T17:51:18Z

@slorber wow, this is amazing work 😄 I'll take a look at all the code today, but from what I've seen it's looking very good. thank you so much for the help

codemonkey800 · 2020-07-08T00:03:23Z

@slorber I finished reviewing the new code. I love all the new changes you added, it makes the PWA a lot more robust 😄

slorber · 2020-07-08T10:31:56Z

awesome @codemonkey800 , let's merge this finally! 🎉

yangshun · 2020-07-08T10:40:34Z

Finally! Thanks @codemonkey800 @slorber @lex111!

codemonkey800 · 2020-07-08T18:13:51Z

Thank you @slorber and @lex111 with all the help in reviewing this and finally getting it merged. I couldn't have done it without y'all 😃

hobadams · 2020-07-09T08:14:31Z

Nice work guys! Can't wait for the release :-) @codemonkey800 @slorber @lex111

slorber · 2020-07-24T18:10:37Z

Hey all, this is feature is now released!
If you like it, don't forget to retweet 😄
https://twitter.com/docusaurus/status/1286715187983048704

codemonkey800 requested review from wgao19 and yangshun as code owners January 11, 2020 12:12

facebook-github-bot added the CLA Signed Signed Facebook CLA label Jan 11, 2020

lex111 added the pr: new feature This PR adds a new API or behavior. label Jan 24, 2020

codemonkey800 requested a review from lex111 as a code owner February 26, 2020 08:42

codemonkey800 commented Feb 26, 2020

View reviewed changes

packages/docusaurus-plugin-pwa/src/index.js Outdated Show resolved Hide resolved

slorber added 16 commits July 6, 2020 17:05

minor refactors + add workbox debug mode

28ade9a

env PWA_ prefix

cf0dd42

typo

1b13340

minor refactor

5a7ed5a

fix file output

da67c42

add serve:v2:ssl yarn command

e71280a

minor pwa fixes

9bc28c5

typo

f8b71ec

add dynamic import comment in SW

c71373f

comment

bad56ac

let the PWA plugin implement its reload popup on his own

4e49004

pwa: add Joi options validation

fedc331

pwa plugin should have its own webpack/babel custom setup

313e8c5

PWA:

dc063de

- debug logs - better SW params system - offline mode activation strategies - docs

add pwa install gif

3becaa7

pwa: popup -> reloadPopup + minor refactors

dfbc75b

slorber added 2 commits July 7, 2020 18:18

fix process.env reading + better debug log

820128a

minor fixes

8112260

slorber added 2 commits July 8, 2020 12:08

minor changes

788f831

minor changes

0e0b219

slorber merged commit 9b3da59 into facebook:master Jul 8, 2020

ssalbdivad mentioned this pull request Jul 15, 2020

Integrate @docusaurus/plugin-pwa with website and import "typeface-ubuntu" arktypeio/arktype#211

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(v2): Plugin for Offline/PWA support #2205

feat(v2): Plugin for Offline/PWA support #2205

codemonkey800 commented Jan 11, 2020 •

edited

Loading

facebook-github-bot commented Jan 11, 2020

docusaurus-bot commented Jan 11, 2020 •

edited

Loading

facebook-github-bot commented Jan 11, 2020

bravo-kernel commented Jan 11, 2020

nebrelbug commented Jan 12, 2020

yangshun commented Jan 12, 2020

codemonkey800 commented Jan 13, 2020

lex111 commented Jan 17, 2020

codemonkey800 commented Jan 17, 2020 •

edited

Loading

codemonkey800 commented Jan 17, 2020

lex111 commented Feb 24, 2020

codemonkey800 commented Feb 25, 2020

lex111 commented Feb 26, 2020

codemonkey800 commented Feb 26, 2020

lex111 commented Feb 26, 2020 •

edited

Loading

codemonkey800 commented Feb 26, 2020

lex111 commented Feb 26, 2020

codemonkey800 commented Feb 26, 2020 •

edited

Loading

lex111 commented Feb 26, 2020

nebrelbug commented Feb 26, 2020

codemonkey800 commented Feb 27, 2020

codemonkey800 commented Feb 27, 2020 •

edited

Loading

lex111 commented Feb 28, 2020

codemonkey800 commented Mar 6, 2020

slorber commented Jul 7, 2020 •

edited

Loading

codemonkey800 commented Jul 7, 2020

codemonkey800 commented Jul 8, 2020

slorber commented Jul 8, 2020

yangshun commented Jul 8, 2020

codemonkey800 commented Jul 8, 2020

hobadams commented Jul 9, 2020

slorber commented Jul 24, 2020

feat(v2): Plugin for Offline/PWA support #2205

feat(v2): Plugin for Offline/PWA support #2205

Conversation

codemonkey800 commented Jan 11, 2020 • edited Loading

Changes

Motivation

Have you read the Contributing Guidelines on pull requests?

Test Plan

Loading Offline

Installing as a Chrome App

Open app in chrome://apps

Lighthouse Audit

Update Popup

Related PRs

facebook-github-bot commented Jan 11, 2020

docusaurus-bot commented Jan 11, 2020 • edited Loading

facebook-github-bot commented Jan 11, 2020

bravo-kernel commented Jan 11, 2020

nebrelbug commented Jan 12, 2020

yangshun commented Jan 12, 2020

codemonkey800 commented Jan 13, 2020

lex111 commented Jan 17, 2020

codemonkey800 commented Jan 17, 2020 • edited Loading

Updating

FB Images

Lighthouse Errors

codemonkey800 commented Jan 17, 2020

lex111 commented Feb 24, 2020

codemonkey800 commented Feb 25, 2020

lex111 commented Feb 26, 2020

codemonkey800 commented Feb 26, 2020

lex111 commented Feb 26, 2020 • edited Loading

codemonkey800 commented Feb 26, 2020

lex111 commented Feb 26, 2020

codemonkey800 commented Feb 26, 2020 • edited Loading

lex111 commented Feb 26, 2020

nebrelbug commented Feb 26, 2020

codemonkey800 commented Feb 27, 2020

codemonkey800 commented Feb 27, 2020 • edited Loading

lex111 commented Feb 28, 2020

codemonkey800 commented Mar 6, 2020

slorber commented Jul 7, 2020 • edited Loading

codemonkey800 commented Jul 7, 2020

codemonkey800 commented Jul 8, 2020

slorber commented Jul 8, 2020

yangshun commented Jul 8, 2020

codemonkey800 commented Jul 8, 2020

hobadams commented Jul 9, 2020

slorber commented Jul 24, 2020

codemonkey800 commented Jan 11, 2020 •

edited

Loading

Open app in `chrome://apps`

docusaurus-bot commented Jan 11, 2020 •

edited

Loading

codemonkey800 commented Jan 17, 2020 •

edited

Loading

lex111 commented Feb 26, 2020 •

edited

Loading

codemonkey800 commented Feb 26, 2020 •

edited

Loading

codemonkey800 commented Feb 27, 2020 •

edited

Loading

slorber commented Jul 7, 2020 •

edited

Loading