Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: dhis2 connection status [LIBS-315] #1203

Merged
merged 93 commits into from
Mar 2, 2023

Conversation

KaiVandivier
Copy link
Contributor

@KaiVandivier KaiVandivier commented Jun 6, 2022

Part of LIBS-315

Adds a DHIS2 connection status service as part of the offline tools. Coordinates with the offline interface to deduce the status by incidental network requests, then sends pings periodically if there is no other traffic.

Depends on dhis2/app-platform#718; would also benefit from DHIS2-14531: Create dedicated ping endpoint

The spec

(This section is the same text as in the Platform PR above)

  • We want to detect connection to the DHIS2 server.
  • We want to minimize the network and server resources used to do so
  • The service worker watches app network traffic and uses the success or failure of responses to determine connectivity status, then send messages to the client to broadcast the updates
    • Only requests to the DHIS2 server (based on the Base URL) are used to determine the connection status
  • If there is no app network traffic to the DHIS2 server, the client will send pings
    • The service worker is configured to only handle pings over the network, i.e. not to cache ping requests/responses, so the result observed on the client will reflect the server connection status
    • If the connection status detected by the ping is the same as the previous status, then the time to the next ping will increase. This will continue to create an exponential back-off, which will max at out at a value, for example 5 minutes.
    • If the connection status has changed (either by ping or by app network traffic), the exponential back-off will be reset and the timeout to the next ping will be reduced back to the initial timeout.
    • If the app loses focus, a “standby” procedure is followed to stop pinging when it’s unnecessary
      • Nothing happens initially. If the user refocuses the app before the next ping is scheduled to be sent, the behavior will not change.
      • If the next-scheduled ping would take place while the app is defocused, instead the hook enters a “standby” mode.
      • When the app is refocused, if the hook is in “standby”, a ping will be sent immediately to detect the current connection status, following other rules for back-off
      • If the app is opened while defocused initially, it will also start running in standby and will not ping until refocused.
    • If the app detects an ‘offline’ event from the browser, it will ping immediately to check the connection status, since that should be a low-cost action
    • If the app detects an ‘offline’ event from the browser and is defocused, a special thing happens: it enters standby to send a ping imediately when the app is refocused again
      • If the app is refocused before the next regularly-scheduled ping, the ping timeout will be restarted but not incremented.
      • Otherwise, if the regularly scheduled ping is ready before the app is refocused, the normal standby procedure is followed (e.g. the next timeout can be incremented if the connection status hasn’t changed)

Notes for this PR

  • The new functionality is made up primarily of two parts:
    • the SmartInterval class, which handles the timeout backoffs, resets, and snoozes
    • the useDhis2ConnectionStatus hook, which integrates the SmartInterval to handle ping timing and the offline interface to listen to updates from the service worker.
  • It would be nice to export one useOnlineStatus hook which returns ConnectionStatus by default but could return NetworkStatus by passing an options.useNetworkStatus parameter, but that turned out to be complicated for several reasons (for example the rule of hooks). After discussing with Hendrik, we decided to leave the existing hook’s name, and export a new useDhis2ConnectionStatus hook
    • Under the hood, the useOnlineStatus hook has been renamed to useNetworkStatus to better reflect its function. Still, it’s exported as useOnlineStatus for backward-compatibility
    • The term “online” is a bit ambiguous, but seems to best match the names of the browser APIs used for NetworkStatus
    • The useDhis2ConnectionStatus hook then does what it says on the tin
  • Smaller decisions are mentioned in inline comments

To test locally:

To do:

  • Remove console logs when done testing
  • If DHIS2-14531: Create dedicated ping endpoint is finished, undo the 'band aid' in the RestAPILink for the current ping endpoint
  • Adjust timeout defaults (example: 30 second initial timeout, 1.5x increment factor, 5 min max timeout?)

Done:

  • Add automated tests
  • Create follow-up issues for todos mentioned in the comments

@KaiVandivier KaiVandivier marked this pull request as draft June 6, 2022 21:47
Copy link
Contributor

@HendrikThePendric HendrikThePendric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think before you continue with this PR we need to settle on a valid strategy to check the connectivity status. I have proposed one in this Google Doc.

Comment on lines 26 to 37
/**
* Band-aid for the ping API, which can throw an error when being processed
* as JSON. Hopefully this will be superseded by a new ping endpoint:
* https://dhis2.atlassian.net/browse/DHIS2-14531
*/
const getAcceptHeader = (query: ResolvedResourceQuery): string => {
if (query.resource === 'system/ping') {
return 'text/plain'
}

return 'application/json'
}
Copy link
Contributor Author

@KaiVandivier KaiVandivier Feb 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be removed if we get a dedicated ping endpoint before merging, as requested in the Jira issue mentioned in the comment: DHIS2-14531: Create dedicated ping endpoint

Comment on lines 74 to 82
<CustomDataProvider data={{}}>
<OfflineProvider
offlineInterface={mockOfflineInterface}
{...props}
>
<TestControls id={'1'} {...props} />
<TestSection id={'1'} {...props} />
</OfflineProvider>
</CustomDataProvider>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OfflineProvider now uses a data provider to use the engine to make a data query to /api/system/ping

The same changes will be repeated in the next few test files

Comment on lines 62 to 70
// The offline interface persists the latest update from the SW so that
// this hook can initialize to an accurate value. The App Adapter in the
// platform waits for this value to be populated before rendering the
// the App Runtime provider (including this), but if that is not done,
// `latestIsConnected` may be `null` depending on the outcome of race
// conditions between the SW and the React component tree.
const [isConnected, setIsConnected] = useState(
offlineInterface.latestIsConnected
)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented in this commit: dhis2/app-platform@5644f0d

Comment on lines -25 to +28
- [Offline tools](advanced/offline)
- [Offline tools](advanced/offline/)
- [Cacheable Sections](advanced/offline/CacheableSections.md)
- [useDhis2ConnectionStatus](advanced/offline/useDhis2ConnectionStatus.md)
- [useOnlineStatus](advanced/offline/useOnlineStatus.md)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I improved the organization and discoverability here

Copy link
Contributor

@HendrikThePendric HendrikThePendric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to do something about all that logging to the console. Other than that it all looks great to me.


The app platform provides some support for PWA features, including a `manifest.json` file for installability and service worker which can provide offline caching. In addition to those features, the app runtime provides support for ["cacheable sections"](advanced/offline/CacheableSections), which are sections of an app that can be individually cached on-demand. The [`useCacheableSection` hook](advanced/offline/CacheableSections#usecacheablesection-api) and the [`CacheableSection` component](advanced/offline/CacheableSections#cacheablesection-api) provide the controls for the section and the wrapper for the section, respectively. The [`useCachedSections` hook](advanced/offline/CacheableSections#usecachedsections-api) returns a list of sections that are stored in the cache and a function that can delete them.

An important tool for offline-capable apps is the [`useDhis2ConnectionStatus` hook](advanced/offline/useDhis2ConnectionStatus.md), which can be used to determine whether or not the app can connect to the DHIS2 server. There is also a [`useOnlineStatus` hook](advanced/offline/useOnlineStatus.md) which returns the whether or not the client is connected to the internet, but `useDhis2ConnectionStatus` is probably the one you want to use.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding an explanation as to why this is the one people would want to use. I.e. provide the most common example of local/on-site/intranet DHIS2 Core instances.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a bit here: 1585c50

Comment on lines 49 to 52
headers: {
...requestHeadersForContentType(contentType),
Accept: getAcceptHeader(query),
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps cleaner to call the getAcceptHeader helper in requestHeadersForContentType

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 On second thought.... I can see why you kept this separate, since the helper is suffixed with ForContentType and the accept header does not actually depend on the content type. Still it would be in good company... requestBodyForContentType has committed the same sin.

On balance I don't think one option is significantly better than the other. And hopefully we can remove this helper soon anyway. So just leave it if you'd like.

/** Called when SW reports updates from incidental network traffic */
const onUpdate = useCallback(
({ isConnected: newIsConnected }) => {
console.log('handling update from sw')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, but since this is a PR against master and I feel it's in its final stages we should now either get rid of them or keep them in but ensure they only show on the console in development mode. I.e. sth like this:

function devLog() {
    if (process.env === 'development') {
        console.log(arguments)
    }
}

Copy link
Contributor

@HendrikThePendric HendrikThePendric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to approve this PR because the code is looking good. But before merging, please take care to either remove the console.log statements or replace them with something that only logs in development mode, see #1203 (comment).

Copy link
Contributor

@HendrikThePendric HendrikThePendric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reviewed again, and only focussed on the commits that had been added approximately since the last time I reviewed, so starting at 1585c50 until the last commit. All looked good. to me. Great work @KaiVandivier

@KaiVandivier KaiVandivier enabled auto-merge (squash) March 2, 2023 15:40
@KaiVandivier KaiVandivier merged commit 6a4156e into master Mar 2, 2023
@KaiVandivier KaiVandivier deleted the feat-dhis2-connection-status branch March 2, 2023 18:48
dhis2-bot added a commit that referenced this pull request Mar 2, 2023
# [3.9.0](v3.8.0...v3.9.0) (2023-03-02)

### Features

* dhis2 connection status [LIBS-315] ([#1203](#1203)) ([6a4156e](6a4156e))
@dhis2-bot
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants