Dependency Management in Dynamically Imported Modules

[gtrufitt]

The Problem

Dynamically importing modules into a platform requires a contract to avoid double-bundling shared dependencies. For example, a React Component that is included has a peer dependency of React, though if it is a dynamic import (in the http dynamically imported sense), there's no way to know of that peer dependency or whether the platform includes that dependency at run time.

To solve, we need to:

• Expose the peer dependencies of the module (i.e what we expect the platform to always contain - React, Emotion etc) and make sure that the platform has a way of ensuring those are available
• A version of those dependencies - it's sometimes not enough to say 'React' when we need a specific version
• Ensure performant bundles of those dependencies
• Ensure that there is no race condition between the modules and their dependencies

Prior work

Automat expects dependencies on the window of the platform like https://github.com/guardian/dotcom-rendering/blob/55ccdbe592a5895133b67234ea18fb81f1467e0c/window.guardian.d.ts#L36

@nicl @tjmw have thought about this a lot so I am very interested in their input. @sndrs raised this in the Client Side Infra meeting, could you re-share the CMP use case?

An Idea

A dynamically imported module exposes a thin NPM module that defines the contract between the Platform and the Module.

The Module:

• Is Versioned
• Includes in package.json the required peer dependencies
• Shares the dependencies that are required on the window, preferable as TS types

The Platform:

• Makes a dynamic import request for the current major version of the Module
• Ensures that peer dependency requirements are installed
• Includes the typed window requirements into the window interface of the Platform, along with the specific version of the dependencies.

If The Module wants to update it's dependency versions or add new dependencies, this will require a major bump to the module installed in the platforms as the contract with the platform changes. This is as expected like any other NPM module.

Iteration outside of that contract can be quick and easily deployed and dynamically imported safely knowing that the dependencies will be available.

This feels like the 'best of both worlds' - safe contract with the platforms, but quick iteration and deployment outside of the platform's deployment mechanisms.

[gtrufitt]

Ah I think this is related to the below write-up https://github.com/orgs/guardian/teams/client-side-infra/discussions/12

[JamieB-gu]

I like this approach 🎉.

A variation of the above occurred to me while reading through, based on some of the other conversations we had in the meeting. To be clear, this is largely speculative, I don't claim to know if it's any better or worse (tbh it might be more fiddly) and I think @gtrufitt's approach is sensible 🙂. Feel free to pick holes in it.

Alternative

The module expects to be passed its dependencies on initialisation. Using React as an example:

type Dependency = {
  dependency: string;
  version: string;
};

const init = (react: Dependency): ModuleAPI => { ... };

The platform can provide an instance of that dependency:

// Platform code

import React from 'react';

const myModule = import('...').init({ dependency: React, version: '16.1.1' });

Optionally, because the module has been given the version of the dependency, it can decide whether that version is a close enough match to what it needs.

It might also be possible to leverage the typed API for the dependency:

type Dependency<A> = {
  dependency: A;
  version: string;
};

const init = (react: Dependency<React>): ModuleAPI => { ... };

Assuming the dependency has TypeScript declarations (e.g. @types/react) this may be able to tell us that:

a) The dependency passed has the expected API for React
b) The dependency has the correct version. In theory if you passed React 14 when the module expects React 16, the types for the APIs won't match and TypeScript would throw an error at platform compile-time.

I haven't tested the above so I don't know if it's true. That said, even if we don't actually take this approach, it would make me happy about TypeScript if we were just able to prove it were possible 🙂.

[nicl]

We want:

• clarity over who is using what (so that we can identify performance culprits and also safely and easily remove dependencies as appropriate)
• re-use - we don't want to load more than one or two versions of Preact or other larger libraries such as Emotion
• a thin interface - where possible (libraries are not too big), it is better for dynamically imported modules to bundle their own deps rather than relying on the host platform.

ES Modules

Ultimately, I think dynamic ES modules here are the solution. We provide a service that hosts supported libraries and tracks usage. Any code that wishes can import these dynamically. Common libraries like Preact/Emotion will inevitaby be included in the initial page load so subsequent calls will be cached. (ES Modules are cached both at HTTP level and parsing/execution too so extra efficient.)

The blocker here is browser support. The main polyfill for dynamic modules is lightweight but requires basic ES module support. IE doesn't support modules. :(

Can we just drop IE support given the benefits here?!

[gtrufitt]

Can we just drop IE support given the benefits here?!

There's a wider conversation about our browser support https://trello.com/c/9VcUuLxa/2343-update-recommended-browsers-list but IE11 is still an important browser for accessibility reasons: https://www.hassellinclusion.com/blog/should-accessible-websites-still-support-ie11/

That doesn't mean we cannot progressively enhance IE11+ but it does restrict the usage of this solution.

Ultimately, I think dynamic ES modules here are the solution.

I don't disagree with this, I think it's the most modern and performant direction but I think that each dynamically imported module should have a thin contract module that defines the required peerDependency it expects the platform to expose. This manages your "easily remove dependencies as appropriate" as it's clear that ModuleA needs Emotion10 by its peerDepenency, so when ModuleA has a breaking change (I.E the contract with the platforms changes), it can release a major version with new peerDependencies.

[gtrufitt]

Why did I get notified on this 😆 Did you drop IE11 support yet 😉

[sndrs]

no idea! but we have yeah. have you?