Skip to content

[Proposal] MapStore Modular Plugins

Jump to bottom
Lorenzo Natali edited this page Jul 20, 2022 · 11 revisions

Overview

The goal of this improvement is to allow MapStore runtime load of plugins and their side-effects, to reduce the size of the initial JS downloaded and to allow to create extensions that do not affect the other pages.

Proposed By

  • Lorenzo Natali

Assigned to Release

The proposal is for 2022.02.00 (first version integrated with GeOrchestra).

State

  • TBD
  • Under Discussion
  • In Progress
  • Completed
  • Rejected
  • Deferred

Motivation

The improvement is proposed to make MapStore initial load faster, and allow sandboxing of plugins in the same time, by loading epics/reducers in a second time.

Tecnical Background

Actually MapStore provides some partial implementations for dynamic load of plugins:

  • loadPlugin/enabler: Defined as lazy plugins allows to load only the component of plugins. It is actually used by: Print, MapImport, ThematicLayer. It can be replaced by React.lazy + a connect for the enabler. So we may decide to deprecate this to clean up the code.
  • Extensions: allows to load a plugin for a module. Implemented by withExtensions enhancer applied to the StandardApp allows to import plugins definition, but these are loaded only at the beginning.

The new feature should allow to:

  • Load the code only when effectively required (e.g. when plugin is rendered)
  • Allow the extensions to do it too.

After a first investigation and a implmentation attempt, we had a sync with dev team.

We noticed that @allyoucanmap already developed a similar work for GeoNode, that looks a lot similar to first attempt, so to illustrate the main concept we can refer to it:

Here the plugins.js file imports dynamically the files:

https://github.com/GeoNode/geonode-mapstore-client/blob/master/geonode_mapstore_client/client/js/plugins/index.js

an hook here:

https://github.com/GeoNode/geonode-mapstore-client/blob/master/geonode_mapstore_client/client/js/hooks/useLazyPlugins.js

is used in the page to load pending plugins:

https://github.com/GeoNode/geonode-mapstore-client/blob/master/geonode_mapstore_client/client/js/routes/Viewer.jsx#L76-L79

The difference between this solution and the one we want to apply to the main project are:

  • The plugins in this case are loaded in page context. For the main project we should instead load plugins once for all, incrementally on needing
  • A nice to have is to create several modules, reusing and powering the extensions system, in order to make MapStore more modular.

Proposal

Definitions

Module API

An API provided by MapStore StandardApp (using an enhancer or some hooks) to the react context to allow to:

  • Register plugins (with a part called pluginManager)
  • register reducers and epics (with a part called storeManager)

MapStore Module Plugin

A MapStore Module Plugin is a MapStore Plugin that has an initial definition that needs to be completed, using the module API, provided by MapStore, to:

  • Register Plugins, epics, reducers

The PluginContainer or the Page component (to decide at development time) will be enhanced to load plugins implmentations incrementally using the pluginManager, that will cache the implementations, before the plugin have to be effectively rendered.

On implementation load the plugin can:

  • Register reducers and epics to the store
  • Define/ReDefine plugins implementations (maybe using with loadPlugin).

Design

The general idea is to delegate to the the possibility to dynamically load implementation of plugins and so register reducers, epics and "complete" plugins definitions on the fly.

The main differences between the solution used in GeoNode and the one proposed here are:

  • Instead of calling augmentStore after import, delegate to the plugin function to do the registering plugins/epics/reducers throw the Module API. The load operation in the plugin should be delegated anyway to a Promise, and the caller (to define page or plugincontainer) have to simply wait that all the load promises are resolved to render.
  • We need to evaluate if doing the load operations enhancing the Page or the PluginRenderer, taking into account the possible reuse/refactor of actual loadPlugin/enabler system and the needing that may be required at development time (e.g. showing a loading mask...).
  • Generalizes the augmentStore with the concept of storeManager and the hook used with the concept of pluginManager, delegating to the plugin (usually the module transformation utility for the plugin)
  • The separation between module plugins (GNlazy) should be applied at the later level possible, in the Page/PluginContainer, anyway just before plugin rendering (in the GN implementation is done before, but making it transparent to the rest of the system should simplify the general implementation).

Plugins

A utility function will allow to transform a plugin into a module plugin with proper options to override some or all their parts. At development time we will evaluate if makes sense to enhance the current loadPlugin API with possibility of adding the rest of the definition of plugin (containers, epics, reducers) or to reimplment it.

Also the extensions can be transformed into module plguins using the same utility functions. In this case the withExtensions enhancer will load the initial definition of plugin, with its load function, and then when the extension is effectively rendered it will load the rest of the plugin.

This is the most delicate part of this proposal and so we should verify it with a proof of concept before to proceed with the rest of the implemnentation

Flexibility of modular design

This design that implements the module API allows to modularize MapStore in multiple ways.

Here some ideas:

  • The module API can be reused in many context, allowing to load a whole page at runtime, with it's own plugins, in a single bundle.
  • In the future we can extend this API with the possibility to add routes to the router, and so pages. This may make MapStore event more modular and pluggable.

For the moment we will try to limit the loading of Modules only by plugin theirself.

Suggested Implementation

Store Mananger

The store is already provided in the react context. The idea is to add to the store the possibility to add/remove reducers and epics with a store manager like this. The manager should have additional functions to register/unregister epics, in the way that actual augmentStore does on startup (note the usage of rootEpic.next(epic) that doesn't stop the current epics running) with the possibility to unregister the epic too, by name.

Finally we should change the implementation of augmentStore to use the new methods from storeManager Api.

Module API support

The module support is provided at StandardApp level, with an enhancer called withModuleSupport.

The enhancer, applied after the withExtensions enhancer, will add the following functionalities:

  • Override pluginsDef property in a similar way of withExtensions registering plugins loaded dynamically.
  • Adding a moduleAPI objects to the react context.

The pluginManager and the storeManager will take care of registering and caching registered epics/reducers/plugins.

Plugins modularization utility

A function should be provided to transform a plugin into a module. note some epics should need anyway to be loaded at runtime. For this reason, the plugin initial definition should allow anyway to register some epics/reducers, while the load allows to register all the epics that are needed at runtime.

Basically the implementation in MapStore is very good, the only thing to change is to invert the control.

Actually it has a load function that does the import.

function toModulePlugin({name, loadModule, overrides}) { // other parameters may be needed
    const getLazyPlugin = ({pluginManager, storeManager}) => { // functions of Module API passed by the container.
        return loadModule().then((mod) => {
            // register plugins and epics to the API
            };
        });
    };
    getLazyPlugin.isModule = true;
    return getLazyPlugin;

Isolating epics

The original needing of sandboxing some extensions, not well written, that may affect the rest of the application made we think to allow to unregister epics/reducers. Anyway this may become very hard to manage. We may suggest instead some guidelines for writing plugins. Dynamic loading of the code when effectively used is already an improvement in that way.

In order to satisfy this task, we may simply add an option to createPlugin utility (verify it is reusable for extensions) isolateEpics. This option should block the epics listening if the component of the plugin is not rendered.

Here some pseudo code to do it:

[...]
const startStop = new Subject();
const rendered$ = startStop.asObservable();

if(isolateEpics) {
    // function to isolate an epic (this is only a attempt implementation, please try and see what makes sense).
    const isolateEpic = (epic) => (action$, store) => epic(action$.let(semaphore(rendered$.startWith(false))), store).let(semaphore(
            rendered$.startWith(false)
        ))
    
    // enhancer to apply to the plugin component
    componentEnhancer = Cmp => props => {
         useEffect( () => {
              startStop.next(true);
              return () => startStop.next(false);
          }, []);

     }
     // ...enhance the component
     // wrap the wpics with isolateEpic
     epics = Object.entries(epics).reduce( (out, [k, epic]) => ({ [k]: isolateEpic(epic) })
     // ...return the plugin modified
}

Suggested Tasks

  • Implement storeManager

  • Implement pluginManager (caching etc...)

  • Implement support for Module API

  • Implement utility function

  • Implement utility to isolate epics

  • Optional:

    • Deprecate loadPlugin/enabler in favor of React.lazy (enabler function can be easely applied to a connect and used with React.lazy to have the same effect of defer the loading of certain big libs to the effective plugin opening). Note: loadPlugin is actually used by withExtension, anyway we can clean up also that implementation.
Clone this wiki locally