Improving MapStore Performances

[tdipisa]

Description

MapStore need to be improved in order to optimize some loading performances and functional behaviors.
We should take into account PageSpeed Insights hints to optimize performance and verify at the end the effective improvement. The list below comes from a preliminary analysis of PageSpeed Insights and this Epic collects the main involved points of interest:

• MS bundle: the bundle of MS need to be reduced, the issue Investigated on bundle sizes( too high) and memory usage during build #4297 provides a recent reports of currently used libraries. Action here are:

  • ogc-schemas: GetCapabilites parsing error for inspire_vs:ExtendedCapabilities #6489 is the biggest library (4.21 MB (11.8%)) and it is currently used mainly for some specific purposes like CSW requests and parsing of CSW results. It would be useful to remove this dependency by replacing it with a proprietary solution in MS (estimate for this are ~5/8 pts). This library, together with pdfjs and shpjs anyway is loaded dynamically, therefore should not affect the initial load of the bundle. We have to verify if it is imported somewhere else in the code.

  • Review of dependencies: the whole list of dependencies need to be analized to check if something can be excluded (clean up of unused/old dependencies).

• Mapping Libraries: by default MS uses Leaflet for mobile and OL for desktop (Cesium is an option). Mapping libraries should be loaded only when needed depending on the used device. This means that we can avoid loading OL in Mobile devices and Leaflet in Desktop if not specifically configured. At the same time Cesium JS should not be loaded if the plugin is not enabled in configuration. This point also involves the MS bundle, because Investigated on bundle sizes( too high) and memory usage during build #4297 reports also part of the bundle that currently include some OL and Leaflet scripts (is the report considering dynamic dependencies?). We should split the bundle also per mapping libraries if needed.

• Other dependencies loading: a lazy loading of scripts should be implemented to load only needed dependencies related to the application context in use (home/viewer/dashboard/stories). This means to properly split the bundle according to required functional behaviors. For example I don't need to load viewer scripts if I'm loading the home page.

• Home Page Loading: the loading of the MS's landing page can be improved also in other terms. Currently all first pages of resource categories (Maps, Dashboards, Stories) are loaded at the same time: by default only the list of Maps should be retrieved from the backend and the other should be loaded only if visualized.

• CDN: at the release we can evaluate to speed-up script loading by deploying and getting the new release bundles from a CDN.

What kind of improvement you want to add? (check one with "x", remove the others)

• Minor changes to existing features
• Code style update (formatting, local variables)
• Refactoring (no functional changes, no api changes)
• Build related changes
• CI related changes
• Other... Please describe:

Other useful information

Consider that MS already implements some lazy loading of scripts; therefore existing mechanisms, currently used for specific piece of the framework, can be extend to the entire application.
For each of the above point we can create a separate issue to connect to this epic.

Tasks

Preview Give feedback

1. Improve performances of initial page load #6735
   Epic enhancement +2
   
2. Modularize the FeatureEditor plugin from being statically loaded #9237
   Internal investigation +2
   
3. Remove deprecated styler plugins and related components #5628
   Accepted StyleEditor +2
   
4. Remove wellknown lib in favor of internal parser #9426
   Good first issue enhancement +2
   
5. GetCapabilites parsing error for inspire_vs:ExtendedCapabilities #6489
   Accepted StyleEditor bug investigation +4
   

[mbarto]

A quick report on reducing initial js loading. Playing with dynamic splitting of pages and plugins this is what we can get.

This is the current loading of js files for the homepage.

This is the after code splitting.

Instead of 2.5Mb we download about 782kb.

Moreover the code that is not red boxed can be removed from initial loading, because it's not needed by the homepage. So 782kb could be the complete payload for js code.

The experiment code is here: https://github.com/mbarto/MapStore2/tree/split_pages

[mbarto]

Some stats about home page loading payloads:

Type of Resource	Size
JS	3590Kb
IMG	1300Kb
CSS	125Kb
Font	63Kb
XHR	44Kb
HTML	21Kb

[tdipisa]

Thank you @mbarto, let's discuss this in a meeting.

[tdipisa]

As a first step, we agreed to move on with the following:

• Mapping Libraries
• Other dependencies loading
• Home Page Loading

[srtonz]

Just subscribing here as we're very interested in seeing improvements on this issue and #4297 - let me know if there's anything you need help with

[offtherailz]

Added an improvement to increase general rendering performances, see #5133
This should improve general responsiveness of the application on tile loading and when, in general, many redux actions are emitted frequently.

[allyoucanmap]

@tdipisa Updete 01 September 2022

MS bundle: the bundle of MS need to be reduced, the issue #4297 provides a recent reports of currently used libraries. Action here are:

ogc-schemas: #6489 is the biggest library (4.21 MB (11.8%)) and it is currently used mainly for some specific purposes like CSW requests and parsing of CSW results. It would be useful to remove this dependency by replacing it with a proprietary solution in MS (estimate for this are ~5/8 pts). This library, together with pdfjs and shpjs anyway is loaded dynamically, therefore should not affect the initial load of the bundle. We have to verify if it is imported somewhere else in the code.

Review of dependencies: the whole list of dependencies need to be analized to check if something can be excluded (clean up of unused/old dependencies).

ogc-schemas is still inside the source code but is not imported in the main bundle, we should remove these library anyway if possible. We analyzed some dependencies and removed from the main bundle the one related to specific components, these library will be loaded only when the related component is loaded (see #8535).

• We could improve other dependencies such as jsts included by turf not needed in the initial load (see Improve performances of initial page load #6735 (comment))

Mapping Libraries: by default MS uses Leaflet for mobile and OL for desktop (Cesium is an option). Mapping libraries should be loaded only when needed depending on the used device. This means that we can avoid loading OL in Mobile devices and Leaflet in Desktop if not specifically configured. At the same time Cesium JS should not be loaded if the plugin is not enabled in configuration. This point also involves the MS bundle, because #4297 reports also part of the bundle that currently include some OL and Leaflet scripts (is the report considering dynamic dependencies?). We should split the bundle also per mapping libraries if needed.

We improved the mapping libraries loading so now they should be loaded once the map type match the name of the library and removed also the leaflet cdn in the html head.

There are still some issue to be investigated:

• OpenLayers is included in the main bundle anyway, probably some classes are used in the utils
• we are importing all the components related to map supports as soon as the map type is selected, we should instead verify if some of these supports are importing additional dependencies and in that case opt for a dynamic loading

Other dependencies loading: a lazy loading of scripts should be implemented to load only needed dependencies related to the application context in use (home/viewer/dashboard/stories). This means to properly split the bundle according to required functional behaviors. For example I don't need to load viewer scripts if I'm loading the home page.

The modularization of plugins introduced the lazy loading of plugins (see #8423). We have still a list of static plugins because of issue related of the app lifecycle, these are the plugin still static that could be additionally be transformed in dynamic:

ContextCreatorPlugin,
ContextPlugin,
Dashboard,
DashboardsPlugin,
FeedbackMaskPlugin,
GeoStoryPlugin,
GeoStoriesPlugin,
MapsPlugin,
PrintPlugin,
RulesDataGridPlugin,
RulesEditorPlugin,
RulesManagerFooter,
LoginPlugin

We did some tests but at this time the modularization of these plugins was not improving much the initial loading (proof of concept here #8501). Anyway we could take this into account for future improvements.

Home Page Loading: the loading of the MS's landing page can be improved also in other terms. Currently all first pages of resource categories (Maps, Dashboards, Stories) are loaded at the same time: by default only the list of Maps should be retrieved from the backend and the other should be loaded only if visualized.

Some improvement has been included with the modularization reducing the number plugin in the homepage.

• We could additionally think to use the lazy+suspence react approach to trigger requests for specific resources only when the l specific list is mounted

CDN: at the release we can evaluate to speed-up script loading by deploying and getting the new release bundles from a CDN.

We did not worked on this yet. Is this still relevant?

[tdipisa]

The following are next steps to be evaluated and possibly implemented for 2024.02.00 (@allyoucanmap)

• Deprecation/dead code removal
• ogc-schema removal and refactor of the catalog tool for CSW
• Investigation on lazy loading capabilities using React and review of the current module loading