Improve performance for big data sets

[stefandesu]

When working with large systems without hierarchy or after working in the application for a long time without reloading, i.e. every time a lot of items are stored in the Vuex store, there are performance issues that affect the whole application. The cause is likely Vue's reactivity for all the objects stored in Vuex (thousands of objects each with a bunch of properties that all have to be watched by Vue). Right now, the performance penalty is not too big (it's still usable, just a little slow), so this is not the highest priority, but long term we should aim to improve the performance for these cases.

Related: vuejs/vue#4384

[stefandesu]

Performance was already drastically improved by recent refactoring (as a side effect), but certain things still slow down the application. What I noticed in particular is that both selecting and clearing a concept scheme with a large number of top concepts takes a long time. I would guess that it takes that long for Vue to render or remove the thousands of new nodes in ConceptTree. Ideally, ConceptTree would only render the nodes that are currently in view, but this is not a trivial thing to implement (though not impossible of course).

Edit: I was able to confirm this by making ConceptTree just show a subset of all items. In that case, even with vocabularies with 5000 top concepts, the loading times were insignificant.

[stefandesu]

Reopening for #513. Suggestion: Try to improve performance for large data sets by inlining ConceptListItem into ConceptList (because it's only used there).

[stefandesu]

Looking back at this issue, I think the main problem is the big number of Vue components that are created with big datasets. One step would be to inline as many components as possible (where it makes sense of course), as I mentioned in the previous comment. It would also be good to simplify the code of ConceptList and ConceptListItem. If it's possible, it would make sense to turn off certain functionality for items that are not currently visible.

[stefandesu]

If we use vue-virtual-scroller, we can keep using a proper component for ItemName, even in the concept lists, but have only the ones rendered that are actually shown. This should improve performance for long lists significantly.

[stefandesu]

I'm working on getting vue-virtual-scroller to work in Cocoda. Progress is slow because everything's pretty complicated. We will also lose the scrolling animation when a concept is selected (scrolling still works, but just not animated). But when this is properly implemented, we could have VERY long lists without performance issues.

[stefandesu]

I've been having more problems with this and opened an issue in vue-virtual-scroller for something that might not be possible to do (yet). So I'm putting this on hold. Maybe we have to reconsider and find a different solution.

[stefandesu]

Here's a custom virtual scroller implementation that seems to be doing exactly what we need: https://stackoverflow.com/questions/48515997/how-to-handle-vue-2-memory-usage-for-large-data-50-000-objects

Example: https://codepen.io/zupkode/pen/MWwgLyb

[stefandesu]

With #584, this isn't really an issue anymore and even lists with 50k concepts work fairly well. There's still room for improvement, especially when it comes to reactivity, but that will be a separate issue.