Next Gen Che as an extensible platform

[slemeur]

Description

Motivations

There is a difference between white labelling a product and making a product extensible.

When, someone is white labelling Che, he is going to create a complete new product, add his own specific tools and manage the distribution of the new product. This worked well with the current Che.

But for a product to be an extensible platform - it requires different aspects. As contributor, I don’t want to deliver my tools as a custom Che and manage the distribution myself, I’m expecting to find a base platform where I can propose my tools to users. As a user, I’m expecting to find a list of plugins that I can add to my Che without having to rebuild a new custom Che.
For example: Mercurial plugin, Custom theme...

In order to achieve this, we will introduce Theia as the future of Che's IDE, and build the foundations of the next generation of Che to become a complete extensible platform.

Theia Integration

Our current IDE is built with GWT and our stack is quite complex, requiring to understand many different layers of the architecture. As a result, the developer experience is not seen as efficient and does not allow fast turnarounds (compared to what a developer would expect when working on web applications).

In order to increase the engagement of the community, we need to provide Che's IDE with a more up-to-date technology stack.

The community has shown a growing interest for Theia and during the latest Eclipse Con, the Eclipse community was looking into a solution which would bring the power of Che's workspaces + Theia IDE - as an extensible platform for cloud native tooling.

We want to enable Theia as soon as possible in Che and allow anyone being able to use Theia in Che.

Until Theia replace the current GWT IDE, Theia could be activate either:

• by a user preferences (available in the dashboard)
• as a flag used when starting Che
• as che env variable.

Define new architecture for ws-agent

We need to define a new architecture for the ws-agent which will keep only the pieces which will be needed for Theia.

Moreover, for a microservice based approach, we need to consider that the server side of a plugin could be getting a different set of dependencies than the ones used by the default IDE. The server side of a plugin, might be written in NodeJS and run in a sidecar of the other workspace's containers.

We could have two kind of server extensions:

• plugins of ws-agent (like REST components written in java, can be dynamically added/removed on top of the ws-agent without stopping the agent)
• container providing the service. Of course only for an hello world it may consume memory but if someone has a full blown set of services, it may be helpful)

Packaging

We will need to define a packaging format for Che plugins.

The packaging might include:

• client side plugin
• server side plugin
• (or both)
• metadatas

Next Gen Che Dogfooding

In order to dogfood the next gen Che, we will start building real plugins with it which will be leveraging the extensibility model.

We will be targeting

• the Git Panel: As part of this epic, all the features from the Git Panel will not need to be implemented, we will only target the features which integrate the git status in project explorer and editor.
• Work on the project explorer and reuse the icons, implement few convenient features (collapse, expand all)

Linked Issues

Sub Tasks

[garagatyi]

With server-side plugins running with their own dependencies and probably in their own container with containers limits on RAM we may face the situation where developer tools take a lot of RAM in the workspace. For example:

• ws agent takes 500MB and uses Java
• Java/Maven LS takes 1500MB and uses Java in a second container
• PHP LS takes 300MB in a third container
• Some NodeJS plugin takes 300MB in a fourth container
• Git plugin takes 500MB in a fifth container
  and so on and so on. This brings the situation where dev tools without any user application take gigabytes of RAM. In this situation, there is the other problem - when a plugin doesn't perform any active operations it still consumes all the RAM that is provided for it. Also when plugin declared 300MB RAM for the peak consumption but now uses 50MB it still would take 300MB and waste 250MB of RAM.
  I know that @slemeur dreams of the on-demand launching of plugins in separate containers. This would solve one side of the problem but I'm not sure how to implement this on OpenShift in common case. Addressing the other problem with wasted RAM reserved for plugins peak RAM consumption should also be considered.
  I do not have a solution to these issues right now and bringing up this to ensure that these problems don't get forgotten. I would like to participate in a discussion of these problems if someone has any ideas about that.

[benoitf]

I would also add for ease the pluggability of user docker image to move workspace agent to its own docker image/container so it would be easy to monitor memory used by the "user container" and the "tooling memory" required by "workspace agent" (exec agent and terminal can still be started into the user docker image as they're native tools)

@garagatyi it may allow also use to adjust memory for your current usage.
If I import che project I might require a "jdt.ls" container with many memory but If I'm playing with an hello world project, I can easily reduce the memory.

Also, if you're doing php you may not need all Java plugins enabled into memory as well (because for example for now all workspace agent plugins are loaded into the memory even if you don't need them for example when you work on different languages)

[sr229]

This would also enable packaging of the IDE to Electron as Theia has support for Electron packaging, thereby allowing the usage of the client to connect to the Che server without needing to open the browser.

[slemeur]

Closing this epic is covered over different efforts.

• A first integration of Theia has been done with the introduction of a dedicated stack: Theia stacks from Florent's theia prototype  #8600
• Some plugins have been created for Theia running on Che: https://github.com/eclipse/che-theia-machines-plugin / https://github.com/eclipse/che-theia-github-plugin
• Host mode is currently in progress: Run separate instance of Theia server with plugin that is developing #9212
• An isolated plugin system for Theia needed with Che's workspaces is in progress: Plug-in System eclipse-theia/theia#1482
• Workspace Agent will not be needed and tools will be provided as sidecars: Workspace.Next  #8265