ws.next walking skeleton

[l0rd]

Walking Skeleton Details

Goal: Integrate ws.next and theia plugin + introduce the plugin registry

User Story 1: As a Che developer I want to start a ws.next based workspace that is configured to load theia as the editor and an “hello world” frontend theia plugin (using an existing che stack from the dashboard and with the editor and plugins definition available in a remote plugin registry) so that I will be able to easily extend this model developing new theia plugins.

User Story 2: Same as Story 1 but instead of the “hello world” frontend theia plugin the workspace should be configured to load a backend Che plugin that includes a theia plugin part (frontend) so that I will be able to easily extend this model developing new che plugins.

Success Criteria The plugin registry is deployed on OSIO and rh-che is configured to use it. 2 new stacks (for Story1 and Story2) are available on rh-che and I can start a workspace based on both on OSIO.

What’s in the walking skeleton Support of 2 simple stories (not a feature complete IDE), 2 che stacks, works on Kubernetes, works on OSIO

What’s not in the walking skeleton A feature complete IDE, Self Hosting, Che Terminal, Java LS, Documentation, Performance tuning, UI components, ws.next Factories

Target 11 September

Architectural Details

Che Plugin Registry

Packaging and Runtime

The registry is an http server (apache, nginx or whatever fits better) packaged as a docker image (will be eclipse/che-plugin-registry now it's garagatyi/che-marketplace:os). The docker image contains also the plugins metadata (for both Che plugins and Che editors). This list will be served statically by the apache server.

The Dockerfile (that defines the registry Docker image) and the plugins metadata will be in a repository eclipse/che-plugin-registry. A CI will build automatically the Docker image after each PR is merged (it can be the Docker hub autobuild for example).

For development purposes the registry can be started locally and in general anyone can start it's own registry in his infrastructure. For OpenShift.io we will run a Che plugin registry on DSAAS that will serve che.openshift.io.

Plugins (and Editor) Metadata

The plugins and editors metadata will be in some subfolders of the plugins folder of the git repository:

github.com/eclipse/che-plugin-registry/
    |
     --- plugins
            |
             --- che-jdt-ls       <- folder name matches the plugin id
                    |
                     --- 0.0.1    <- folder name matches plugin version
                          |
                           --- meta.yaml

The meta.yml file has the following format for a plugin

id: che-jdt-ls
version: 0.0.1
type: Che Plugin   # can be `Che Plugin` or `Che Editor` 
name: che-java
title: Che Java Plugin
description: ...
icon: https://....
url: https://.... # this points to the URL of the tar.gz of the plugin (usually this is the plugin release URL on github)

Note that both plugins and editors are under the plugins folder

Contributing (adding a new plugin to the registry)

A plugin author can contribute to the plugin registry with a PR. He will need to add a new folder matching the plugin id and version. The folder should contain a meta.yaml file:

github.com/eclipse/che-plugin-registry/
    |
     --- plugins
            |
             --- che-jdt-ls
            | 
             --- che-hello-world
                        |
                         --- 0.0.1
                               |
                                --- meta.yaml

The Che Plugin Packaging

Every Che plugin/editor should be packaged as a tarball + one or more Docker images.

A tarball contains the following files:

• che-plugin.yaml (mandatory): the plugin details like the plugin containers, endpoints and theia plugins
• dependencies.yaml (only if the plugin has dependencies): the details of the plugin required dependencies (typically a list of theia plugins)
• example.theia (only example.theia is a plugin's dependency): a theia plugin

che-hello-world.tar.gz
             |
              --- che.plugin.yaml
             |
              --- dependencies.yaml
             |
              --- hello-world.theia

The docker image contains the backend services (e.g. a language server) or command line tooling (e.g. kubectl or oc)

An example of che-plugin.yaml:

endpoints:
 -  name: "hello-ls"
    public: true
    targetPort: 8080
containers:
 - name: che-hello
   image: eclipse/che-hello:nightly   #  <-- the plugin docker image
   env:
     - name: ENV1
       value: VAL1
   editor-commands:  #   <-- commands that should be visible in the editor (e.g. theia)
     - name: build
       working-dir: "$(project)"
       command: ["mvn", "clean", "install"] 
   volumes:
     - mountPath: "/projects"
       name: projects
   ports:
     - exposedPort: 8080
editors:  #   <-- list of editors plugins that should be loaded with this plugin  
   - id: org.eclipse.che.editor.theia
     plugins: [theia-hello-world:1.0.0]   # the format is id:version
   - id: org.eclipse.che.editor.jupyter
     plugins: [jupyter-hello-client, jupyter-hello2-client]

An example of dependency.yaml:

plugins:
    - id: theia-hello-world
      version: 1.0.0 
      location: /theia-hello-world-v1.0.0.theia # use `location` if the plugin is included inside the tarball
      url: https://…                            # use `url` if the plugin is published as as a github project release

Notes:

• A Che plugin can contain only one Theia frontend plugin: in this case it won't have containers and endpoints
• For a Che editor it is basically the same except that it doesn't have dependencies (at least for now)

Che Workspace Lifecycle

Configuration

The Che plugins and the Che editor of a workspace are defined as workspace attributes:

"attributes": {
    "plugins": ["che-jdt-ls:1.0.0", "theia-hello-world:1.0.0"],
    "editor": "theia-ide:1.0.0"
}

The format of the plugins name is <id>:<version> where id and version are defined in the plugin meta.yaml

Start

1. Download of the plugins and editor meta.yaml from the plugin registry
   The URL of the plugin registry should be a wsmaster property
2. Download and extract the plugins tarballs
3. Copy the editor plugins in a PV folder that will be shared with the editor container
4. Start the editor container (e.g. theia)
5. Start the plugins containers (e.g. jdt-ls)

Subtasks

OSIO team

• Workspace.next - CheService model adjustment Workspace.next - CheService model adjustment #9964
• Load Theia in Workspace.Next workspaces Load Theia in Workspace.Next workspaces #9870
• Service discovery of APIs running in tooling containers of Workspace.Next Service discovery of APIs running in tooling containers of Workspace.Next #9913
• Update Che workspace config model: add CheEditor Update workspace.next model: add CheEditor #10203
• Implement Plugin broker
• Download Che Plugins and Editor meta.yaml and invoke Che Plugin Broker Delivering of ChePlugins binaries to the editor container #10561
• Update Che workspace config model: replace Feature/CheService with ChePlugin Rework Workspace.Next services/features to ChePlugin model #10202
• Make che-plugin-broker push result to JSON_RPC of Che master Make che-plugin-broker push result to JSON_RPC of Che master #10878
• Add an ability to use different PVC strategies in Walking skeleton instead of hardcoding behavior of common strategy Add an ability to use different PVC strategies in Walking skeleton instead of hardcoding behavior of common strategy #10879
• BONUS Support of RWX Mode and running containers in multiple pods

IDE2 team

• Implement the theia-hello-world plugin. Package it as a .theia file. Publish it in a github release
• Embed the theia-hello-world plugin inside a tar.gz + che-plugin.yaml + dependency.yaml and publish it in a repository
• Package theia as a che-plugin [Walking Skeleton] Package Theia as a Che Plugin #10648
• Implement/Package che-hello-world plugin [Walking Skeleton] Implement/Package plugin che-hello-world #10649
• Create a Che Stack to support User Story 1
• Create a Che Stack to support User Story 2

PLATFORM team

• Move the Plugin Registry to its own repository Move the Plugin Registry to its own repository #10615
• Support multi-user Che Plugin Broker doesn't work on multi-user Che #11071
• Setup a CI for the Plugin Registry [Walking Skeleton] Setup a CI for the Plugin Registry #10651
• Factory and stack features are not supported in Workspace.Next Factory and stack features are not supported in Workspace.Next #10368
• Deploy the Plugin Registry on OSIO Deploy the Plugin Registry on OSIO #10652
• Edit Workspace Attributes from the dashboard [Walking Skeleton] Edit Workspace Attributes from the dashboard #10650
• Move Plugin broker to start of Kubernetes runtime Move Plugin broker to start of Kubernetes/OpenShift runtime #11042
• Implement permissions support on Json/RPC calls Implement permissions support on Json/RPC calls #10861

[ghost]

@l0rd what is Che backend plugin? Is this a scenario where a backend plugin is a sidecar? If not, don't we want to target such a scenario?

[l0rd]

@eivantsov yes the scenario is about that. A che plugin that runs in a sidecar container and communicates with a theia plugin to provide the UI functionality (like a language server and its theia client).

What I wanted to stress is that the user need to specify one plugin only (che-hello-world:1.0.0) and this will be composed by a backend (che plugin) and a frontend (theia plugin).

[ghost]

@l0rd thanks. got it. SO, a feature, speaking in old fashioned ws.next terms :)

For such a scenario, there's one important dependency - Theia LSP client being able to connect to LS over host:port.

[l0rd]

Theia clients should contact a Che service using the Plugin endpoint name. No port (port 80).

For this first scenario it's ok to hardcode the endpoint name in the theia client. We should make it a parameter to pass to the theia client in future iterations.

[ghost]

@l0rd not Che API (its clear that we have a plugin that provides such endpoints), but LS running in tcp mode. That would be the most interesting scenario - Che plugin bringing over Theia plugin + a sidecar.

[skabashnyuk]

A couple of remarks and changes.

Marketplace content

theia-plugins/
	|---- theia-hello-world-v1.0.0
                  |---- meta.yaml
                  |---- theia-hello-world-v1.0.0.tar.gz
                           |---- package.json
                           |---- ......  
che-plugins/
	|---- theia-ide-v1.0.0/
	      |---- meta.yaml
	      |---- che-editor.yml

	|---- che-hello-world-v0.0.1/
	      |---- meta.yaml
	      |---- che-plugin.yml

Where

che-plugin.yml

version: 0.0.1
type: Che Plugin
name: che-hello-world
id: che-java
title: Che Hello plugin
description: ...
icon: https://....
endpoints:
 -  name: "hello-ls"
    public: true
    targetPort: 8080
containers:
 - name: che-hello
   image: eclipse/che-hello:nightly
   env:
     - name: ENV1
       value: VAL1
   editor-commands:
     - name: build
       working-dir: "$(project)"
       command: ["mvn", "clean", "install"] 
   volumes:
     - mountPath: "/projects"
       name: projects
   ports:
     - exposedPort: 8080
editors:
   - name: org.eclipse.che.editor.theia
     plugins: [theia-hello-world:1.0.0]
   - name: org.eclipse.che.editor.jupyter
     plugins: [jupyter-hello-client, jupyter-hello2-client]

che-editor.yml

version: 1.0.0
type: Che Editor
name: org.eclipse.che.editor.theia
id: theia-ide
plugins: theia-plugins
title: Theia for Che
Description: ...
Icon: https://....
endpoints:
 -  name: "ide"
    public: true
    targetPort: 8080
containers:
 - name: theia-ide
   image: eclipse/che-editor-theia:nightly
   env:
       - name: theia-plugins
         value: file://plugins/
       - name: ENV1
         value: VAL1
   editor-commands:
      - name: reload-plugins
        working-dir: "$(plugins)"
        command: ["node", "reload"] 
   volumes:
       - mountPath: "/plugins"
         name: plugins
       - mountPath: "/project"
         name: project
    ports:
       - exposedPort: 8080

Notes

• In a marketplace, our first level citizens are che-plugin and theia-plugin. That's why I would like to remove Che editor from the root into to che-plugins section
• In che-plugin.yaml/editors/name - is reference to che-plugins/theia-ide-v1.0.0/che-editor.yml/name
• in che-editor.yml/plugins - is reference about kind of plugins in che-plugin.yaml/editors/[]/plugins . This is also important for the usecase1 to make sure plugins are compatible with an editor.
• I see two issues with .spec.initContainers idea. It's very connected to K8s/Openshift and will be
  complicated for Docker. 2. It will have troubles in a multicluster environment when a master is working in one cluster and workspace working in another.

Usecase 1

1. theia plugin is referenced with extra theia-plugin/ since we don't want to duplication it in che-plugin section.

# workspace config
plugins:
- theia-plugin/theia-hello-world:1.0.0  
editor: theia-ide:1.0.0 

2. The user starts the workspace and wsmaster downloads the editor metadata from a marketplace to be able to start theia container.

curl $(marketplace_url)/che-plugins/theia-ide-v1.0.0/che-editor.yml

3. Thiea container will query ws-master and figure out a list of Theia plugins to start. After that query marketplace to download Thiea plugins

curl $(marketplace_url)/theia-plugins/theia-hello-world-v1.0.0/theia-hello-world-v1.0.0.tar.gz

Usecase 2

# workspace config
plugins:
- che-hello-world:1.0.0
editor: theia-ide:1.0.0 

2. The user starts the workspace and wsmaster downloads the editor and plugin metadata from a marketplace to be able to start theia container and hello container.

curl $(marketplace_url)/che-plugins/theia-ide-v1.0.0/che-editor.yml
curl $(marketplace_url)/che-plugins/che-hello-world-v0.0.1/che-plugin.yml

3. Theia container will query ws-master and figure out list of theia plugins to start. After that query marketplace to download Thiea plugins

curl $(marketplace_url)/theia-plugins/theia-hello-world-v1.0.0/theia-hello-world-v1.0.0.tar.gz

[garagatyi]

I see two issues with .spec.initContainers idea. It's very connected to K8s/Openshift and will be
complicated for Docker. 2. It will have troubles in a multicluster environment when a master is working in one cluster and workspace working in another.

I agree that we should not use kubernetes as a common base and that there are might be ussies that prevent master from using init containers directly without additional operation. Apart from that copying binaries from marketplace to workspaces may lead to heavy load of workspace master and bad response time to API requests.
So +1 to consider making downloading inside of the workspace somehow. Sergii suggested downloading it by Theia, but this would require us to add somehow custom logic to Theia.
I can suggest another approach - we can specify another way of delivering of plugins binaries. I mean docker images. If we have a dockerimage property in plugin definition and maybe path to binaries inside of this image we may use it as init containers in kubernetes. It is similar to what Mario suggested but doesn't involve actual copying of files on workspace master side. I think that we can also use it on Docker, for example as data containers added to the workspace.
But as Mario previously said to me a variant with the copying might be the easiest one for the POC which is what this issue about.

The flow looks pretty complete. I can just suggest considering to include factories into it. Maybe as a second step. And I just figured out that Che factories feature doesn't have attributes. So, we can't use factories with workspace.next. Should we add those to factories?

[benoitf]

Making the workspace in charge of provisioning the plugins before the container is started allow to handle more efficiently

• caching pattern
• authorization pattern
• access/credentials to some registries/marketplace/whatever. (we probably don't want to give credentials to theia container in order to download)

[benoitf]

@skabashnyuk
I've a question on your change:

why in usecase1 you prefixed with "theia-plugin/" the plugin ?
AFAIK we give the id:version and then that's all, marketplace know if it's a theia plugin, a che plugin, a whatever plugin, etc.
so

plugins:
- theia-plugin/theia-hello-world:1.0.0  

is enough

here what you said

Usecase 1
theia plugin is referenced with extra theia-plugin/ since we don't want to duplication it in che-plugin section.
# workspace config
plugins:
- theia-plugin/theia-hello-world:1.0.0  

[l0rd]

@eivantsov yes let's support HTTP only for this walking skeleton

@skabashnyuk great review thanks a lot of interesting points:

Change proposals that I have integrated in the issue body

• che-plugin.yml and che-editor.yml samples
• che-plugin.yaml/editors/name - is reference to che-plugins/theia-ide-v1.0.0/che-editor.yml/name

Change proposals that I don't think are needed for the skeleton

Implementing a walking skeleton is about making the minimum to build something that works end to end. With that in mind, even if these are good ideas, I would avoid to include it the skeleton:

• the meta.yml files: it looks like at this point we don't need them
• prefixing plugins with theia-plugins and che-plugins (in the marketplace URL and in the workspace config): it looks like we are adding a convention that no component really will use. That make the plugin definition more error prone (user need to remember the prefix) and complicated (user need to understand the difference between Che and Theia plugins). I would rather leave all plugins and editors at the same level and prefix it (or not) when we won't have other choices.
• validation of the plugin type: I would like to postpone it

Proposals that need further discussion

• theia plugins have .theia extension but in your proposal they are tar.gz. Can we just leave them as .theia files (extraction will be a theia responsibility)?
• in the first scenario the theia plugin has both a frontend part and a backend part, in the second scenario the theia plugin has only a frontend part. You treat them as the same theia plugin but these are really two different plugins. Do you agree?
• the theia plugin used in the second scenario is useless without a backend part: it should not be available as a standalone plugin in the marketplace. For that reason I had included it inside che-hello-world.tar.gz. Does it make sense?
• in your proposal Che plugins are served as files in a folder, not as a tar.gz. This would be ok if we don't include other files than che-plugin.yml. But as discussed in previous point we may need to distribute che plugins that include one ore more theia-plugins. In that case packaging plugins in a tar.gz would make more sense what do you think?
• about .spec.initContainers: I agree with you and I feel hat's the weakest part of the skeleton. It may be ok for the walking skeleton but we need to find a better solution after that. Your proposal to have theia quering Theia plugins is interesting but has it's drawbacks too as emphasized by @benoitf. Maybe we could investigate @garagatyi solution using a docker image?

@garagatyi ok to investigate using a docker image to copy the theia plugins in the plugins folder. That may be simpler than spec.initContainers.

[skabashnyuk]

@l0rd ok

[tamirmenahem]

Do you have resource consumption (e.g. memory) figures impact of multiple sidecar containers comparing to single container with multiple agents? Is it worth for example to have a dedicated container just for the IDE (Serve Static Resources)? Do you have target memory figure for typical developer? (don’t get me wrong - I totally understand the benefit of the multiple containers: decoupling, stability, LCM, etc.)

[garagatyi]

IDE won't be a static hosting service. IDE always has some backend, e.g. API that provides access to source files.
But the question regarding memory consumption of overall toolings list still remains. Personally, I don't have the answer for the time being.

[l0rd]

@tamirmenahem 2 processes in 2 distinct containers or in one container should consume the same amount of memory. It would have been a concern if we had to run more JVMs (one per containers). But that's not the case (IDE is run by nodejs, jdt.ls is run by the JVM, exec agent is go etc...).

Anyway the implementation of this skeleton should provide rapidly a factual answer and confirm (or not) the initial assumptions.

[garagatyi]

I created a general task of ChePlugin binaries delivering #10561 and moved all the corresponding subtasks from this epic to that issue.

[rsankarprgs]

Sorry for asking pretty basic question. I am new to this world.

What is the difference between che-plugin & theia-plugin? I read through this issue, but it isn't clear to me.
My understanding is that che-plugin is a wrapper over theia-plugin which tells the theia ide container which plugins to load ? Am I correct?
Does che-plugin relation to theia frontend and backend plugins differ?

Thanks,
Ravi

[benoitf]

@rsankarprgs you're correct.

A theia plug-in is a plug-in for the IDE/editor (that can run on theia nodejs side or inside browse web worker)

A che plug-in is a super set of plug-ins.
It can include :

• some container definitions (like if you need some docker images in your workspaces)
• some theia plug-ins (these plug-ins will be loaded by Theia)

[rsankarprgs]

Thanks.

[skabashnyuk]

Added a note that Move preparing of RuntimeContext to the async phase of workspace start #8989 may not be delivered until 11 September

[benoitf]

Fix for che-service plug-in : ws-skeleton/che-service-plugin#2

[garagatyi]

I moved the task with QE tests to the Eclipse Con epic since during our meeting with Mario we decided that it is not blocking this epic

[garagatyi]

@l0rd Looks like we can close this epic since all the tasks were completed. Do you mind if I close it?

[l0rd]

@garagatyi yep, the skeleton has been built ☠️

Congrats to all the engineers involved 🎉 🎉