Consume VS Code built-in extensions in Che-Theia

[benoitf]

Is your enhancement related to a problem? Please describe.

VS Code has builtin extensions https://github.com/microsoft/vscode/tree/master/extensions

Lot of them are grammars.
We should rely on these extensions instead of https://github.com/eclipse-theia/theia/tree/master/packages/textmate-grammars

For example there is no syntax highlight when editing Dockerfile

while there is one in VS Code: https://github.com/microsoft/vscode/tree/master/extensions/docker

Describe the solution you'd like

Eclipse Che Theia should embed these built-in VS Code extensions to match behaviour of VS Code.

Describe alternatives you've considered

N/A

Additional context

https://www.npmjs.com/search?q=vscode-builtin

Accepted

@theia/vscode-builtin-bat
@theia/vscode-builtin-clojure
@theia/vscode-builtin-coffeescript
@theia/vscode-builtin-configuration-editing
@theia/vscode-builtin-cpp
@theia/vscode-builtin-csharp
@theia/vscode-builtin-css
@theia/vscode-builtin-debug-auto-launch
@theia/vscode-builtin-docker
@theia/vscode-builtin-fsharp
@theia/vscode-builtin-go
@theia/vscode-builtin-groovy
@theia/vscode-builtin-grunt
@theia/vscode-builtin-gulp
@theia/vscode-builtin-handlebars
@theia/vscode-builtin-hlsl
@theia/vscode-builtin-html
@theia/vscode-builtin-ini
@theia/vscode-builtin-jake
@theia/vscode-builtin-java
@theia/vscode-builtin-javascript
@theia/vscode-builtin-json
@theia/vscode-builtin-less
@theia/vscode-builtin-log
@theia/vscode-builtin-lua
@theia/vscode-builtin-make
@theia/vscode-builtin-markdown
@theia/vscode-builtin-merge-conflict
@theia/vscode-builtin-npm
@theia/vscode-builtin-objective-c
@theia/vscode-builtin-perl
@theia/vscode-builtin-powershell
@theia/vscode-builtin-pug
@theia/vscode-builtin-python
@theia/vscode-builtin-r
@theia/vscode-builtin-razor
@theia/vscode-builtin-ruby
@theia/vscode-builtin-rust
@theia/vscode-builtin-scss
@theia/vscode-builtin-shaderlab
@theia/vscode-builtin-shellscript
@theia/vscode-builtin-sql
@theia/vscode-builtin-swift
@theia/vscode-builtin-typescript
@theia/vscode-builtin-vb
@theia/vscode-builtin-xml
@theia/vscode-builtin-yaml

Known issues

• support vscode theming eclipse-theia/theia#4831 for:
  @theia/vscode-builtin-theme-abyss
  @theia/vscode-builtin-theme-defaults
  @theia/vscode-builtin-theme-kimbie-dark
  @theia/vscode-builtin-theme-monokai
  @theia/vscode-builtin-theme-monokai-dimmed
  @theia/vscode-builtin-theme-quietlight
  @theia/vscode-builtin-theme-red
  @theia/vscode-builtin-theme-solarized-dark
  @theia/vscode-builtin-theme-solarized-light
  @theia/vscode-builtin-theme-tomorrow-night-blue

• Implement SelectionRange and SelectionRangeProvider eclipse-theia/theia#6623 for:
  @theia/vscode-builtin-css-language-features
  @theia/vscode-builtin-html-language-features
  @theia/vscode-builtin-json-language-features

• @theia/vscode-builtin-debug-server-ready@0.2.1 does not work eclipse-theia/theia#6628 for:
  @theia//vscode-builtin-debug-auto-launch

• @theia/vscode-builtin-debug-auto-launch@0.2.1 does not work eclipse-theia/theia#6630 for:
  @theia/vscode-builtin-debug-server-ready

• @theia/vscode-builtin-extension-editing@0.2.1 does not work eclipse-theia/theia#6634 for:
  @theia/vscode-builtin-extension-editing

• [vscode] support CustomEditor API eclipse-theia/theia#6636 for:
  @theia/vscode-builtin-image-preview@latest

• There is no auto-completion for @theia/vscode-builtin-markdown-language-features@0.2.1 eclipse-theia/theia#6646
  @theia/vscode-builtin-markdown-language-features

• Error registering grammar configuration for @theia/vscode-builtin-php@0.2.1 eclipse-theia/theia#6672 for:
  @theia/vscode-builtin-php

Check license

• @theia/vscode-builtin-git
• @theia/vscode-builtin-git-ui@latest
• @theia/vscode-builtin-emmet
• @theia/vscode-builtin-markdown-language-features

There is no public release

• @theia/vscode-builtin-search-result
• @theia/vscode-builtin-emmet

Rejected

@theia/vscode-builtin-typescript-language-features:
We have a typescript che-plugin:

@theia/vscode-builtin-php-language-features:
We have a php che-plugin:

[tsmaeder]

Not sure what the current story is with theia and vscode built-ins. We would have to find out how to consume those.

[benoitf]

Theia example assemblies should also move to vscode built-ins eclipse-theia/theia#6260 (discussed on dev meetings) to be aligned with VS Code.
I don't know if it is answering the current story ?

[tsmaeder]

I guess the "todo" here is this:

1. Figure out how built-ins are published from https://github.com/theia-ide/vscode-builtin-extensions
   We have some special requirements, for example, the typescript language features should not be included, since we have a separate Che plugin for that.
2. Update che:theia init to install the built-in plugins in a usable way: without the built-ins, theia will not be fully functional. Don't forget to exclude extensions we no longer need. Maybe we need to exclude some built-ins for now, because they conflict with content from Che 7 plugins.
3. Update the process in Contributing.md in che-theia if necessary. We absolutely need a working self-hosting process.
4. Update the che-theia image build scripts

[tsmaeder]

The extensions are published to NPM. There is a conversation related to this here: https://spectrum.chat/theia/dev/consuming-vs-code-built-ins~48a3feb9-fddb-42c3-9e0f-a317d81488c0

[tsmaeder]

While most extensions (exception git, I believe) are MIT licensed, we neeed to make sure we're ok consuming the extension: a CQ might be needed,

[tolusha]

While most extensions (exception git, I believe)

Where does this caution come from? I don't see anything suspicions https://www.npmjs.com/package/@theia/vscode-builtin-git

[tsmaeder]

I seem to remember that one of the dependencies of the git extensions is LGPL, but i don't remember the details. I think @vparfonov would know the details.

[tsmaeder]

@tolusha thanks for the filed issues: it would be interesting to know for each issue whether it causes a loss of functionality relative to current che-theia.

[marcdumais-work]

I seem to remember that one of the dependencies of the git extensions is LGPL, but i don't remember the details. I think @vparfonov would know the details.

Hi @tsmaeder - the offending LGPL dependency is jschardet@1.6.0, for which we already have an approved CQ

[tsmaeder]

@marcdumais-work thanks for the info, makes our job easier.

[tolusha]

How to validate license

1. Download extension from https://github.com/theia-ide/vscode-builtin-extensions/releases/tag/v1.39.1-prel
2. Generate package-lock.json
3. Use https://clearlydefined.io/workspace to validate open source licenses from the list https://opensource.org/licenses/alphabetical

[marcdumais-work]

Hi @tolusha ,

I was thinking about the license-check for these extensions, e.g. for their eventual use as part of Theia's example application, but was unsure how to proceed. The GH release above has each built-in extension packaged as a .vsix, which is basically a zip file with some expectations about the content.

Do you have a plan how you'll achieve step 2? One not so straight-forward way I can see is to extract each extension's package.json and figure-out what the runtime dependencies are, then obtain the recursive list of their dependencies as well, and check all these for license compatibility.

But I am hoping there is a better way...

[tolusha]

I use the following command to generate package-lock.json

> npm install --package-lock

I performed manual check for some extensions.
i.e. : downloading vsix, unpacking, generating package-lock.json, validating licenses