feat: Embed OpenVsx in plug-in registry with few vsix files #1324

benoitf · 2022-07-01T13:45:22Z

What does this PR do?

Include OpenVSX in the plug-in registry

It will include some vsix coming from the openvsx-sync.json file (like 5 extensions for now)

remove alpine based image and use only ubi8 base image

Screenshot/screencast of this PR

What issues does this PR fix or reference?

eclipse-che/che#20549

How to test this PR?

Build the image (or use my image) and then starts Che using that image:

example: chectl server:deploy --platform=... --che-operator-cr-patch-yaml=...patch-pluginregistry.yaml with cr patch being

spec:
  components:
    pluginRegistry:
      deployment:
        containers:
          - image: 'quay.io/fbenoit/che-plugin-registry:openvsx-20220729'
            resources:
              limits:
                memory: '1536Mi'

you should be able to use it by using CheTheia (which consume the plugin-registry API)

Then, for example by using VS Codium or another instance, edit the product.json of the app to point to the instance of this openvsx instance

For example on macOS, edit ./Contents/Resources/app/product.json from /Applications/VSCodium.app folder
by updating content to

  "extensionsGallery": {
    "serviceUrl": "https://192.168.205.14.nip.io/plugin-registry/openvsx/vscode/gallery",
    "itemUrl": "https://192.168.205.14.nip.io/plugin-registry/openvsx/vscode/item"
  },

(using your correct nip.io/location)

Then, reset file permissions xattr -r -d com.apple.quarantine /Applications/VSCodium.app

Make sure the Che CaCert certificate is trusted on your mac, (import the file generated by chectl cacert:export command)

and then if you go in extensions page on VSCodium you should see the few list

installation should also work (for example redhat.vscode-yaml extension)

PR Checklist

As the author of this Pull Request I made sure that:

Reviewers

Reviewers, please comment how you tested the PR when approving it.

Change-Id: I6b42aa61a6d6b8077441fab270d6950954e29698
Signed-off-by: Florent Benoit fbenoit@redhat.com

openshift-ci · 2022-07-01T13:45:27Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

github-actions · 2022-07-01T13:45:35Z

Click here to review and test in web IDE:

openshift-ci · 2022-07-04T08:46:39Z

@benoitf: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

github-actions · 2022-07-26T08:39:35Z

Click here to review and test in web IDE:

github-actions · 2022-07-26T08:47:54Z

Click here to review and test in web IDE:

github-actions · 2022-07-26T10:19:44Z

Click here to review and test in web IDE:

github-actions · 2022-07-27T12:21:29Z

Click here to review and test in web IDE:

svor

Che-Theia works as before, it's possible to install/uninstall plugins from the registry:

svor · 2022-07-28T12:10:12Z

@benoitf it seems openshift-ci built is failed because it uses rhel.entrypoint.sh that was removed

benoitf · 2022-07-28T12:36:50Z

@svor ok 👍

benoitf · 2022-07-28T12:53:30Z

@svor it looks like tests have their own Dockerfiles so it looks odd to test stuff using their own copy/fork of the project's Dockerfiles

svor · 2022-07-28T13:20:00Z

@benoitf I agree openshift.ci tests look odd, they should be reworked for sure. But maybe in another PR, without this check we can't merge current PR.

github-actions · 2022-07-28T17:00:45Z

Click here to review and test in web IDE:

github-actions · 2022-07-28T17:11:05Z

Click here to review and test in web IDE:

openshift-ci · 2022-07-29T14:09:02Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: benoitf, svor

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

l0rd · 2022-08-03T17:02:55Z

@benoitf I was not able to test it.

I see that open-vsx starts successfully in the plugin registry container but then neither Theia/VSCode are consuming it.

Is there a particular Che Theia image that I should use? And if I want to try it with Che code should this work?

patch the CheCluster has you mentioned above
start a workspace sample with VS Code (i.e. golang sample)
patch the che-code DevWorkspaceTemplate to use the image quay.io/che-incubator-pull-requests/che-code:pr-85-amd64
restart the devworkspace

benoitf · 2022-08-03T17:20:45Z

@l0rd

CheTheia is not consuming at all OpenVSX (it's still consuming the plugin registry as before)
but you can still check that it's working fine there

So basically, there is no change if you're still using CheTheia

but you can look at the 'how to test' section of this PR and try the registry with some Desktop tool (like VSCodium, etc)

and for VSCode, yes if you edit the DevWorkspaceTemplate to use a new image, when workspace will start it'll use the new registry.
It's quite easy to check by looking at the extensions panel as it should display the url of the plug-in registry and you'll have very few extensions if you search for some

benoitf · 2022-08-03T17:21:42Z

you can also look at https://<your-che-cluster>/plugin-registry/openvsx/api/redhat URL

l0rd · 2022-08-05T05:58:30Z

I tried to test yesterday but I had issues with the plugin registry first:

After patching the CheCluster (I did that after the installation) the new plugin registry Pod failed to start

$ kubectl describe pod plugin-registry-85d75dbf64-w6vq9
Name:         plugin-registry-85d75dbf64-w6vq9
Namespace:    eclipse-che
Priority:     0
Node:         ip-10-0-138-61.ec2.internal/10.0.138.61
Start Time:   Thu, 04 Aug 2022 18:09:46 +0300
Labels:       app=che
              app.kubernetes.io/component=plugin-registry
              app.kubernetes.io/instance=che
              app.kubernetes.io/managed-by=che-operator
              app.kubernetes.io/name=che
              app.kubernetes.io/part-of=che.eclipse.org
              component=plugin-registry
              pod-template-hash=85d75dbf64
Annotations:  k8s.v1.cni.cncf.io/network-status:
                [{
                    "name": "openshift-sdn",
                    "interface": "eth0",
                    "ips": [
                        "10.128.2.56"
                    ],
                    "default": true,
                    "dns": {}
                }]
              k8s.v1.cni.cncf.io/networks-status:
                [{
                    "name": "openshift-sdn",
                    "interface": "eth0",
                    "ips": [
                        "10.128.2.56"
                    ],
                    "default": true,
                    "dns": {}
                }]
              openshift.io/scc: restricted
Status:       Running
IP:           10.128.2.56
IPs:
  IP:           10.128.2.56
Controlled By:  ReplicaSet/plugin-registry-85d75dbf64
Containers:
  plugin-registry:
    Container ID:   cri-o://a7dba0608d551f094e454fc7e769ba33b88a9f709412ec795516f6de33fb1c7e
    Image:          quay.io/fbenoit/che-plugin-registry:openvsx-20220729
    Image ID:       quay.io/fbenoit/che-plugin-registry@sha256:532071687d9d920e7326aa1a6d42cee249cf601707ef0dc4a260e539d8100720
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    124
      Started:      Thu, 04 Aug 2022 18:10:19 +0300
      Finished:     Thu, 04 Aug 2022 18:10:25 +0300
    Ready:          False
    Restart Count:  2
    Limits:
      cpu:     700m
      memory:  1536Mi
    Requests:
      cpu:      100m
      memory:   32Mi
    Liveness:   http-get http://:8080/plugins/ delay=30s timeout=3s period=10s #success=1 #failure=10
    Readiness:  http-get http://:8080/plugins/ delay=3s timeout=3s period=10s #success=1 #failure=10
    Environment Variables from:
      plugin-registry  ConfigMap  Optional: true
    Environment:
      CM_REVISION:  45920
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8pmbl (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kube-api-access-8pmbl:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: org.eclipse.che/v2
kind: CheCluster
metadata:
  creationTimestamp: "2022-08-04T08:19:31Z"
  finalizers:
  - checluster.che.eclipse.org
  - cheGateway.clusterpermissions.finalizers.che.eclipse.org
  - cheWorkspaces.clusterpermissions.finalizers.che.eclipse.org
  - namespaces-editor.permissions.finalizers.che.eclipse.org
  - devWorkspace.permissions.finalizers.che.eclipse.org
  - oauthclients.finalizers.che.eclipse.org
  - dashboard.clusterpermissions.finalizers.che.eclipse.org
  - consolelink.finalizers.che.eclipse.org
  generation: 4
  name: eclipse-che
  namespace: eclipse-che
  resourceVersion: "184882"
  uid: 981d7d67-8552-4e38-95f7-30914f9bcb3b
spec:
  components:
    cheServer:
      debug: false
      logLevel: INFO
    dashboard: {}
    database:
      credentialsSecretName: postgres-credentials
      externalDb: false
      postgresDb: dbche
      postgresHostName: postgres
      postgresPort: "5432"
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
    ConfigMapName:           openshift-service-ca.crt
    ConfigMapOptional:       <nil>
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason          Age                From               Message
  ----     ------          ----               ----               -------
  Normal   Scheduled       54s                default-scheduler  Successfully assigned eclipse-che/plugin-registry-85d75dbf64-w6vq9 to ip-10-0-138-61.ec2.internal
  Normal   AddedInterface  53s                multus             Add eth0 [10.128.2.56/23] from openshift-sdn
  Normal   Pulled          53s                kubelet            Successfully pulled image "quay.io/fbenoit/che-plugin-registry:openvsx-20220729" in 130.576845ms
  Normal   Pulled          46s                kubelet            Successfully pulled image "quay.io/fbenoit/che-plugin-registry:openvsx-20220729" in 194.583121ms
  Normal   Pulling         22s (x3 over 53s)  kubelet            Pulling image "quay.io/fbenoit/che-plugin-registry:openvsx-20220729"
  Normal   Created         22s (x3 over 53s)  kubelet            Created container plugin-registry
  Normal   Started         22s (x3 over 53s)  kubelet            Started container plugin-registry
  Normal   Pulled          22s                kubelet            Successfully pulled image "quay.io/fbenoit/che-plugin-registry:openvsx-20220729" in 148.184037ms
  Warning  BackOff         1s (x5 over 41s)   kubelet            Back-off restarting failed container

I changed the memory limit to 1800Mi and the plugin registry started successfully. But then the dashboard started to be unresponsive:

In the logs I got "Unexpected error null"

Starting Dashboard backend server...
Static server's serving "/public" on 0.0.0.0:8080/
Che Dashboard swagger is running on "dashboard/api/swagger".
├── * (OPTIONS)
└── / (-)
    ├── * (HEAD, GET)
    ├── dashboard (-)
    │   ├── /api (-)
    │   │   ├── /cluster-config (GET)
    │   │   ├── /cluster-info (GET)
    │   │   ├── /namespace/:namespace (-)
    │   │   │   ├── /devworkspaceId/:devworkspaceId/kubeconfig (POST)
    │   │   │   ├── /devworkspaces (GET, POST)
    │   │   │   │   └── /:workspaceName (GET, PATCH, DELETE)
    │   │   │   ├── /devworkspacetemplates (GET, POST)
    │   │   │   │   └── /:templateName (PATCH)
    │   │   │   └── /dockerconfig (GET, PUT)
    │   │   ├── /server-config (GET)
    │   │   ├── /swagger (GET)
    │   │   │   ├── / (GET)
    │   │   │   ├── /* (GET)
    │   │   │   ├── /initOAuth (GET)
    │   │   │   ├── /json (GET)
    │   │   │   ├── /static/* (HEAD, GET)
    │   │   │   ├── /uiConfig (GET)
    │   │   │   └── /yaml (GET)
    │   │   └── /websocket (GET)
    │   └── /f (GET)
    └── f (GET)

(node:6) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Server listening at http://0.0.0.0:8080
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null
Unexpected error null

I have tried to reinstall Che but the dashboard was still not working. I am retrying today.

l0rd · 2022-08-05T06:08:16Z

And as discussed we should introduce a check of the variable START_OPENVSX: if it's not set or if it's set to false we should not start the embedded open-vsx.

I have created a separate issue for the opererator to set this variable to true if the pluginRegistry.openVSXURL hasn't been specified in the CheCluster: eclipse-che/che#21611. We don't need to wait for the operator issue to be resolved to merge this PR though.

openshift-ci · 2022-08-05T06:53:01Z

New changes are detected. LGTM label has been removed.

github-actions · 2022-08-05T06:53:10Z

Click here to review and test in web IDE:

benoitf · 2022-08-05T06:53:55Z

Added a fixup for START_OPENVSX

github-actions · 2022-08-05T06:55:43Z

Click here to review and test in web IDE:

l0rd · 2022-08-05T16:29:42Z

I was able to move a little bit forward.

Although I couldn't test with che-code (but that's probably related to this PR, thanks @benoitf for fixing it right away) I could list the extensions at /openvsx/api/redhat:

Anyway I still faced the memory problem:

Based on that I would recommend a more robust memory setup done by che-operator:

if pluginRegistry.openVSXURL is NOT set we should set plugin registry memory:
- limit to 2Gi
- request to 500Mi
otherwise set the old (and less demanding) memory settings

Note to self: to test with che-code of che-code PR #85 use

https://<my-che-url>/#https://github.com/che-samples/python-hello-world/tree/devfilev2&storageType=ephemeral&che-editor=https://gist.githubusercontent.com/l0rd/0cf6f155f49d0ab92b70986ceabacc58/raw/40be93578f4478b9bdd5a5a27e35da24ab7f9b8a/devfile.yaml

Change-Id: I6b42aa61a6d6b8077441fab270d6950954e29698 Signed-off-by: Florent Benoit <fbenoit@redhat.com>

Change-Id: I5ce218802e2f17453b3cc4f1f0ccdde12914eb8b Signed-off-by: Florent Benoit <fbenoit@redhat.com>

github-actions · 2022-08-16T07:33:25Z

Click here to review and test in web IDE:

benoitf · 2022-08-18T12:47:04Z

merging as now che-operator should default to open-vsx.org and then it means the local OpenVSX instance is not started

openshift-ci bot added the do-not-merge/work-in-progress label Jul 1, 2022

openshift-ci bot added the needs-rebase label Jul 4, 2022

benoitf force-pushed the CHE-20549 branch from d408a75 to e8c531a Compare July 26, 2022 08:39

openshift-merge-robot removed the needs-rebase label Jul 26, 2022

benoitf changed the title ~~[wip] feat: Allow to use OpenVsx in plug-in registry~~ feat: Allow to embed OpenVsx in plug-in registry Jul 26, 2022

benoitf marked this pull request as ready for review July 26, 2022 13:11

benoitf requested review from vinokurig, vitaliy-guliy and svor as code owners July 26, 2022 13:11

openshift-ci bot removed the do-not-merge/work-in-progress label Jul 26, 2022

benoitf force-pushed the CHE-20549 branch from 4740134 to 68c6f75 Compare July 27, 2022 12:21

svor approved these changes Jul 28, 2022

View reviewed changes

openshift-ci bot assigned svor Jul 28, 2022

openshift-ci bot added the lgtm label Jul 28, 2022

benoitf marked this pull request as draft July 28, 2022 14:07

openshift-ci bot added do-not-merge/work-in-progress and removed lgtm labels Jul 28, 2022

openshift-ci bot added the lgtm label Jul 29, 2022

This was referenced Aug 1, 2022

Use either OpenVSX.org or local OpenVSX instance when starting VS Code eclipse-che/che#21601

Closed

Airgap support for VS Code as an Eclipse Che editor eclipse-che/che#20549

Closed

l0rd mentioned this pull request Aug 4, 2022

Che operator should propagate an env varialble START_OPENVSX to the plugin registry eclipse-che/che#21611

Closed

openshift-ci bot removed the lgtm label Aug 5, 2022

benoitf force-pushed the CHE-20549 branch from 0956848 to b03f8b6 Compare August 5, 2022 06:55

svor mentioned this pull request Aug 8, 2022

chore: Update from ubi8/httpd-24:1-199 to ubi8/httpd-24:1-209 #1359

Closed

benoitf mentioned this pull request Aug 8, 2022

adjust memory of plug-in registry container based on CheCluster settings eclipse-che/che#21620

Closed

svor mentioned this pull request Aug 8, 2022

Extend the list of embedded plugins for internal Open VSX registry eclipse-che/che#21621

Closed

benoitf added 2 commits August 16, 2022 09:32

feat: Allow to use OpenVsx in plug-in registry

c7aca23

Change-Id: I6b42aa61a6d6b8077441fab270d6950954e29698 Signed-off-by: Florent Benoit <fbenoit@redhat.com>

fixup! feat: Allow to use OpenVsx in plug-in registry

a7ed16e

Change-Id: I5ce218802e2f17453b3cc4f1f0ccdde12914eb8b Signed-off-by: Florent Benoit <fbenoit@redhat.com>

benoitf force-pushed the CHE-20549 branch from b03f8b6 to a7ed16e Compare August 16, 2022 07:33

l0rd mentioned this pull request Aug 16, 2022

The default VSX registry should be openvsx.org eclipse-che/che#21637

Closed

benoitf merged commit 773ae3f into eclipse-che:main Aug 18, 2022

che-bot added this to the 7.53 milestone Aug 18, 2022

This was referenced Aug 18, 2022

update Che CRD for DWO per-user strategy storage configuration eclipse-che/che-operator#1442

Merged

Che plugin registry pod is failed during the che deployment on OS eclipse-che/che#21641

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Embed OpenVsx in plug-in registry with few vsix files #1324

feat: Embed OpenVsx in plug-in registry with few vsix files #1324

benoitf commented Jul 1, 2022 •

edited

Loading

openshift-ci bot commented Jul 1, 2022

github-actions bot commented Jul 1, 2022

openshift-ci bot commented Jul 4, 2022

github-actions bot commented Jul 26, 2022

github-actions bot commented Jul 26, 2022

github-actions bot commented Jul 26, 2022

github-actions bot commented Jul 27, 2022

svor left a comment

svor commented Jul 28, 2022

benoitf commented Jul 28, 2022

benoitf commented Jul 28, 2022

svor commented Jul 28, 2022

github-actions bot commented Jul 28, 2022

github-actions bot commented Jul 28, 2022

openshift-ci bot commented Jul 29, 2022

l0rd commented Aug 3, 2022

benoitf commented Aug 3, 2022

benoitf commented Aug 3, 2022 •

edited

Loading

l0rd commented Aug 5, 2022 •

edited

Loading

l0rd commented Aug 5, 2022 •

edited

Loading

openshift-ci bot commented Aug 5, 2022

github-actions bot commented Aug 5, 2022

benoitf commented Aug 5, 2022

github-actions bot commented Aug 5, 2022

l0rd commented Aug 5, 2022 •

edited

Loading

github-actions bot commented Aug 16, 2022

benoitf commented Aug 18, 2022

feat: Embed OpenVsx in plug-in registry with few vsix files #1324

feat: Embed OpenVsx in plug-in registry with few vsix files #1324

Conversation

benoitf commented Jul 1, 2022 • edited Loading

What does this PR do?

Screenshot/screencast of this PR

What issues does this PR fix or reference?

How to test this PR?

PR Checklist

Reviewers

openshift-ci bot commented Jul 1, 2022

github-actions bot commented Jul 1, 2022

openshift-ci bot commented Jul 4, 2022

github-actions bot commented Jul 26, 2022

github-actions bot commented Jul 26, 2022

github-actions bot commented Jul 26, 2022

github-actions bot commented Jul 27, 2022

svor left a comment

Choose a reason for hiding this comment

svor commented Jul 28, 2022

benoitf commented Jul 28, 2022

benoitf commented Jul 28, 2022

svor commented Jul 28, 2022

github-actions bot commented Jul 28, 2022

github-actions bot commented Jul 28, 2022

openshift-ci bot commented Jul 29, 2022

l0rd commented Aug 3, 2022

benoitf commented Aug 3, 2022

benoitf commented Aug 3, 2022 • edited Loading

l0rd commented Aug 5, 2022 • edited Loading

l0rd commented Aug 5, 2022 • edited Loading

openshift-ci bot commented Aug 5, 2022

github-actions bot commented Aug 5, 2022

benoitf commented Aug 5, 2022

github-actions bot commented Aug 5, 2022

l0rd commented Aug 5, 2022 • edited Loading

github-actions bot commented Aug 16, 2022

benoitf commented Aug 18, 2022

benoitf commented Jul 1, 2022 •

edited

Loading

benoitf commented Aug 3, 2022 •

edited

Loading

l0rd commented Aug 5, 2022 •

edited

Loading

l0rd commented Aug 5, 2022 •

edited

Loading

l0rd commented Aug 5, 2022 •

edited

Loading