Generate remote binary #410

AndrienkoAleksandr · 2019-08-19T11:32:31Z

What does this PR do?

Add image with remote-plugin-endpoint binary

What issues does this PR fix or reference?

eclipse-che/che#13387

dmytro-ndp · 2019-09-04T09:09:38Z

crw-ci-test

benoitf · 2019-09-04T09:47:30Z

dockerfiles/theia-endpoint-runtime/Dockerfile

-        fi \
-    fi
+# Light image without node. We include remote binary to this image.
+FROM alpine


hello, please add the version for the image (to be sure to not get unwanted) updates

benoitf · 2019-09-04T09:49:41Z

dockerfiles/theia-endpoint-runtime/Dockerfile

-#invalidate cashe
-ADD https://${GITHUB_TOKEN}:x-oauth-basic@api.github.com/repos/theia-ide/theia/git/refs/head /tmp/branch_info.json
-ADD https://${GITHUB_TOKEN}:x-oauth-basic@api.github.com/repos/eclipse/che-theia/git/refs/head /tmp/branch_info.json
+RUN mkdir /home/theia


Do we need this docker image if we do injection of the binary ?

or it's to have the binary for injection that we're making this image ?

This image will be init container for Che theia editor plugin.

benoitf · 2019-09-04T09:50:19Z

dockerfiles/theia-endpoint-runtime/etc/entrypoint.sh

-trap 'responsible_shutdown' SIGHUP SIGTERM SIGINT
-
-cd ${HOME}
+trap 'responsible_shutdown' HUP TERM INT


is there a reason to change the traps ?

Yes, of course, SIGHUP SIGTERM SIGINT doesn't work on fedora.

benoitf · 2019-09-04T09:51:00Z

extensions/eclipse-che-theia-plugin-ext/src/node/che-plugin-api-provider.ts

@@ -22,7 +21,7 @@ export class ChePluginApiProvider implements ExtPluginApiProvider {
                initFunction: 'initializeApi',
                initVariable: 'che_api_provider'
            },
-            backendInitPath: path.join(__dirname, '../plugin/node/che-api-node-provider.js')
+            backendInitPath: '@eclipse-che/theia-plugin-ext/lib/plugin/node/che-api-node-provider.js'


maybe this change and the next one could be merged in another PR

dmytro-ndp · 2019-09-04T09:54:49Z

crw-ci-test

che-bot · 2019-09-04T10:39:39Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has been successful:

build details
"che-theia" docker image: maxura/che-theia:410

Signed-off-by: Oleksandr Andriienko <oandriie@redhat.com>

che-bot · 2019-09-29T22:51:16Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:410
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

che-bot · 2019-09-29T23:28:26Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:410
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

…moteBinary

che-bot · 2019-10-03T10:47:01Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has been successful:

build details
"che-theia" docker image: maxura/che-theia:410

AndrienkoAleksandr · 2019-10-03T12:22:29Z

ci-build-check

AndrienkoAleksandr · 2019-10-03T12:36:22Z

ci-build

AndrienkoAleksandr · 2019-10-03T13:13:03Z

@benoitf @evidolob @mmorhun Could you review, please?

benoitf · 2019-10-03T13:19:00Z

dockerfiles/theia-endpoint-runtime-binary/Dockerfile

+    nexe node_modules/@eclipse-che/theia-remote/lib/node/plugin-remote.js -t alpine-x64-10.16.0 -o /plugin-remote-endpoint
+
+# Light image without node. We include remote binary to this image.
+FROM alpine:3.10.2


what is the difference of using FROM ALPINE:3.10.2 and using FROM scratch if we don't install anything and that the binary is self-contained ? it's due to musl linking I suppose ?

Sorry, @benoitf, Seems, scratch has only latest tag. And this image has not cp and has not shell(even without sh). Dockerfile RUN section doesn't work without sh, so I can not install cp. And I think scratch has not any package manager, because it's empty image, so I don't know how to install cp or so on.

ok I see.

I might see another problem.
We generate nexe for alpine but if the runtime is not alpine, what is the behavior ?
like using debian, centos, fedora, etc images ?

as we'll copy the bootstrap to another container ?

@benoitf I tested with centos, fedora debian and it is works. List tested images was pretty big. You can see this list 99ae354#diff-f7eaf7e082319e42b2dfd26a86ca0736R80 search by from, I commented code, but each image was tested.

ok, still strange but if it works.

benoitf · 2019-10-03T13:19:14Z

dockerfiles/theia-endpoint-runtime-binary/build.sh

+. "${base_dir}"/../build.include
+
+init --name:theia-endpoint-runtime-binary "$@"
+build


very minor: missing EOF

benoitf · 2019-10-03T13:20:48Z

dockerfiles/theia-endpoint-runtime-binary/Dockerfile

+
+USER root
+
+RUN cd /home/theia && \


it seems usually linters want that we use WORKDIR /home/theia before but I haven't run linter there so maybe it's fine

benoitf · 2019-10-03T13:22:04Z

dockerfiles/theia-endpoint-runtime-binary/Dockerfile

+
+FROM ${BUILD_ORGANIZATION}/${BUILD_PREFIX}-theia:${BUILD_TAG} as builder
+
+USER root


do we need to be root ?
AFAIK we can install stuff globally in theia images ?

Signed-off-by: Oleksandr Andriienko <oandriie@redhat.com>

che-bot · 2019-10-03T20:57:05Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:410
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

benoitf · 2019-10-04T11:59:27Z

hello, how to test ?

AndrienkoAleksandr · 2019-10-04T14:25:16Z

@benoitf hello, how to test ?

All stuff I tested on the minishift.

Build che-theia and theia-endpoint-runtime-binary images

You need to build che-theia, and theia-endpoint-runtime-binary images, so from my branch:

./build.sh --pr

You can retag and push images to the dockerhub.

Use image with upgraded che-plugin-broker

For integration image theia-endpoint-runtime-binary like init-container for che-theia, you need to use upgraded che-plugin-broker: eclipse-che/che-plugin-broker#75 . You can use mine image: aandrienko/che-unified-plugin-broker:latest, or build own using script(but patched to use your dockerhub account): https://github.com/eclipse/che-plugin-broker/tree/startRemoteBinaryWithOverrideEntryPoint/.ci/build.sh

To activate this plugin-broker image in the che, you need to open deployment che-server and apply env variable:

CHE_WORKSPACE_PLUGIN__BROKER_UNIFIED_IMAGE aandrienko/che-unified-plugin-broker:latest

Redeploy che.

Test devfile

And you can use some test devfile with your che-theia and theia-endpoint-runtime-binary images. Example of mine devfile:

https://gist.github.com/AndrienkoAleksandr/ec1efffcbb4b25d52c22029bb7b6ac69

Interesting part of the test devfile.

We are using our builded image with che-theia:

https://github.com/AndrienkoAleksandr/che-plugin-registry/blob/master/v3/plugins/eclipse/che-theia/remote2/meta.yaml#L63

Init container section here with theia-endpoint-runtime-binary:

https://github.com/AndrienkoAleksandr/che-plugin-registry/blob/master/v3/plugins/eclipse/che-theia/remote2/meta.yaml#L51

Sidecar image for typescript lsp

And we need image for sidecar container, here is I used fedora:30:

DockerFile
Fedora with node image build Script(should be used for typescript lsp)
meta.yaml

In the devfile sidecar plugin here:
https://gist.github.com/AndrienkoAleksandr/ec1efffcbb4b25d52c22029bb7b6ac69#file-gistfile1-txt-L16

AndrienkoAleksandr · 2019-10-07T12:03:19Z

@l0rd @benoitf Is it OK to merge it? It's planned for 7.3 and rework sidecar images depends on it.

benoitf · 2019-10-07T12:05:24Z

yes
it's not harmful to merge it BTW

AndrienkoAleksandr · 2019-10-07T12:10:04Z

ci-build

AndrienkoAleksandr · 2019-10-07T13:07:34Z

crw-ci-test

musienko-maxim · 2019-10-07T13:18:28Z

crw-ci-test

che-bot · 2019-10-07T13:35:23Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
test report
logs and configs
"che-theia" docker image: maxura/che-theia:410
"che-theia-endpoint-runtime" docker image: maxura/che-theia-endpoint-runtime:410
"che-remote-plugin-runner-java8" docker image: maxura/che-remote-plugin-runner-java8:410
"che-remote-plugin-kubernetes-tooling-1.0.0" docker image: maxura/che-remote-plugin-kubernetes-tooling-1.0.0:410
Happy path tests DevFile
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

l0rd

LGTM Tested on che.osio and worked fine. I kind of agree on @benoitf comments but these are things that can be addressed in the later.

che-bot · 2019-10-07T15:13:23Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
test report
logs and configs
"che-theia" docker image: maxura/che-theia:410
"che-theia-endpoint-runtime" docker image: maxura/che-theia-endpoint-runtime:410
"che-remote-plugin-runner-java8" docker image: maxura/che-remote-plugin-runner-java8:410
"che-remote-plugin-kubernetes-tooling-1.0.0" docker image: maxura/che-remote-plugin-kubernetes-tooling-1.0.0:410
Happy path tests DevFile
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

che-bot · 2019-10-07T18:08:21Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
test report
logs and configs
"che-theia" docker image: maxura/che-theia:410
"che-theia-endpoint-runtime" docker image: maxura/che-theia-endpoint-runtime:410
"che-remote-plugin-runner-java8" docker image: maxura/che-remote-plugin-runner-java8:410
"che-remote-plugin-kubernetes-tooling-1.0.0" docker image: maxura/che-remote-plugin-kubernetes-tooling-1.0.0:410
Happy path tests DevFile
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

musienko-maxim · 2019-10-08T06:17:53Z

The PR was checked manually. All steps in the HappyPath test were passed.

AndrienkoAleksandr mentioned this pull request Aug 19, 2019

Remote binary #267

Closed

AndrienkoAleksandr force-pushed the GenerateRemoteBinary branch from beab63e to 6b2e379 Compare August 23, 2019 16:35

AndrienkoAleksandr marked this pull request as ready for review August 26, 2019 13:48

AndrienkoAleksandr requested review from benoitf, evidolob, l0rd, mmorhun, rhopp and vparfonov as code owners August 26, 2019 13:48

This was referenced Aug 26, 2019

Use plugin patcher to inject remote plugin runtime. eclipse-che/che-plugin-registry#210

Closed

Theia remote plug-in runtime in sidecar container improvement eclipse-che/che#13387

Closed

eclipse-che deleted a comment from che-bot Sep 4, 2019

benoitf reviewed Sep 4, 2019

View reviewed changes

mmorhun approved these changes Sep 16, 2019

View reviewed changes

Add remote binary image.

4ba4ebf

Signed-off-by: Oleksandr Andriienko <oandriie@redhat.com>

AndrienkoAleksandr force-pushed the GenerateRemoteBinary branch from 6b2e379 to 4ba4ebf Compare September 29, 2019 22:16

Fix up.

4e0e945

Signed-off-by: Oleksandr Andriienko <oandriie@redhat.com>

AndrienkoAleksandr mentioned this pull request Sep 30, 2019

Add init container to inject remote binary. eclipse-che/che-plugin-registry#233

Merged

Merge branch 'master' of github.com:eclipse/che-theia into GenerateRe…

fa0951f

…moteBinary

AndrienkoAleksandr requested a review from mmorhun October 3, 2019 13:13

benoitf reviewed Oct 3, 2019

View reviewed changes

Handle some review feedback.

be0c576

Signed-off-by: Oleksandr Andriienko <oandriie@redhat.com>

evidolob approved these changes Oct 7, 2019

View reviewed changes

mmorhun approved these changes Oct 7, 2019

View reviewed changes

benoitf approved these changes Oct 7, 2019

View reviewed changes

l0rd approved these changes Oct 7, 2019

View reviewed changes

musienko-maxim self-requested a review October 8, 2019 06:18

musienko-maxim approved these changes Oct 8, 2019

View reviewed changes

AndrienkoAleksandr merged commit 40a345b into master Oct 8, 2019

AndrienkoAleksandr deleted the GenerateRemoteBinary branch October 8, 2019 06:18


		FROM ${BUILD_ORGANIZATION}/${BUILD_PREFIX}-theia:${BUILD_TAG} as builder

		USER root

Generate remote binary #410

Generate remote binary #410

Conversation

AndrienkoAleksandr commented Aug 19, 2019

What does this PR do?

What issues does this PR fix or reference?

dmytro-ndp commented Sep 4, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmytro-ndp commented Sep 4, 2019

che-bot commented Sep 4, 2019

che-bot commented Sep 29, 2019

che-bot commented Sep 29, 2019

che-bot commented Oct 3, 2019

AndrienkoAleksandr commented Oct 3, 2019

AndrienkoAleksandr commented Oct 3, 2019

AndrienkoAleksandr commented Oct 3, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

che-bot commented Oct 3, 2019

benoitf commented Oct 4, 2019

AndrienkoAleksandr commented Oct 4, 2019 • edited Loading

Build che-theia and theia-endpoint-runtime-binary images

Use image with upgraded che-plugin-broker

Test devfile

Interesting part of the test devfile.

Sidecar image for typescript lsp

AndrienkoAleksandr commented Oct 7, 2019

benoitf commented Oct 7, 2019

AndrienkoAleksandr commented Oct 7, 2019

AndrienkoAleksandr commented Oct 7, 2019

musienko-maxim commented Oct 7, 2019

che-bot commented Oct 7, 2019

l0rd left a comment

Choose a reason for hiding this comment

che-bot commented Oct 7, 2019

che-bot commented Oct 7, 2019

musienko-maxim commented Oct 8, 2019

AndrienkoAleksandr commented Oct 4, 2019 •

edited

Loading