Build Che Theia generator on image build if it was changed in any way #441

mmorhun · 2019-09-17T13:03:56Z

Signed-off-by: Mykola Morhun mmorhun@redhat.com

What does this PR do?

Rebuild Che Theia generator when cache assembly is out of date.

Before we used previously built artifact if it existed on disk and with such approach we're loosing new changes in the generator.
Now the generator is built when there is a change in the generator source code or the last commit which has changed the generator is newer than built assembly.

Should provide fix for eclipse-che/che#14555

What issues does this PR fix or reference?

eclipse-che/che#14555

benoitf

Can we compare date of tgz and generator directory ?

If directory is more recent then we regenerate the tgz ?

mmorhun · 2019-09-17T13:29:36Z

@benoitf I see no clear way of doing it. I've tried on my Fedora 30 to change a file inside subdirectory of tested directory and modification time of the top directory hasn't been changed. Also we do not know FS settings, like noatime etc. So it is safer to rebuild each time (and the script is easier to understand and maintain). Yes we'll lose ~20-30 seconds on build, but it worst them, I think.
But if you have some ides - will be glad to know.

mmorhun · 2019-09-17T13:52:39Z

crw-ci-test

che-bot · 2019-09-17T13:56:20Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has been successful:

build details
"che-theia" docker image: maxura/che-theia:441

dmytro-ndp · 2019-09-17T14:48:25Z

ci-build

che-bot · 2019-09-17T15:20:45Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has been successful:

build details
"che-theia" docker image: maxura/che-theia:441

benoitf · 2019-09-17T20:52:49Z

@mmorhun

I would check the last git modification for generator folder

git log -1 --pretty=%ct -- generator

and compare it to .tgz file (if tgz is older than the git log date it needs to be generated again)

and check if we have dirty files in generator folder as well and if it's the case, we generate again as well

$ git status generator --porcelain

(if not empty output, generate again)

mmorhun · 2019-09-18T07:49:04Z

@benoitf I've updated the PR, please review it again. Also would be great if you test the changes.

che-bot · 2019-09-18T08:35:30Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

mmorhun · 2019-09-18T09:35:07Z

crw-ci-test

che-bot · 2019-09-18T10:22:23Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has been successful:

build details
"che-theia" docker image: maxura/che-theia:441

benoitf · 2019-09-19T08:39:58Z

@mmorhun
FYI I have on macOS:

$ ./build.sh 
stat: illegal option -- -
usage: stat [-FlLnqrsx] [-f format] [-t timefmt] [file ...]
./build.sh: line 33: [: 1566559750: unary operator expected
Copying Che Theia generator assembly

you could use date command instead as it's working fine on BSD/macOS

on macOs:

$ date -r generator/eclipse-che-theia-generator.tgz +%s
1564557705

on Linux:

$ docker run --rm -it -v $(pwd):/foo fedora stat --format=%Y /foo/generator/eclipse-che-theia-generator.tgz
1564557705

$ docker run --rm -it -v $(pwd):/foo fedora  date -r /foo/generator/eclipse-che-theia-generator.tgz +%s
1564557705

mmorhun · 2019-09-19T13:23:42Z

@benoitf thanks for letting me know. I have no Mac device to test, so please test it again)

benoitf

I've tested without tgz file, with a local change or a re-run and all is working as expected 👍

thx @mmorhun

che-bot · 2019-09-19T14:12:31Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

mmorhun · 2019-09-19T14:17:32Z

crw-ci-test

che-bot · 2019-09-19T15:19:43Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

mmorhun · 2019-09-20T06:47:28Z

crw-ci-test

che-bot · 2019-09-20T07:58:13Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

che-bot · 2019-09-20T13:33:41Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

mmorhun · 2019-09-24T13:24:51Z

crw-ci-test

che-bot · 2019-09-24T14:14:00Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

benoitf · 2019-09-24T14:18:51Z

do we know what is the issue with the happy path ?

mmorhun · 2019-09-24T14:50:30Z

@benoitf the test failed with timeout on clone project, I think we can try to rerun it.

mmorhun · 2019-09-24T14:50:39Z

crw-ci-test

che-bot · 2019-09-24T15:55:31Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

mmorhun · 2019-09-25T11:22:29Z

crw-ci-test

che-bot · 2019-09-25T11:46:59Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

Signed-off-by: Mykola Morhun <mmorhun@redhat.com>

mmorhun · 2019-09-25T11:51:58Z

Rebased

che-bot · 2019-09-25T12:25:03Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

mmorhun · 2019-09-25T12:35:19Z

crw-ci-test

benoitf · 2019-09-25T12:59:50Z

@mmorhun I think you need to rebase again

che-bot · 2019-09-25T13:46:35Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

mmorhun · 2019-09-25T13:53:28Z

crw-ci-test

che-bot · 2019-09-25T14:23:44Z

E2E Happy path tests of Eclipse Che Single User on K8S (minikube v1.1.1) has failed:

build details
"che-theia" docker image: maxura/che-theia:441
test artifacts
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.
Use command "crw-ci-test" to rerun the test.

dmytro-ndp · 2019-09-25T14:41:56Z

@mmorhun: this PR check had failed because of new Happy path tests issue eclipse-che/che#14660

Signed-off-by: Eric Williams <ericwill@redhat.com>

mmorhun requested review from benoitf, tolusha and AndrienkoAleksandr September 17, 2019 13:03

mmorhun requested review from evidolob, l0rd, rhopp and vparfonov as code owners September 17, 2019 13:03

mmorhun self-assigned this Sep 17, 2019

benoitf reviewed Sep 17, 2019

View reviewed changes

tolusha approved these changes Sep 17, 2019

View reviewed changes

mmorhun force-pushed the fix-generator-cache-issue branch from 5367cab to ba1dc6a Compare September 18, 2019 07:42

mmorhun changed the title ~~Always build Che Theia generator on image build~~ Build Che Theia generator on image build if it was changed in any way Sep 18, 2019

AndrienkoAleksandr approved these changes Sep 18, 2019

View reviewed changes

mmorhun force-pushed the fix-generator-cache-issue branch from ba1dc6a to ba70de1 Compare September 19, 2019 13:22

benoitf approved these changes Sep 19, 2019

View reviewed changes

benoitf mentioned this pull request Sep 25, 2019

eclipse/che-theia:next still contains @theia/cpp extension eclipse-che/che#14555

Closed

eclipse-che deleted a comment from che-bot Sep 25, 2019

Build Che Theia generator on image build if it was changed

3079b0d

Signed-off-by: Mykola Morhun <mmorhun@redhat.com>

mmorhun force-pushed the fix-generator-cache-issue branch from ba70de1 to 3079b0d Compare September 25, 2019 11:51

mmorhun merged commit 2e5ef70 into master Sep 25, 2019

mmorhun deleted the fix-generator-cache-issue branch September 25, 2019 14:43

vinokurig pushed a commit that referenced this pull request Apr 6, 2021

[release] Release 7.11 and bump to 7.12 (#441)

e8fa212

Signed-off-by: Eric Williams <ericwill@redhat.com>

Build Che Theia generator on image build if it was changed in any way #441

Build Che Theia generator on image build if it was changed in any way #441

Conversation

mmorhun commented Sep 17, 2019 • edited Loading

What does this PR do?

What issues does this PR fix or reference?

benoitf left a comment

Choose a reason for hiding this comment

mmorhun commented Sep 17, 2019

mmorhun commented Sep 17, 2019

che-bot commented Sep 17, 2019

dmytro-ndp commented Sep 17, 2019

che-bot commented Sep 17, 2019

benoitf commented Sep 17, 2019

mmorhun commented Sep 18, 2019

che-bot commented Sep 18, 2019

mmorhun commented Sep 18, 2019

che-bot commented Sep 18, 2019

benoitf commented Sep 19, 2019

mmorhun commented Sep 19, 2019

benoitf left a comment

Choose a reason for hiding this comment

che-bot commented Sep 19, 2019

mmorhun commented Sep 19, 2019

che-bot commented Sep 19, 2019

mmorhun commented Sep 20, 2019

che-bot commented Sep 20, 2019

che-bot commented Sep 20, 2019

mmorhun commented Sep 24, 2019

che-bot commented Sep 24, 2019

benoitf commented Sep 24, 2019

mmorhun commented Sep 24, 2019

mmorhun commented Sep 24, 2019

che-bot commented Sep 24, 2019

mmorhun commented Sep 25, 2019

che-bot commented Sep 25, 2019

mmorhun commented Sep 25, 2019

che-bot commented Sep 25, 2019

mmorhun commented Sep 25, 2019

benoitf commented Sep 25, 2019

che-bot commented Sep 25, 2019

mmorhun commented Sep 25, 2019

che-bot commented Sep 25, 2019

dmytro-ndp commented Sep 25, 2019 • edited Loading

mmorhun commented Sep 17, 2019 •

edited

Loading

dmytro-ndp commented Sep 25, 2019 •

edited

Loading