Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E tests suite for Che 7 #12728

Closed
8 of 25 tasks
rhopp opened this issue Feb 21, 2019 · 7 comments
Closed
8 of 25 tasks

E2E tests suite for Che 7 #12728

rhopp opened this issue Feb 21, 2019 · 7 comments
Assignees
Labels
area/qe kind/epic A long-lived, PM-driven feature request. Must include a checklist of items that must be completed.
Milestone

Comments

@rhopp
Copy link
Contributor

rhopp commented Feb 21, 2019

As per latest discussions we decided, that best approach for E2E UI testing of Eclipse Che 7 would be to implement "happy path" suite:

  • This suite will try to cover as much ground possible, but will not test any component too deeply.
  • It should be based on real E2E user scenarios (new user, user with existing codebase&apps, ...)
  • This suite should run on as many possible (supported) environments and product configs (pvc strategy, http/https, ...)
  • This suite shouldn't be running for too long, this will allow it to be used as quality gate to eclipse/che repository.

E2E Test Script

1 - Deploy Che on minikube using the operator with authentication enabled

Steps

  • Install minikube
  • Pre-pull docker images needed by the workspace:
    • eclipse/che-operator:latest
    • registry.access.redhat.com/rhscl/postgresql-96-rhel7
    • registry.access.redhat.com/redhat-sso-7/sso72-openshift
    • eclipse/che-server:nightly
    • busybox
    • centos:centos7
    • eclipse/che-init-plugin-broker:v0.17
    • eclipse/che-unified-plugin-broker:v0.17
    • mariolet/petclinic:d2831f9b
    • centos/mysql-57-centos7
    • eclipse/che-machine-exec
    • eclipse/che-theia:next
    • eclipse/che-remote-plugin-runner-java8:next
    • maven:3.6.0-jdk-11
  • Deploy Che: run chectl server:start --installer=operator
  • Create a new Che user and login as that user

Success criteria

  • Minikube status is running
  • Che, Keycloak and Postre are Running
  • It's possible to login as a new user and access the dashboard
  • Images are pulled

Blockers and workarounds

2 - Generate a devfile from a running application

Steps

  • 1 - Deploy Petclinic:
kubectl create namespace prod
kubectl apply -n prod -f https://gist.githubusercontent.com/l0rd/52eda1d1b57878a218cba17c0c93a477/raw/d3f962bdf373406eb57be20efcdea2e4da8b19f9/petclinic.yaml
  • 2 - Generate a devfile for the development of that application:
chectl devfile:generate \
        --namespace=prod \
        --selector="app.kubernetes.io/name=petclinic" \
        --language=java \
        --git-repo='https://github.com/spring-projects/spring-petclinic.git' \
        --plugin='{"type": "chePlugin", "id": "redhat/vscode-yaml/0.4.0"}' > petclinic-devfile.yaml

Success criteria

  • The CRUD application is responding
  • A valid devfile is generated

Blockers and workarounds

3 - Start a Workspace from a devfile

Steps

  • 1 - Start a workspace using the devfile generated at step 2.2 using command
chectl workspace:start --devfile=petclinic-devfile.yaml

Success criteria
- Workspace started successfully
- Workspace is started in less than X seconds
- Project is cloned
- JDT LS and Java debug adapter are initialized correctly
- YAML LS is initialized correctly
- Kubernetes tooling is initialized correctly
- GitHub PR plugin is initialized correctly
- Terminal work

Blockers and workarounds

chectl workspace:start --devfile=https://gist.githubusercontent.com/Ohrimenko1988/93f5426f4ebc1705c55feb8ff0396a49/raw/19f626e545372d8607514d88aa13ef0ab03b144c/yaml-ls-bug.yaml

4 - Use Java IDE features and the inner loop

Steps

  • 1 - Open a Java file, change some code and verify that error highlighting, code completion and code navigation work
  • 2 - Use the "Build" task and verify that the application is build successfully
  • 3 - Use the "Run" task and and verify that the application is updated successfully
  • 4 - Set a breakpoint and use the "Debug" task to verify that the application stops at the breakpoint

Success criteria

  • Java Code completion, error highlighting and code navigation should work successfully
  • The provided "Build" task executes successfully
  • The provided "Run" task executes successfully and a pop-up ask to open the application URL
  • Changes done at step 4.1 should be applied
  • Java Debugger should be able to attach successfully when executing the "Debug" task.
  • Java Debugger should be stop at a breakpoint and resume application execution when clicking on resume.

Blockers and workarounds

kubectl --namespace=che patch cm che --patch '{"data":{"CHE_WORKSPACE_JAVA__OPTIONS": "-Xmx1500m", "CHE_WORKSPACE_MAVEN__OPTIONS": "-Xmx1500m"}}'

5 - Outer loop

Steps

  • 1 - Create a local git branch, commit locally, push on a remote branch and open a GitHub PR using the GitHub plugin
  • 2 - Merge the Pull request and verify that the build of the Docker image is started
  • 3 - Inject the Kubernetes Context in the workspace
  • 4 - Once the build completes use Che Kubernetes Tooling to perform a rolling update of the application

Success criteria

  • A Pull Request can be created using the GitHub plugin
  • Once the PR is merged a new image should be built
  • The kubernetes context is inected successfully and it's possible to read/create k8s objects in production namespace
  • The rolling update done with the Kubernetes Tooling Plugin is run successfully
  • The application in production is updated successfully

Blockers

  • No blockers

6 - Workspace sharing

Steps

  • 1 - Create a factory from the current workspace
  • 2 - Logout and login as a second user (without stopping the first workspace)
  • 3 - Start the workspace as a second user using the factory link

Success criteria

  • It is possible to create a factory
  • Logout and login as another user are successful
  • The workspace is started successfully using the factory (see criterias of STEP 3)

Blockers

  • For now (CHE 7.0.0-RC-2.0-SNAPSHOT) it is not possible to create a factory from workspace that was created from devfile. Only workspaces created from configuration are supported..

7 - Persistence and workspace status restoring

Steps

  • 1 - Modify some code stop/start the workspace and verify that the modifications are still there
  • 2 - Stop one of the workspaces
  • 3 - Stop/Start Che with chectl and verify that the workspaces status are recovered correctly (1 running and 1 stopped)

Success criteria

Blockers

  • No blockers

8 - Uninstall

Steps

  • 1 - Stop and delete both workspaces
  • 2 - Stop and Uninstall Che using chectl and verify that all the resources have been free up

Success criteria

Blockers

  • No blockers

Subtasks:

@rhopp rhopp added kind/epic A long-lived, PM-driven feature request. Must include a checklist of items that must be completed. team/che-qe labels Feb 21, 2019
@l0rd
Copy link
Contributor

l0rd commented Mar 6, 2019

As mentioned by @gorkem we should make sure that the happy path scenario take into account Che workshop scenario as well

@dmytro-ndp
Copy link
Contributor

dmytro-ndp commented Apr 12, 2019

FYI: links to various samples for Java projects could be found in description of issue #12620

@l0rd
Copy link
Contributor

l0rd commented May 28, 2019

@Ohrimenko1988 @musienko-maxim @rhopp I have updated the description of the e2e steps in this issue and the blockers/workaround I am aware of. What I ask you is to:

  • Review the first 4 steps and verify that they correspond with the tests you are implementing. If not please comment so that we can discuss and update.
  • Check [x] the steps that you have already implemented
  • Add blockers and workarounds that are not already listed

@rhopp
Copy link
Contributor Author

rhopp commented May 30, 2019

@l0rd As we would like to adapt the (slightly modified) happy path scenario also for che saas offering (che.openshift.io), I've tried to create workspace from the devfile mentioned in this issue.. This failed, with not enough resources (I was kinda expecting that).

(combined from similar events): Error creating: pods "workspacelfm70rbloln3cf4v.workspace-c6bf9f5c6-kg29l" is forbidden: [maximum cpu usage per Pod is 6, but limit is 8711m., maximum memory usage per Pod is 3Gi, but limit is 4679041024.]

Should we use something different for happy path test on saas che, or should we raise this with rh-che team and think about solution there?

@l0rd
Copy link
Contributor

l0rd commented May 30, 2019

Good point @rhopp. Let me try to create a "lighter" devfile for that purpose.

@slemeur
Copy link
Contributor

slemeur commented Jul 16, 2019

@rhopp : Is the scope for Che 7 GA been covered? so we can check that from the end game plan and move forward?

@rhopp
Copy link
Contributor Author

rhopp commented Aug 6, 2020

Happy path was implemented and is being run as a PR check on multiple repositories.
I'm closing this one... For any future improvements, let's have separate issues.

@rhopp rhopp closed this as completed Aug 6, 2020
@dmytro-ndp dmytro-ndp removed the severity/blocker Causes system to crash and be non-recoverable or prevents Che developers from working on Che code. label Jun 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/qe kind/epic A long-lived, PM-driven feature request. Must include a checklist of items that must be completed.
Projects
None yet
Development

No branches or pull requests

5 participants