Integration DevWorkspace generator tool; acceptance test suite for eclipse-che/che-devfile-registry repo

[nallikaea]

What does this PR do?

Task: Integration DevWorkspace generator tool into che test automation framework
As subtask of Create acceptance tests to go through all devfile samples and all editors using API

• added dependency @eclipse-che/che-devworkspace-generator
• new class DevWorkspaceConfigurationHelper added that helps to generate DevfileContext and get DewWorkspace configuration as a string from DevfileContext to use in kubectl apply commands;
• added DevfileAcceptanceTestAPI.spec.ts that cteate DevWorkspace with meta.yaml file from https://github.com/eclipse-che/che-devfile-registry
• added dynamic devfileRegistry object for DevSpaces
• fixed some naming,
• fixed types in RecomendedExtentions.spec.ts

Next steps:

1. Verify that the project has been cloned successfully
2. Execute build commands and verify that they complete successfully
3. Execute run/debug commands and verify the lifeness probe
4. The list of samples should be grabbed automatically from the devfile registry main branch

Screenshot/screencast of this PR

Log attached in comments

What issues does this PR fix or reference?

#22212
#21514

How to test this PR?

export USERSTORY=DevfileAcceptanceTestAPI.spec.ts && npm run driver-less-test
works for one link to meta.yaml - TS_API_TEST_LINK_TO_META_YAML parameter;

PR Checklist

As the author of this Pull Request I made sure that:

• The Eclipse Contributor Agreement is valid
• Code produced is complete
• Code builds without errors
• Tests are covering the bugfix or new feature
• The repository devfile is up to date and works
• Sections What issues does this PR fix or reference and How to test this PR completed
• Relevant user documentation updated
• Relevant contributing documentation updated
• CI/CD changes implemented, documented and communicated

Reviewers

Reviewers, please comment how you tested the PR when approving it.

[nallikaea]

/usr/bin/npm run driver-less-test

> @eclipse-che/che-e2e@7.66.0-SNAPSHOT driver-less-test
> export TS_USE_WEB_DRIVER_FOR_TEST=false && npm run test


> @eclipse-che/che-e2e@7.66.0-SNAPSHOT test
> ./configs/sh-scripts/initDefaultValues.sh npm run lint && npm run tsc && export USERSTORY=$USERSTORY && mocha --config dist/configs/mocharc.js

Initialized default values

TS_SELENIUM_VALUE_TLS_SUPPORT =       
TS_SELENIUM_VALUE_OPENSHIFT_OAUTH =   true
TS_OCP_LOGIN_PAGE_PROVIDER_TITLE =    htpasswd
E2E_OCP_CLUSTER_VERSION =             4.x
TS_SELENIUM_LOG_LEVEL =               TRACE
TS_SELENIUM_OCP_USERNAME =            admin
TS_SELENIUM_W3C_CHROME_OPTION =       true
NODE_TLS_REJECT_UNAUTHORIZED =        0
TS_SELENIUM_EDITOR =                  che-code

> @eclipse-che/che-e2e@7.66.0-SNAPSHOT tsc
> rm -rf ./dist && ./configs/sh-scripts/generateIndex.sh && tsc -p .

Generating index.ts file...

################## Launch Information ##################

      TS_SELENIUM_BASE_URL: https://devspaces.apps.ocp412-mdolhal.crw-qe.com
      TS_SELENIUM_HEADLESS: false
      TS_SELENIUM_OCP_USERNAME: admin
      TS_SELENIUM_EDITOR:   che-code

      TS_SELENIUM_HAPPY_PATH_WORKSPACE_NAME: petclinic-dev-environment
      TS_SELENIUM_DELAY_BETWEEN_SCREENSHOTS: 1000
      TS_SELENIUM_REPORT_FOLDER: ./report
      TS_SELENIUM_EXECUTION_SCREENCAST: false
      DELETE_SCREENCAST_IF_TEST_PASS: true
      TS_SELENIUM_REMOTE_DRIVER_URL: 
      DELETE_WORKSPACE_ON_FAILED_TEST: false
      TS_SELENIUM_LOG_LEVEL: TRACE
      TS_SELENIUM_LAUNCH_FULLSCREEN: true

      TS_COMMON_DASHBOARD_WAIT_TIMEOUT: 5000
      TS_SELENIUM_START_WORKSPACE_TIMEOUT: 360000
      TS_WAIT_LOADER_PRESENCE_TIMEOUT: 60000

      TS_SAMPLE_LIST: Node.js MongoDB,Node.js Express

      MOCHA_DRIRECTORY is not set
      USERSTORY is not set

      to output timeout variables, set TS_SELENIUM_PRINT_TIMEOUT_VARIABLES to true
 ######################################################## 


            ‣ DriverHelper.getDriver
  Acceptance devfile registry test
          ▼ GitUtil.getProjectGitLinkFromLinkToMetaYaml - https://raw.githubusercontent.com/eclipse-che/che-devfile-registry/ddce84004d83e858de77bde596ac3b4ebbbbdfec/devfiles/java-web-spring/meta.yaml
curl -k https://raw.githubusercontent.com/eclipse-che/che-devfile-registry/ddce84004d83e858de77bde596ac3b4ebbbbdfec/devfiles/java-web-spring/meta.yaml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   285  100   285    0     0   3427      0 --:--:-- --:--:-- --:--:--  3433
---
displayName: Java Spring Boot
description: Java stack with OpenJDK 11 and Spring Boot Petclinic demo application
tags: ["Community", "Java", "OpenJDK", "Maven", "Spring Boot"]
icon: /images/springboot.svg
links:
  v2: https://github.com/che-samples/java-spring-petclinic/tree/main
          ▼ GitUtil.getProjectGitLinkFromLinkToMetaYaml - https://github.com/che-samples/java-spring-petclinic/tree/main
          ▼ DevWorkspaceConfigurationHelper.generateDevfileContext
No plug-in registry url. Setting to https://eclipse-che.github.io/che-plugin-registry/main/v3
(node:147931) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use `node --trace-warnings ...` to show where the warning was created)
More than one dev container component has been potentially found, taking the first one of tools,mariadb
DevWorkspace che-code-spring-petclinic was generated.
          ▼ KubernetesCommandLineToolsExecutor.loginToOcp - oc: Login to the "OC" client.
          ▼ KubernetesCommandLineToolsExecutor.getServerUrl - oc: Get server api url.
oc whoami && oc whoami --show-server=true
admin
https://api.ocp412-mdolhal.crw-qe.com:6443
          ▼ KubernetesCommandLineToolsExecutor.getServerUrl - oc: Get server api url.
          ▼ KubernetesCommandLineToolsExecutor.loginToOcp - oc: User already logged
          ▼ DevWorkspaceConfigurationHelper.getDewWorkspaceConfigurationYamlAsString
          ▼ KubernetesCommandLineToolsExecutor.applyYamlConfigurationAsStringOutput - oc:
cat <<EOF | oc apply -n admin-devspaces -f - 
apiVersion: workspace.devfile.io/v1alpha2
kind: DevWorkspaceTemplate
metadata:
  name: che-code-spring-petclinic
spec:
  commands:
    - id: init-container-command
      apply:
        component: che-code-injector
    - id: init-che-code-command
      exec:
        component: che-code-runtime-description
        commandLine: nohup /checode/entrypoint-volume.sh > /checode/entrypoint-logs.txt
          2>&1 &
  events:
    preStart:
      - init-container-command
    postStart:
      - init-che-code-command
  components:
    - name: che-code-runtime-description
      container:
        image: quay.io/devfile/universal-developer-image@sha256:98264e81d11bcfd74d46fd2037f5d26ee37794bb57b033328f35d838642e297f
        volumeMounts:
          - name: checode
            path: /checode
        memoryLimit: 1024Mi
        memoryRequest: 256Mi
        cpuLimit: 500m
        cpuRequest: 30m
        endpoints:
          - name: che-code
            attributes:
              type: main
              cookiesAuthEnabled: true
              discoverable: false
              urlRewriteSupported: true
            targetPort: 3100
            exposure: public
            secure: false
            protocol: https
          - name: code-redirect-1
            attributes:
              discoverable: false
              urlRewriteSupported: false
            targetPort: 13131
            exposure: public
            protocol: http
          - name: code-redirect-2
            attributes:
              discoverable: false
              urlRewriteSupported: false
            targetPort: 13132
            exposure: public
            protocol: http
          - name: code-redirect-3
            attributes:
              discoverable: false
              urlRewriteSupported: false
            targetPort: 13133
            exposure: public
            protocol: http
      attributes:
        app.kubernetes.io/component: che-code-runtime
        app.kubernetes.io/part-of: che-code.eclipse.org
        controller.devfile.io/container-contribution: true
    - name: checode
      volume: {}
    - name: che-code-injector
      container:
        image: quay.io/che-incubator/che-code@sha256:235f2028e1869ac4630a61adbc5cd33e75584275f1789eb5cd5ad36ff6713a6a
        command:
          - /entrypoint-init-container.sh
        volumeMounts:
          - name: checode
            path: /checode
        memoryLimit: 256Mi
        memoryRequest: 32Mi
        cpuLimit: 500m
        cpuRequest: 30m
---
apiVersion: workspace.devfile.io/v1alpha2
kind: DevWorkspace
metadata:
  name: spring-petclinic
  annotations:
    che.eclipse.org/devfile: >
      schemaVersion: 2.1.0

      metadata:
        name: spring-petclinic
      components:
        - name: tools
          container:
            image: quay.io/devfile/universal-developer-image:ubi8-latest
            memoryLimit: 3Gi
            endpoints:
              - exposure: none
                name: debug
                protocol: tcp
                targetPort: 5005
              - exposure: public
                name: 8080-tcp
                protocol: http
                targetPort: 8080
            volumeMounts:
              - name: m2
                path: /home/user/.m2
        - name: m2
          volume:
            size: 1G
        - name: mariadb
          container:
            image: >-
              quay.io/eclipse/che--mariadb:10.7.1-5a8009369ee57c85b6f4a08406147bd9c505cde6b8250d16a27d2a5febfdead7
            memoryLimit: 300Mi
            endpoints:
              - exposure: none
                name: db
                protocol: tcp
                targetPort: 3306
            env:
              - name: MARIADB_USER
                value: petclinic
              - name: MARIADB_PASSWORD
                value: petclinic
              - name: MARIADB_DATABASE
                value: petclinic
              - name: MARIADB_ALLOW_EMPTY_ROOT_PASSWORD
                value: allow
              - name: PS1
                value: $(echo ${0}) $
            volumeMounts:
              - name: mysqldata
                path: /var/lib/mysql
        - name: mysqldata
          volume:
            size: 256Mi
      commands:
        - id: prepare-database
          exec:
            component: mariadb
            workingDir: ${PROJECT_SOURCE}
            commandLine: >
              mysql -u root < ${PROJECT_SOURCE}/src/main/resources/db/mysql/user.sql
              && \

              mysql -u root petclinic <
              ${PROJECT_SOURCE}/src/main/resources/db/mysql/schema.sql && \

              mysql -u root petclinic <
              ${PROJECT_SOURCE}/src/main/resources/db/mysql/data.sql && \

              echo "\e[32mDone.\e[0m Database petclinic was configured"
            group:
              kind: run
              isDefault: true
        - id: maven-build
          exec:
            component: tools
            workingDir: ${PROJECT_SOURCE}
            commandLine: mvn clean install
            group:
              kind: build
              isDefault: true
        - id: run-with-hsqldb
          exec:
            component: tools
            workingDir: ${PROJECT_SOURCE}
            commandLine: >
              mvn spring-boot:run -DskipTests \

              -Dspring-boot.run.jvmArguments='-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005,quiet=y'
            group:
              kind: run
              isDefault: true
        - id: run-with-mysql
          exec:
            component: tools
            workingDir: ${PROJECT_SOURCE}
            commandLine: >
              mvn spring-boot:run -DskipTests -Dspring-boot.run.profiles=mysql \

              -Dspring-boot.run.jvmArguments='-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005,quiet=y'
            group:
              kind: run
              isDefault: true
      projects:
        - name: java-spring-petclinic
          git:
            remotes:
              origin: https://github.com/che-samples/java-spring-petclinic.git
            checkoutFrom:
              revision: main
spec:
  started: true
  routingClass: che
  template:
    components:
      - name: tools
        container:
          image: quay.io/devfile/universal-developer-image:ubi8-latest
          memoryLimit: 3Gi
          endpoints:
            - exposure: none
              name: debug
              protocol: tcp
              targetPort: 5005
            - exposure: public
              name: 8080-tcp
              protocol: http
              targetPort: 8080
          volumeMounts:
            - name: m2
              path: /home/user/.m2
        attributes:
          controller.devfile.io/merge-contribution: true
      - name: m2
        volume:
          size: 1G
      - name: mariadb
        container:
          image: quay.io/eclipse/che--mariadb:10.7.1-5a8009369ee57c85b6f4a08406147bd9c505cde6b8250d16a27d2a5febfdead7
          memoryLimit: 300Mi
          endpoints:
            - exposure: none
              name: db
              protocol: tcp
              targetPort: 3306
          env:
            - name: MARIADB_USER
              value: petclinic
            - name: MARIADB_PASSWORD
              value: petclinic
            - name: MARIADB_DATABASE
              value: petclinic
            - name: MARIADB_ALLOW_EMPTY_ROOT_PASSWORD
              value: allow
            - name: PS1
              value: $(echo ${0}) $
          volumeMounts:
            - name: mysqldata
              path: /var/lib/mysql
      - name: mysqldata
        volume:
          size: 256Mi
    commands:
      - id: prepare-database
        exec:
          component: mariadb
          workingDir: ${PROJECT_SOURCE}
          commandLine: >
            mysql -u root <
            ${PROJECT_SOURCE}/src/main/resources/db/mysql/user.sql && \

            mysql -u root petclinic < ${PROJECT_SOURCE}/src/main/resources/db/mysql/schema.sql && \

            mysql -u root petclinic < ${PROJECT_SOURCE}/src/main/resources/db/mysql/data.sql && \

            echo "\e[32mDone.\e[0m Database petclinic was configured"
          group:
            kind: run
            isDefault: true
      - id: maven-build
        exec:
          component: tools
          workingDir: ${PROJECT_SOURCE}
          commandLine: mvn clean install
          group:
            kind: build
            isDefault: true
      - id: run-with-hsqldb
        exec:
          component: tools
          workingDir: ${PROJECT_SOURCE}
          commandLine: >
            mvn spring-boot:run -DskipTests \

            -Dspring-boot.run.jvmArguments='-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005,quiet=y'
          group:
            kind: run
            isDefault: true
      - id: run-with-mysql
        exec:
          component: tools
          workingDir: ${PROJECT_SOURCE}
          commandLine: >
            mvn spring-boot:run -DskipTests -Dspring-boot.run.profiles=mysql \

            -Dspring-boot.run.jvmArguments='-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005,quiet=y'
          group:
            kind: run
            isDefault: true
    projects:
      - name: java-spring-petclinic
        git:
          remotes:
            origin: https://github.com/che-samples/java-spring-petclinic.git
          checkoutFrom:
            revision: main
  contributions:
    - name: editor
      kubernetes:
        name: che-code-spring-petclinic

EOF
devworkspacetemplate.workspace.devfile.io/che-code-spring-petclinic created
devworkspace.workspace.devfile.io/spring-petclinic created
    ✔ Create DevWorkspace (2179ms)
          ▼ KubernetesCommandLineToolsExecutor.waitDevWorkspace - oc: Wait till workspace ready.
oc wait -n admin-devspaces --for=condition=Ready dw spring-petclinic --timeout=360s
devworkspace.workspace.devfile.io/spring-petclinic condition met
    ✔ Wait until DevWorkspace has status "ready" (19842ms)
          ▼ KubernetesCommandLineToolsExecutor.deleteDevWorkspace - oc: Delete 'spring-petclinic' workspace
oc patch dw spring-petclinic -n admin-devspaces -p '{ "metadata": { "finalizers": null }}' --type merge || true
devworkspace.workspace.devfile.io/spring-petclinic patched
oc delete dw spring-petclinic -n admin-devspaces || true
devworkspace.workspace.devfile.io "spring-petclinic" deleted
oc delete dwt che-code-spring-petclinic -n admin-devspaces || true
devworkspacetemplate.workspace.devfile.io "che-code-spring-petclinic" deleted


  2 passing (39s)

            ‣ DriverHelper.wait (5000 milliseconds)

Process finished with exit code 0

[dmytro-ndp]

@nallikaea: taking into account, that DevfileAcceptanceTestAPI.spec.ts will "Verify that the project has been cloned successfully", what do you think about renaming CloneGitRepoAPI.spec.ts test to EmptyWorkspaceAPI.spec.ts, to avoid confusion in future?

[nallikaea]

@nallikaea: taking into account, that DevfileAcceptanceTestAPI.spec.ts will "Verify that the project has been cloned successfully", what do you think about renaming CloneGitRepoAPI.spec.ts test to EmptyWorkspaceAPI.spec.ts, to avoid confusion in future?

Not cloned but imported during workspace creation the way we check it in our API tests, anyway I don`t have objections to rename it.
But it was planing that CloneGitRepoAPI.spec.ts will also include additional test cases (see comments):

• public repo and SSH key secret configured (with GIT + SSH URL)
• with private repo and PAT secret configured (with https URL)
  I can not say will be the difference to use empty or not empty workspace to test this git clone API.

[nallikaea]

@nallikaea: taking into account, that DevfileAcceptanceTestAPI.spec.ts will "Verify that the project has been cloned successfully", what do you think about renaming CloneGitRepoAPI.spec.ts test to EmptyWorkspaceAPI.spec.ts, to avoid confusion in future?

@dmytro-ndp I created new issue #22223 and will update PR #22224 after current will be merged. PR will contain this changes because this one already has a lot.

[dmytro-ndp]

Looks good to merge.