Add docker files and instructions for debugging Selenium functional tests #3747

ananzh · 2023-03-30T19:33:33Z

Description

This PR introduces new Docker files to enable debugging of Selenium functional tests for Docker users. It configures a VNC viewer for real-time browser interaction monitoring during test execution.

Additionally, a new section is added to the documentation detailing the process of running and debugging Selenium functional tests using Docker and a VNC viewer.

Issues Resolved

#3700

Check List

codecov-commenter · 2023-03-30T20:41:55Z

Codecov Report

Merging #3747 (e5db2f8) into main (6b42669) will decrease coverage by 0.07%.
The diff coverage is n/a.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main    #3747      +/-   ##
==========================================
- Coverage   66.43%   66.37%   -0.07%     
==========================================
  Files        3210     3210              
  Lines       61677    61677              
  Branches     9522     9522              
==========================================
- Hits        40977    40939      -38     
- Misses      18419    18450      +31     
- Partials     2281     2288       +7

Flag	Coverage Δ
Linux	`66.37% <ø> (-0.01%)`	⬇️
Windows	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

see 8 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

joshuarrrr · 2023-04-10T23:58:38Z

All checks pass

joshuarrrr · 2023-04-11T00:00:43Z

docs/docker-dev/docker-dev-setup-manual.md

@@ -57,3 +57,38 @@ yarn start:docker
    * Files are constantly watched, so when you make code changes, OpenSearch Dashboards will rebuild and restart automatically. Refresh the link in the browser and the new changes should be applied.

 14. `Git` is already configured in the `entrypoint.sh` file, and the remote is already tracking the fork repository. You can start contributing by creating your branch off the main, and commit your first PR!
+
+# Debug Functional Tests


I think it would be helpful to link to this from TESTING.md.

Will update both sides

abbyhu2000 · 2023-04-13T16:37:14Z

docs/docker-dev/Dockerfile.selenium

I think we should build this image and publish it to the Docker hub like how we did with dashboard dev set up, so there is one less file in the folder, and it will also save the building time.

Yeah good point. But I think maybe not necessary for Selenium test. First of all, it doesn't take a long while to build (2 mins) even at first time compared to the base image. Second, we might not need vnc viewer after node 18 and lxde can be changed to others. I think for functional test purpose, providing contributors a sample Dockerfile is more flexible, allowing them easily modify the Dockerfile to fit their specific requirements or preferences. Also it is lower maintenance. If any contributor found a better viewer or setting that can improve the performance, they could raise a PR to update Dockerfile. Contributors can also create their own customized images based on the Dockerfile, which can be helpful if they have specific needs or configurations.

abbyhu2000 · 2023-04-13T16:39:57Z

docs/docker-dev/docker-dev-setup-manual.md

+```bash
+docker-compose -f docker-compose.yml -f docker-compose.selenium.yml down
+docker-compose -f docker-compose.yml -f docker-compose.selenium.yml build
+docker-compose -f docker-compose.yml -f docker-compose.selenium.yml up -d


Should we combine the last two commands like:
docker-compose -f docker-compose.yml -f docker-compose.selenium.yml up -d --build

will update

abbyhu2000 · 2023-04-13T17:21:22Z

docs/docker-dev/docker-dev-setup-manual.md

+
+7. After entering the VNC viewer, click the menu at the bottom left. Then click `Internet` to ensure Google works. Also, click `System Tools` and select `LXTerminal` to verify you can type in the terminal. The default terminal path should be `/docker-workspace/OpenSearch-Dashboards`.
+
+8. Run OpenSearch and OpenSearch Dashboards. First, start OpenSearch by executing `yarn opensearch snapshot` in one terminal. In a second terminal, run `yarn start --no-base-path` to start OpenSearch Dashboards on port 5601. Before running any functional tests, make sure OpenSearch Dashboards is running on port 5601. This process may take 2 to 10 mins, as it requires all the bundles to be assembled. Please be patient. However, this only occurs the first time you run it; afterward, everything is cached, significantly reducing the startup time for OpenSearch Dashboards. The initial bundling speed is dependent on your hardware limitations and VNC settings. Feel free to make adjustments to improve the process as needed.


This step can just be run 'yarn start:docker'.

OpenSearch is already running in the opensearch container; and since now i updated the Docker image, the command yarn start:docker is available to use.

I tried but yarn start:docker does not pass --no-base-path which is required by ftr test.

abbyhu2000 · 2023-04-13T17:43:13Z

docs/docker-dev/docker-dev-setup-manual.md

+
+7. After entering the VNC viewer, click the menu at the bottom left. Then click `Internet` to ensure Google works. Also, click `System Tools` and select `LXTerminal` to verify you can type in the terminal. The default terminal path should be `/docker-workspace/OpenSearch-Dashboards`.
+
+8. Run OpenSearch and OpenSearch Dashboards. First, start OpenSearch by executing `yarn opensearch snapshot` in one terminal. In a second terminal, run `yarn start --no-base-path` to start OpenSearch Dashboards on port 5601. Before running any functional tests, make sure OpenSearch Dashboards is running on port 5601. This process may take 2 to 10 mins, as it requires all the bundles to be assembled. Please be patient. However, this only occurs the first time you run it; afterward, everything is cached, significantly reducing the startup time for OpenSearch Dashboards. The initial bundling speed is dependent on your hardware limitations and VNC settings. Feel free to make adjustments to improve the process as needed.


Also i do not think step 8 is necessary here, when we run functional tests, we do not have to start up a dashboard and OpenSearch process.

Yes, it is not necessary. But since it take a bit long while to start OSD at the first time, if run ftr test directly then we will stuck and see errors like below, which might cause some confusion

I think if run step 8 first, then we will make sure the error is caused by ftr test. I think I will update step 8 to make the statement clear.

abbyhu2000 · 2023-04-13T17:44:26Z

docs/docker-dev/docker-dev-setup-manual.md

+
+9. Open a separate terminal and update Chromedriver by executing `node scripts/upgrade_chromedriver.js`, followed by `yarn osd bootstrap`. This will update the Chromedriver version within the node_modules directory, allowing you to use a compatible version with your installed Google Chrome.
+
+10. To execute a single functional test, use the following example: `node scripts/functional_tests.js --config test/functional/config.js --include ciGroup9`. This command runs all the functional tests within ciGroup9.


I ran the chromedriver command and bootstrap, but i still get Chrome failed to start error below. Also when i click the google chrome icon, it failed to load too.

My chromedriver version is ^112

WebDriverError: unknown error: Chrome failed to start: crashed. \u2502 (unknown error: DevToolsActivePort file doesn't exist) \u2502 (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) \u2502 at Object.throwDecodedError (/docker-workspace/OpenSearch-Dashboards/node_modules/selenium-webdriver/lib/error.js:550:15) \u2502 at parseHttpResponse (/docker-workspace/OpenSearch-Dashboards/node_modules/selenium-webdriver/lib/http.js:565:13) \u2502 at Executor.execute (/docker-workspace/OpenSearch-Dashboards/node_modules/selenium-webdriver/lib/http.js:491:26) \u2502 at processTicksAndRejections (internal/process/task_queues.js:95:5) \u2502 at Executor.<anonymous> (/docker-workspace/OpenSearch-Dashboards/test/functional/services/remote/prevent_parallel_calls.ts:65:14) {

will sync up with you on this error

Wired. My guess is that the chrome crash before and got cached. Therefore, you could first try to put down running container, then re-build and bring it up. If this option not working, then maybe make a completely clean:

* Stop all running containers: docker stop $(docker ps -a -q) * Remove all containers: docker rm$(docker ps -a -q) * Remove all images: docker rmi $(docker images -a -q) * Remove all volumes: docker volume rm$(docker volume ls-q) * Remove all networks: docker network rm$(docker network ls-q) * Prune any remaining build caches: docker system prune --all --force --volumes

Then re-run docker compose up -d --build (might take a while). Then run docker-compose -f docker-compose.yml -f docker-compose.selenium.yml up -d --build.

abbyhu2000 · 2023-04-13T17:46:40Z

docs/docker-dev/docker-dev-setup-manual.md

+
+4. Under the Docker tab in VS Code, you should see three containers running: `opensearchproject/opensearch:latest`, `abbyhu/opensearch-dashboards-dev:latest`, and `selenium-test`.
+
+5. First, right-click on `opensearch-dashboards-docker-dev-selenium-test` (which we'll refer to as `selenium-test` for simplicity) and choose `Attach Visual Studio Code`. This action mirrors [Step 8](#install-step-8) from the [Docker Development Environment Setup](#docker-development-environment-setup). By doing this, you'll be able to edit the functional test directly within the `selenium-test` container using VS Code.


I think it would be helpful to add a functional directory file path here?
Something like Go to /docker-workspace/OpenSearch-Dashboards/ and functional tests are located in test/functional

will update.

AMoo-Miki

Have a few nits.

AMoo-Miki · 2023-04-17T16:58:16Z

TESTING.md

@@ -80,13 +82,13 @@ This will create an archive of the data based on the OpenSearch Dashboards versi
 Make sure you run lint checker before submitting a pull request. To run lint checker:
 `node scripts/precommit_hook.js --fix`

-Please ensure that you don't introduce any broken links accidently. For any intentional broken link (e.g. dummy url in unit test), you can add it to [lycheeexclude](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/.lycheeexclude) allow list specifically. 
+Please ensure that you don't introduce any broken links accidently. For any intentional broken link (e.g. dummy url in unit test), you can add it to [lycheeexclude](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/.lycheeexclude) allow list specifically.


Suggested change

Please ensure that you don't introduce any broken links accidently. For any intentional broken link (e.g. dummy url in unit test), you can add it to [lycheeexclude](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/.lycheeexclude) allow list specifically.

Please ensure that you don't introduce any broken links accidently. For any intentional broken link (e.g. dummy url in unit test), you can add it to [lycheeexclude](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/.lycheeexclude) allow-list specifically.

AMoo-Miki · 2023-04-17T17:00:57Z

TESTING.md


 # Writing Tests
 Conventions and best practices for writing tests can be found in [/src/core/TESTING.md](/src/core/TESTING.md)

 # Continuous Integration
-Automated testing is provided with Jenkins for Continuous Integration. Jenkins enables developers to build, deploy, and automate projects and provides us to run groups of tests quickly. CI groups are ran from the [Jenkinsfile](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/Jenkinsfile). 
+Automated testing is provided with Jenkins for Continuous Integration. Jenkins enables developers to build, deploy, and automate projects and provides us to run groups of tests quickly. CI groups are ran from the [Jenkinsfile](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/Jenkinsfile).


Suggested change

Automated testing is provided with Jenkins for Continuous Integration. Jenkins enables developers to build, deploy, and automate projects and provides us to run groups of tests quickly. CI groups are ran from the [Jenkinsfile](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/Jenkinsfile).

Automated testing is provided by Jenkins for Continuous Integration. Jenkins enables developers to build, deploy, and automate projects, and permits us to run groups of tests quickly. CI groups are run from [Jenkinsfile](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/Jenkinsfile).

AMoo-Miki · 2023-04-17T17:02:00Z

TESTING.md

Suggested change

However, these tests will eventually be migrated. Please avoid writing new Mocha tests. For further questions or to check the status please see this [issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/215).

However, these tests will eventually be migrated; please avoid writing new Mocha tests. For further questions or to check the status, please see this [issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/215).

abbyhu2000 · 2023-04-17T19:24:29Z

Verified using docker file for selenium functional test

This PR introduces new Docker files to enable debugging of Selenium functional tests for Docker users. It configures a VNC viewer for real-time browser interaction monitoring during test execution. Additionally, a new section is added to the documentation detailing the process of running and debugging Selenium functional tests using Docker and a VNC viewer. Issue Resolve opensearch-project#3700 Signed-off-by: Anan Zhuang <ananzh@amazon.com>

This PR introduces new Docker files to enable debugging of Selenium functional tests for Docker users. It configures a VNC viewer for real-time browser interaction monitoring during test execution. Additionally, a new section is added to the documentation detailing the process of running and debugging Selenium functional tests using Docker and a VNC viewer. Issue Resolve #3700 Signed-off-by: Anan Zhuang <ananzh@amazon.com> (cherry picked from commit e4fccfc) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…) (#3866) This PR introduces new Docker files to enable debugging of Selenium functional tests for Docker users. It configures a VNC viewer for real-time browser interaction monitoring during test execution. Additionally, a new section is added to the documentation detailing the process of running and debugging Selenium functional tests using Docker and a VNC viewer. Issue Resolve #3700 (cherry picked from commit e4fccfc) Signed-off-by: Anan Zhuang <ananzh@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…nsearch-project#3747) This PR introduces new Docker files to enable debugging of Selenium functional tests for Docker users. It configures a VNC viewer for real-time browser interaction monitoring during test execution. Additionally, a new section is added to the documentation detailing the process of running and debugging Selenium functional tests using Docker and a VNC viewer. Issue Resolve opensearch-project#3700 Signed-off-by: Anan Zhuang <ananzh@amazon.com> Signed-off-by: David Sinclair <david@sinclair.tech>

ananzh requested a review from a team as a code owner March 30, 2023 19:33

ananzh force-pushed the add-ftr-for-docker branch from fc2ecea to dd7ad14 Compare March 30, 2023 19:41

ananzh requested a review from abbyhu2000 March 30, 2023 19:46

ananzh assigned abbyhu2000 Mar 30, 2023

ananzh added docs Improvements or additions to documentation backport 2.x v2.7.0 labels Mar 30, 2023

ananzh force-pushed the add-ftr-for-docker branch 3 times, most recently from 57a7737 to 2ef681f Compare March 30, 2023 22:51

ashwin-pc mentioned this pull request Apr 5, 2023

2.7.0 Platform Upgrades for Plugins (O11y, MDS, Geospatial,+) #3674

Closed

23 tasks

joshuarrrr reviewed Apr 11, 2023

View reviewed changes

abbyhu2000 reviewed Apr 13, 2023

View reviewed changes

ananzh force-pushed the add-ftr-for-docker branch from e5db2f8 to 31a032c Compare April 13, 2023 21:31

ananzh requested review from kavilla, seanneumann, AMoo-Miki, ashwin-pc, zengyan-amazon, kristenTian, zhongnansu and manasvinibs as code owners April 13, 2023 21:31

ananzh force-pushed the add-ftr-for-docker branch from 31a032c to 49f1626 Compare April 13, 2023 21:38

AMoo-Miki previously approved these changes Apr 17, 2023

View reviewed changes

abbyhu2000 previously approved these changes Apr 17, 2023

View reviewed changes

ananzh dismissed stale reviews from abbyhu2000 and AMoo-Miki via f9bab6b April 17, 2023 19:28

abbyhu2000 previously approved these changes Apr 17, 2023

View reviewed changes

AMoo-Miki previously approved these changes Apr 17, 2023

View reviewed changes

ananzh dismissed stale reviews from AMoo-Miki and abbyhu2000 via e14b95b April 17, 2023 20:04

ananzh force-pushed the add-ftr-for-docker branch from f9bab6b to e14b95b Compare April 17, 2023 20:04

abbyhu2000 approved these changes Apr 17, 2023

View reviewed changes

AMoo-Miki approved these changes Apr 17, 2023

View reviewed changes

ananzh merged commit e4fccfc into opensearch-project:main Apr 17, 2023

opensearch-trigger-bot bot mentioned this pull request Apr 17, 2023

[Backport 2.x] Add docker files and instructions for debugging Selenium functional tests #3866

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add docker files and instructions for debugging Selenium functional tests #3747

Add docker files and instructions for debugging Selenium functional tests #3747

ananzh commented Mar 30, 2023

codecov-commenter commented Mar 30, 2023 •

edited

Loading

joshuarrrr commented Apr 10, 2023

joshuarrrr Apr 11, 2023

ananzh Apr 13, 2023 •

edited

Loading

abbyhu2000 Apr 13, 2023

ananzh Apr 13, 2023 •

edited

Loading

abbyhu2000 Apr 13, 2023

ananzh Apr 13, 2023

abbyhu2000 Apr 13, 2023

ananzh Apr 13, 2023 •

edited

Loading

abbyhu2000 Apr 13, 2023

ananzh Apr 13, 2023 •

edited

Loading

abbyhu2000 Apr 13, 2023 •

edited

Loading

ananzh Apr 13, 2023

ananzh Apr 13, 2023

abbyhu2000 Apr 13, 2023

ananzh Apr 13, 2023 •

edited

Loading

AMoo-Miki left a comment

AMoo-Miki Apr 17, 2023

AMoo-Miki Apr 17, 2023

AMoo-Miki Apr 17, 2023

abbyhu2000 commented Apr 17, 2023


		7. After entering the VNC viewer, click the menu at the bottom left. Then click `Internet` to ensure Google works. Also, click `System Tools` and select `LXTerminal` to verify you can type in the terminal. The default terminal path should be `/docker-workspace/OpenSearch-Dashboards`.

		8. Run OpenSearch and OpenSearch Dashboards. First, start OpenSearch by executing `yarn opensearch snapshot` in one terminal. In a second terminal, run `yarn start --no-base-path` to start OpenSearch Dashboards on port 5601. Before running any functional tests, make sure OpenSearch Dashboards is running on port 5601. This process may take 2 to 10 mins, as it requires all the bundles to be assembled. Please be patient. However, this only occurs the first time you run it; afterward, everything is cached, significantly reducing the startup time for OpenSearch Dashboards. The initial bundling speed is dependent on your hardware limitations and VNC settings. Feel free to make adjustments to improve the process as needed.


		9. Open a separate terminal and update Chromedriver by executing `node scripts/upgrade_chromedriver.js`, followed by `yarn osd bootstrap`. This will update the Chromedriver version within the node_modules directory, allowing you to use a compatible version with your installed Google Chrome.

		10. To execute a single functional test, use the following example: `node scripts/functional_tests.js --config test/functional/config.js --include ciGroup9`. This command runs all the functional tests within ciGroup9.


		4. Under the Docker tab in VS Code, you should see three containers running: `opensearchproject/opensearch:latest`, `abbyhu/opensearch-dashboards-dev:latest`, and `selenium-test`.

		5. First, right-click on `opensearch-dashboards-docker-dev-selenium-test` (which we'll refer to as `selenium-test` for simplicity) and choose `Attach Visual Studio Code`. This action mirrors [Step 8](#install-step-8) from the [Docker Development Environment Setup](#docker-development-environment-setup). By doing this, you'll be able to edit the functional test directly within the `selenium-test` container using VS Code.

	Please ensure that you don't introduce any broken links accidently. For any intentional broken link (e.g. dummy url in unit test), you can add it to [lycheeexclude](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/.lycheeexclude) allow list specifically.
	Please ensure that you don't introduce any broken links accidently. For any intentional broken link (e.g. dummy url in unit test), you can add it to [lycheeexclude](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/.lycheeexclude) allow-list specifically.

	Automated testing is provided with Jenkins for Continuous Integration. Jenkins enables developers to build, deploy, and automate projects and provides us to run groups of tests quickly. CI groups are ran from the [Jenkinsfile](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/Jenkinsfile).
	Automated testing is provided by Jenkins for Continuous Integration. Jenkins enables developers to build, deploy, and automate projects, and permits us to run groups of tests quickly. CI groups are run from [Jenkinsfile](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/Jenkinsfile).

	However, these tests will eventually be migrated. Please avoid writing new Mocha tests. For further questions or to check the status please see this [issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/215).
	However, these tests will eventually be migrated; please avoid writing new Mocha tests. For further questions or to check the status, please see this [issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/215).

Add docker files and instructions for debugging Selenium functional tests #3747

Add docker files and instructions for debugging Selenium functional tests #3747

Conversation

ananzh commented Mar 30, 2023

Description

Issues Resolved

Check List

codecov-commenter commented Mar 30, 2023 • edited Loading

Codecov Report

joshuarrrr commented Apr 10, 2023

Choose a reason for hiding this comment

ananzh Apr 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ananzh Apr 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ananzh Apr 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ananzh Apr 13, 2023 • edited Loading

Choose a reason for hiding this comment

abbyhu2000 Apr 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ananzh Apr 13, 2023 • edited Loading

Choose a reason for hiding this comment

AMoo-Miki left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abbyhu2000 commented Apr 17, 2023

codecov-commenter commented Mar 30, 2023 •

edited

Loading

ananzh Apr 13, 2023 •

edited

Loading

ananzh Apr 13, 2023 •

edited

Loading

ananzh Apr 13, 2023 •

edited

Loading

ananzh Apr 13, 2023 •

edited

Loading

abbyhu2000 Apr 13, 2023 •

edited

Loading

ananzh Apr 13, 2023 •

edited

Loading