Add docker files and instructions for debugging functional tests (ope…

…nsearch-project#3747)

This PR introduces new Docker files to enable debugging of Selenium
functional tests for Docker users. It configures a VNC viewer for
real-time browser interaction monitoring during test execution.

Additionally, a new section is added to the documentation detailing
the process of running and debugging Selenium functional tests
using Docker and a VNC viewer.

Issue Resolve
opensearch-project#3700

Signed-off-by: Anan Zhuang <ananzh@amazon.com>

CHANGELOG.md

@@ -158,6 +158,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - [Doc] Update SECURITY.md with instructions for nested dependencies and backporting ([#3497](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3497))
 - [Doc] [Console] Fix/update documentation links in Dev Tools console ([#3724](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3724))
 - [Doc] Update DEVELOPER_GUIDE.md with added manual bootstrap timeout solution and max virtual memory error solution with docker ([#3764](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3764))
+- [Doc] Add docker files and instructions for debugging Selenium functional tests ([#3747](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3747))
 
 ### 🛠 Maintenance
 

TESTING.md

@@ -18,7 +18,7 @@ OpenSearch Dashboards uses [Jest](https://jestjs.io/) for unit and integration t
 
 In general, we recommend four tiers of tests:
 * Unit tests: Unit tests: small and modular tests that utilize mocks for external dependencies.
-* Integration tests: higher-level tests that verify interactions between systems (eg. HTTP APIs, OpenSearch API calls, calling other plugin). 
+* Integration tests: higher-level tests that verify interactions between systems (eg. HTTP APIs, OpenSearch API calls, calling other plugin).
 * End-to-end tests (e2e): functional tests that verify behavior in a web browser.
 * Backwards Compatibility tests: cypress tests that verify any changes are backwards compatible with previous versions.
 
@@ -57,6 +57,8 @@ Say that you would want to debug a test in CI group 1, you can run the following
 
 This will print off an address, to which you could open your chrome browser on your instance and navigate to `chrome://inspect/#devices` and inspect the functional test runner `scripts/functional_tests.js`.
 
+If you prefer to run functional tests using Docker, you can find instructions on how to set up and debug functional tests in a Docker environment in the [Debug Functional Tests](docs/docker-dev/docker-dev-setup-manual.md#debug-functional-tests) section of the `docker-dev-setup-manual.md` file.
+
 ### Backwards Compatibility tests
 To run all the backwards compatibility tests on OpenSearch Dashboards without security:
 
@@ -80,13 +82,13 @@ This will create an archive of the data based on the OpenSearch Dashboards versi
 Make sure you run lint checker before submitting a pull request. To run lint checker:
 `node scripts/precommit_hook.js --fix`
 
-Please ensure that you don't introduce any broken links accidently. For any intentional broken link (e.g. dummy url in unit test), you can add it to [lycheeexclude](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/.lycheeexclude) allow list specifically. 
+Please ensure that you don't introduce any broken links accidently. For any intentional broken link (e.g. dummy url in unit test), you can add it to [lycheeexclude](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/.lycheeexclude) allow-list specifically.
 
 # Writing Tests
 Conventions and best practices for writing tests can be found in [/src/core/TESTING.md](/src/core/TESTING.md)
 
 # Continuous Integration
-Automated testing is provided with Jenkins for Continuous Integration. Jenkins enables developers to build, deploy, and automate projects and provides us to run groups of tests quickly. CI groups are ran from the [Jenkinsfile](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/Jenkinsfile). 
+Automated testing is provided by Jenkins for Continuous Integration. Jenkins enables developers to build, deploy, and automate projects, and permits us to run groups of tests quickly. CI groups are run from [Jenkinsfile](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/Jenkinsfile).
 
 # Environment misc
 Selenium tests are run in headless mode on CI. Locally the same tests will be executed in a real browser. You can activate headless mode by setting the environment variable:
@@ -114,4 +116,4 @@ sudo apt-get install -y --allow-downgrades /tmp/chrome.deb
 Although Jest is the standard for this project, there are a few Mocha tests that still exist. You can run these tests by running:
 `yarn test:mocha`
 
-However, these tests will eventually be migrated. Please avoid writing new Mocha tests. For further questions or to check the status please see this [issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/215).
+However, these tests will eventually be migrated; please avoid writing new Mocha tests. For further questions or to check the status, please see this [issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/215).

dev-tools/install-docker-dev.sh

@@ -21,8 +21,22 @@ osd_do_copy_dev_docker_files(){
 	mkdir -p "$INSTALL_DIR"
 	osd_download -s "$ENTRYPOINT_SRC" -o "./$INSTALL_DIR/entrypoint.sh"
 	osd_download -s "$DOCKER_COMPOSE_SRC" -o "./$INSTALL_DIR/docker-compose.yml"
+
+	if [ "$1" = "--ftr" ]; then
+	    printf "run ftr"
+        local START_VNC_SRC
+        START_VNC_SRC="https://raw.githubusercontent.com/opensearch-project/OpenSearch-Dashboards/main/docs/docker-dev/start-vnc.sh"
+        local DOCKERFILE_SELENIUM_SRC
+        DOCKERFILE_SELENIUM_SRC="https://raw.githubusercontent.com/opensearch-project/OpenSearch-Dashboards/main/docs/docker-dev/Dockerfile.selenium"
+        local DOCKER_COMPOSE_SELENIUM_SRC
+        DOCKER_COMPOSE_SELENIUM_SRC="https://raw.githubusercontent.com/opensearch-project/OpenSearch-Dashboards/main/docs/docker-dev/docker-compose.selenium.yml"
+
+        osd_download -s "$START_VNC_SRC" -o "./$INSTALL_DIR/start-vnc.sh"
+        osd_download -s "$DOCKERFILE_SELENIUM_SRC" -o "./$INSTALL_DIR/Dockerfile.selenium"
+        osd_download -s "$DOCKER_COMPOSE_SELENIUM_SRC" -o "./$INSTALL_DIR/docker-compose.selenium.yml"
+    fi
 }
 
-osd_do_copy_dev_docker_files
+osd_do_copy_dev_docker_files $1
 
 } # this ensures the entire script is downloaded #

docs/docker-dev/Dockerfile.selenium

@@ -0,0 +1,70 @@
+FROM abbyhu/opensearch-dashboards-dev:latest
+
+# Switch to root user
+USER root
+
+# Install the locales package
+# Uncomment the en_US.UTF-8 UTF-8 line in the sytstem /etc/locale.gen file
+# Then generate the locales and update the system locale to en_US.UTF-8
+# Install all other requested packages
+RUN apt-get update && \
+    apt-get install -y locales && \
+    sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
+    dpkg-reconfigure --frontend=noninteractive locales && \
+    update-locale LANG=en_US.UTF-8 && \
+    apt-get install -y xvfb x11vnc openbox lxde-core lxterminal wget apt-transport-https sudo
+
+ENV LANG en_US.UTF-8
+ENV LC_ALL en_US.UTF-8
+
+# Create the LXTerminal configuration directory and set the encoding
+RUN mkdir -p /etc/xdg/lxterminal && \
+    echo '[General]' >> /etc/xdg/lxterminal/lxterminal.conf && \
+    echo 'encoding=UTF-8' >> /etc/xdg/lxterminal/lxterminal.conf
+
+# Specify the version of Chrome that matches the version of chromedriver in the package.json.
+#ARG CHROME_VERSION=107.0.5304.121-1
+
+## Install Google Chrome version 107
+#RUN curl -sSL https://dl.google.com/linux/linux_signing_key.pub | apt-key add - && \
+#    echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' > /etc/apt/sources.list.d/google-chrome.list && \
+#    apt-get update && \
+#    wget -O /tmp/chrome.deb https://dl.google.com/linux/chrome/deb/pool/main/g/google-chrome-stable/google-chrome-stable_${CHROME_VERSION}_amd64.deb && \
+#    apt-get install -y /tmp/chrome.deb --no-install-recommends && \
+#    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
+
+# Install Google Chrome
+RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
+    echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' > /etc/apt/sources.list.d/google-chrome.list && \
+    apt-get update && \
+    apt-get install -y google-chrome-stable && \
+    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
+
+# Create the directory and set the ownership for osd-dev user
+RUN mkdir -p /docker-workspace/OpenSearch-Dashboards/.opensearch && \
+    chown -R osd-dev /docker-workspace/OpenSearch-Dashboards/.opensearch
+
+COPY entrypoint.sh /entrypoint.sh
+COPY start-vnc.sh /start-vnc.sh
+
+RUN chmod +x /entrypoint.sh /start-vnc.sh
+
+ENTRYPOINT ["/entrypoint.sh"]
+
+# Create a Google Chrome desktop file with the specified launch options.
+# Currently Google Chrome is not available in the menu of your VNC Viewer session.
+# To enable that, you need to open the terminal and run:
+# google-chrome --no-sandbox --disable-gpu --remote-debugging-address=0.0.0.0 --remote-debugging-port=9222
+# This part is added to automate this process by creating a desktop file for Google Chrome.
+RUN echo '[Desktop Entry]\n\
+Version=1.0\n\
+Name=Google Chrome\n\
+GenericName=Web Browser\n\
+Comment=Access the Internet\n\
+Exec=google-chrome --no-sandbox --disable-gpu --remote-debugging-address=0.0.0.0 --remote-debugging-port=9222 %U\n\
+Terminal=false\n\
+Icon=google-chrome\n\
+Type=Application\n\
+Categories=Network;WebBrowser;\n\
+MimeType=text/html;text/xml;application/xhtml_xml;x-scheme-handler/http;x-scheme-handler/https;'\
+> /usr/share/applications/google-chrome.desktop

docs/docker-dev/docker-compose.selenium.yml

@@ -0,0 +1,24 @@
+version: '3'
+services:
+  selenium-test:
+    build:
+      context: .
+      dockerfile: Dockerfile.selenium
+    container_name: selenium-test
+    depends_on:
+      - dev-env
+    environment:
+      - DISPLAY=:99
+    volumes:
+      - /dev/shm:/dev/shm
+      - .:/workspace
+    networks:
+      - opensearch-net
+    ports:
+      - 5900:5900
+    entrypoint: ["/start-vnc.sh"]
+volumes:
+  opensearch-data:
+  osd-dev:
+networks:
+  opensearch-net:

docs/docker-dev/docker-dev-setup-manual.md

@@ -1,18 +1,18 @@
 # Docker Development Environment Setup
 The following instructions demonstrate how to set up a development environment for OpenSearch Dashboards using Docker. It utilizes tools such as `Docker` and `VS Code`, and users should be familiar with the basic usages of them. Users will be able to develop and run the application inside VS Code without additional configurations.
 
-1. Install [Docker](https://docs.docker.com/get-docker/) if not already installed. 
+1. Install [Docker](https://docs.docker.com/get-docker/) if not already installed.
     * Make sure that Docker daemon is running. (For windows and macos，you need to have [Docker Desktop](https://docs.docker.com/desktop/), or its alternatives, such as [Finch](https://github.com/runfinch/finch))
 
 2. In the terminal, run the command below.
-    * This should create a folder named `opensearch-dashboards-docker-dev` and it should contain two files: `docker-compose.yml` and `entrypoint.sh`. 
+    * This should create a folder named `opensearch-dashboards-docker-dev` and it should contain two files: `docker-compose.yml` and `entrypoint.sh`.
     * Here is the link to the installer script: `https://raw.githubusercontent.com/opensearch-project/OpenSearch-Dashboards/main/dev-tools/install-docker-dev.sh` if needed.
 
 ```bash
 curl -o- https://raw.githubusercontent.com/opensearch-project/OpenSearch-Dashboards/main/dev-tools/install-docker-dev.sh | bash
 ```
 
-3. Open VS Code or [install it](https://code.visualstudio.com/download), if it's not already installed. 
+3. Open VS Code or [install it](https://code.visualstudio.com/download), if it's not already installed.
     * Make sure VS Code has the extensions `Dev Containers` and `Docker` installed. If not, go to `Extensions` tab, search and install them.
 
 4. Under the Discover tab, click `Open Folder`, and open the `opensearch-dashboards-docker-dev` folder that we just created.
@@ -21,34 +21,34 @@ curl -o- https://raw.githubusercontent.com/opensearch-project/OpenSearch-Dashboa
     * If fork repo has not been created: Go to [OpenSearch Dashboards github page](https://github.com/opensearch-project/OpenSearch-Dashboards) and under fork, select create a new fork, and then copy the https link of the fork url and use it in the above command. The command needs to be re-run every time it re-start the docker compose file in a new terminal.
 ```bash
 export REPO_URL=[insert your fork repo url here]
-``` 
-    
-6. Run the `docker-compose.yml` file in the background by typing: 
+```
+
+6. Run the `docker-compose.yml` file in the background by typing:
 ```bash
 docker compose up -d --build
 ```
 
-7. Under the `Docker` tab in VS Code, verify that there are two containers running: `opensearchproject/opensearch:latest` and `abbyhu/opensearch-dashboards-dev:latest`. 
-    * This can also be verified by using the command line: 
+7. Under the `Docker` tab in VS Code, verify that there are two containers running: `opensearchproject/opensearch:latest` and `abbyhu/opensearch-dashboards-dev:latest`.
+    * This can also be verified by using the command line:
     ```bash
     docker ps
     ```
 
-8. Right-click `abbyhu/opensearch-dashboards-dev:latest`, and select `Attach Visual Studio Code`. 
-    * This will ssh into the container and you will be able to view and edit the files using VS Code as the code editor. 
-    * If you do not wish to use VS Code as the code editor, the alternative way of ssh into the container is by using the command below: 
+8. <span id="install-step-8"></span> Right-click `abbyhu/opensearch-dashboards-dev:latest`, and select `Attach Visual Studio Code`.
+    * This will ssh into the container and you will be able to view and edit the files using VS Code as the code editor.
+    * If you do not wish to use VS Code as the code editor, the alternative way of ssh into the container is by using the command below:
     ```bash
     docker exec -it dev-env /bin/bash
     ```
 
-9. For the new VS Code window, if it is not showing the repository code, then select `Open Folder`. Then open `/workspace-docker/OpenSearch-Dashboards`.
+9. For the new VS Code window, if it is not showing the repository code, then select `Open Folder`. Then open `/docker-workspace/OpenSearch-Dashboards`.
 
 10. In the terminal, start the OpenSearch Dashboards application by typing:
 ```bash
 yarn start:docker
 ```
 
-11. Now that OpenSearch Dashboards is running, you should be able to see a log line similar to `[info][server][OpenSearchDashboards][http] http server running at http://0.0.0.0:5603/dog`. 
+11. Now that OpenSearch Dashboards is running, you should be able to see a log line similar to `[info][server][OpenSearchDashboards][http] http server running at http://0.0.0.0:5603/dog`.
     * The last three letters `dog` are randomly generated every time we start dashboards.
 
 12. Wait for the optimizer to run, which takes about 100s - 200s. Once the optimizer is finished running, it will show a line such as `[success][@osd/optimizer] 48 bundles compiled successfully after 204.9 sec, watching for changes`.
@@ -57,3 +57,42 @@ yarn start:docker
     * Files are constantly watched, so when you make code changes, OpenSearch Dashboards will rebuild and restart automatically. Refresh the link in the browser and the new changes should be applied.
 
 14. `Git` is already configured in the `entrypoint.sh` file, and the remote is already tracking the fork repository. You can start contributing by creating your branch off the main, and commit your first PR!
+
+# Debug Functional Tests
+This section explains how to run Selenium functional tests using Docker and debug the tests through browser interaction.
+
+1. Install a VNC viewer, such as [RealVNC](https://www.realvnc.com/en/connect/download/viewer/), if you haven't already. You'll need to set up a free account to use RealVNC. This VNC viewer will enable you to view and interact with the browser running the Selenium tests. The subsequent steps will illustrate the setup process using RealVNC as an example.
+
+2. Make sure you have completed steps 1-5 in the [Docker Development Environment Setup](#docker-development-environment-setup). Now, ensure you have 5 files, `docker-compose.yml`, `docker-compose.selenium.yml`, `Dockerfile.selenium`, `entrypoint.sh` and `start-vnc.sh`. You can also download them by running the installer script. This time you need to pass a parameter `--ftr`.
+
+```bash
+curl -o- https://raw.githubusercontent.com/opensearch-project/OpenSearch-Dashboards/main/dev-tools/install-docker-dev.sh | bash -s -- --ftr
+```
+
+3. In the terminal, run the following commands to stop any running containers, build the new configuration, and start the containers:
+
+```bash
+docker-compose -f docker-compose.yml -f docker-compose.selenium.yml down
+docker-compose -f docker-compose.yml -f docker-compose.selenium.yml up -d --build
+```
+
+4. Under the Docker tab in VS Code, you should see three containers running: `opensearchproject/opensearch:latest`, `abbyhu/opensearch-dashboards-dev:latest`, and `selenium-test`.
+
+5. First, right-click on `opensearch-dashboards-docker-dev-selenium-test` (which we'll refer to as `selenium-test` for simplicity) and choose `Attach Visual Studio Code`. This action mirrors [Step 8](#install-step-8) from the [Docker Development Environment Setup](#docker-development-environment-setup). By doing this, you'll be able to edit the functional test directly within the `selenium-test` container using VS Code which is located at `test/functional`.
+
+6. Connect to the VNC server running in the Docker container. First, launch the VNC viewer, then enter `localhost:5900` at the top and press return. You will see a popup window labeled `Unencrypted connection`; click continue. Enter the password (the default password is 123), which is configured in `start-vnc.sh`.
+
+7. After entering the VNC viewer, click the menu at the bottom left. Then click `Internet` to ensure Google works. Also, click `System Tools` and select `LXTerminal` to verify you can type in the terminal. The default terminal path should be `/docker-workspace/OpenSearch-Dashboards`.
+
+8. Run OpenSearch and OpenSearch Dashboards separately before running functional tests. This can help isolate the functional test process and confirm that any issues are caused only by the functional test. Follow these steps:
+   * First, start OpenSearch by executing `yarn opensearch snapshot` in one terminal.
+   * In a second terminal, run `yarn start --no-base-path` to start OpenSearch Dashboards on port 5601.
+   * Make sure OpenSearch Dashboards is running on port 5601 before running any functional tests.
+
+   Please note that the initial startup process may take 2 to 10 minutes, as it requires all the bundles to be assembled. This only occurs the first time you run it; afterward, everything is cached, significantly reducing the startup time for OpenSearch Dashboards. The initial bundling speed is dependent on your hardware limitations and VNC settings. Feel free to make adjustments to improve the process as needed.
+
+9. Open a separate terminal and update Chromedriver by executing `node scripts/upgrade_chromedriver.js`, followed by `yarn osd bootstrap`. This will update the Chromedriver version within the node_modules directory, allowing you to use a compatible version with your installed Google Chrome.
+
+10. To run functional tests, please refer to the [Functional tests section](../../TESTING.md#functional-tests) in the `TESTING.md` file.
+
+11. The Selenium tests will be executed in the browser, viewable through the VNC viewer. You can monitor the test progress in real-time.

docs/docker-dev/start-vnc.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+
+# Start the VNC server
+export DISPLAY=:99
+
+mkdir -p /home/.vnc
+
+Xvfb :99 -screen 0 1280x1024x24 &
+
+x11vnc -forever -display :99 -rfbport 5900 -passwd 123 -bg -o /home/.vnc/x11vnc.log
+
+sudo -u osd-dev startlxde &
+
+# Keep the container running
+tail -f /dev/null