Run tests using a container #2704

narrieta · 2022-11-11T20:46:50Z

Installing LISA typically takes 5-7 min. These changes execute the tests using a container where LISA is pre-installed; pulling the container image typically takes 50-90 seconds.

I also fixed some Python imports that I missed in my previous PR, renamed some resources in our pipeline and made some of those resources parameters to the pipeline.

narrieta · 2022-11-11T20:47:43Z

tests_e2e/azure-pipelines.yml

-    value: '8e037ad4-618f-4466-8bc8-5099d41ac15b'
-  - name: testsSourcesDirectory
-    value: "$(Build.SourcesDirectory)/tests_e2e"
+    value: 'azuremanagement'


I renamed the connection and made the actual subscription ID a parameter.

narrieta · 2022-11-11T20:49:13Z

tests_e2e/docker/Dockerfile

@@ -0,0 +1,74 @@
+#


This is basically the previous install_dependencies.sh now as a Dockerfile

narrieta · 2022-11-11T20:50:35Z

tests_e2e/requirements.txt

@@ -10,4 +10,3 @@ cryptography
 distro
 junitparser
 msrestazure
-python-dotenv


I removed this dependency. We are over-using the environment to pass data across different components. We need to reduce the use of environment variables, so this module should not be needed.

I agree but at the same time it's going to be challenging if the data is scope to that component is needed by other component like resource group and names if any task has hard dependency on it. We need to figure out way to pass between components otherwise we have to live with env.

Use of environment variables should be restricted to the interface with the Azure pipelines and probably a few (if any) places. If used for other things other than communication with the pipeline, we need to be very explicit about what variables we are using. There should not be a need for this module to save and restore the entire environment.

narrieta · 2022-11-11T20:50:59Z

tests_e2e/scenario_utils/azure_models.py

@@ -12,8 +12,8 @@
 from azure.mgmt.resource import ResourceManagementClient
 from msrestazure.azure_exceptions import CloudError

-from dcr.scenario_utils.logging_utils import LoggingHandler
-from dcr.scenario_utils.models import get_vm_data_from_env, VMModelType, VMMetaData
+from tests_e2e.scenario_utils.logging_utils import LoggingHandler


fixing some dependencies I missed in my previous PR

codecov · 2022-11-11T20:52:28Z

Codecov Report

Merging #2704 (33d5d61) into develop (97e4d39) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #2704   +/-   ##
========================================
  Coverage    71.95%   71.95%           
========================================
  Files          104      104           
  Lines        15765    15765           
  Branches      2244     2244           
========================================
  Hits         11343    11343           
  Misses        3906     3906           
  Partials       516      516

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

narrieta · 2022-11-11T20:52:30Z

tests_e2e/scenario_utils/logging_utils.py

@@ -0,0 +1,33 @@
+# Create a base class


When I fixed the dependencies, I realized I missed this file in my previous PR. This file comes from the current Azure Pipeline and needs to be rewritten

narrieta · 2022-11-11T20:53:24Z

tests_e2e/scripts/install_dependencies.sh

@@ -1,43 +0,0 @@
-#!/usr/bin/env bash


The Dockerfile replaces this script

narrieta · 2022-11-11T21:12:06Z

tests_e2e/docker/Dockerfile

+    cd lisa                                                                                                   && \
+    poetry config virtualenvs.in-project true                                                                 && \
+    make setup                                                                                                && \
+    chmod +r /home/waagent/.config/pypoetry/config.toml                                                       && \


this chmod is a left-over from some experimentation I was doing, i'll remove it

narrieta · 2022-11-11T21:16:50Z

tests_e2e/scripts/execute_tests_container.sh

+sudo chown 1000 "$HOME/logs"
+
+docker run --rm \
+      --volume "$BUILD_SOURCESDIRECTORY:/home/waagent/WALinuxAgent" \


"--volume <host_path>:<container_path>" shares the local host_path with the container as container-path.

we are sharing the agent's code and the the ssh key used to connect to the test VMs (both of which Azure Pipelines downloaded to the VM); also, we are specifying a location where the container will drop the logs and then the host VM will pick them up from there to report results.

nagworld9 · 2022-11-18T19:55:11Z

tests_e2e/requirements.txt

@@ -10,4 +10,3 @@ cryptography
 distro
 junitparser
 msrestazure
-python-dotenv


I agree but at the same time it's going to be challenging if the data is scope to that component is needed by other component like resource group and names if any task has hard dependency on it. We need to figure out way to pass between components otherwise we have to live with env.

nagworld9 · 2022-11-18T20:27:39Z

tests_e2e/scripts/execute_tests.sh

+  --log_path "$HOME/logs" \
+  --working_path "$HOME/logs" \
+  -v subscription_id:"$SUBSCRIPTION_ID" \
+  -v identity_file:"$HOME/.ssh/id_rsa.pub"


The path $HOME refers to container path (/home/waagent), right?

Yes, this script is executed within the container using the 'waagent' user. $USER, $HOME, etc refer to 'waagent'.

nagworld9 · 2022-11-18T20:31:04Z

tests_e2e/scripts/execute_tests_container.sh

+
+docker run --rm \
+      --volume "$BUILD_SOURCESDIRECTORY:/home/waagent/WALinuxAgent" \
+      --volume "$DOWNLOADSSHKEY_SECUREFILEPATH:/home/waagent/id_rsa" \


I don't know if it's possible, why don't we directly provide /.ssh folder in the container path that way we could avoid doing extra steps copying from one place to other when lisa script run

$BUILD_SOURCESDIRECTORY and $DOWNLOADSSHKEY_SECUREFILEPATH are inputs to the container, while $HOME/logs is its output.

We should not modify the input directories, since the may be coming from a developer's machine while debugging, writing a test, etc (the script that executes the tests creates a file under '/home/waagent' and changes the mode of '/home/waagent/id_rsa')

I see your point. If the file semantics changes in container path would change the input path too. Makes sense.

nagworld9 · 2022-11-23T19:21:16Z

tests_e2e/scripts/execute_tests_container.sh

+
+docker run --rm \
+      --volume "$BUILD_SOURCESDIRECTORY:/home/waagent/WALinuxAgent" \
+      --volume "$DOWNLOADSSHKEY_SECUREFILEPATH:/home/waagent/id_rsa" \


I see your point. If the file semantics changes in container path would change the input path too. Makes sense.

Run tests within a container

7ed71ef

narrieta requested review from ZhidongPeng, nagworld9 and maddieford as code owners November 11, 2022 20:46

narrieta commented Nov 11, 2022

View reviewed changes

tests_e2e/docker/Dockerfile

@@ -0,0 +1,74 @@

#

Copy link

Member Author

narrieta Nov 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is basically the previous install_dependencies.sh now as a Dockerfile

narrieta commented Nov 11, 2022

View reviewed changes

tests_e2e/scripts/install_dependencies.sh

@@ -1,43 +0,0 @@

#!/usr/bin/env bash

Copy link

Member Author

narrieta Nov 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Dockerfile replaces this script

narrieta commented Nov 11, 2022

View reviewed changes

cleanup

33d5d61

nagworld9 reviewed Nov 18, 2022

View reviewed changes

maddieford approved these changes Nov 21, 2022

View reviewed changes

nagworld9 approved these changes Nov 23, 2022

View reviewed changes

narrieta merged commit a07e48a into Azure:develop Nov 23, 2022

narrieta deleted the docker branch November 23, 2022 19:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run tests using a container #2704

Run tests using a container #2704

narrieta commented Nov 11, 2022

narrieta Nov 11, 2022

narrieta Nov 11, 2022

narrieta Nov 11, 2022

nagworld9 Nov 18, 2022

narrieta Nov 23, 2022

narrieta Nov 11, 2022

codecov bot commented Nov 11, 2022 •

edited

Loading

narrieta Nov 11, 2022

narrieta Nov 11, 2022

narrieta Nov 11, 2022

narrieta Nov 11, 2022 •

edited

Loading

nagworld9 Nov 18, 2022

nagworld9 Nov 18, 2022

narrieta Nov 23, 2022

nagworld9 Nov 18, 2022

narrieta Nov 23, 2022 •

edited

Loading

nagworld9 Nov 23, 2022

nagworld9 Nov 23, 2022

Run tests using a container #2704

Run tests using a container #2704

Conversation

narrieta commented Nov 11, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Nov 11, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

narrieta Nov 11, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

narrieta Nov 23, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Nov 11, 2022 •

edited

Loading

narrieta Nov 11, 2022 •

edited

Loading

narrieta Nov 23, 2022 •

edited

Loading