Python configuration for tests #2793

narrieta · 2023-03-30T16:07:40Z

This PR changes the Python configuration for the end to end tests to an explicit approach: instead of having a default Python, tests need to specify the Python to use (the system's Python or Pypy).

Always install Pypy on test VMs
Install the Test agent as a module for Pypy
Add utilities to find the location of the Agent's executable (waagent) and modules (azurelinuxagent.*), as well as the location of the Python used by the system to execute the Agent.
Install tests libraries to ~/lib, instead of ~/bin

The PR also includes some minor cleanup fixes. I'll point them out in review comments.

narrieta · 2023-03-30T16:09:38Z

tests_e2e/orchestrator/lib/agent_test_suite.py

@@ -277,54 +277,74 @@ def _setup_node(self) -> None:
        log.info("Resource Group: %s", self.context.vm.resource_group)
        log.info("")

-        self.context.ssh_client.run_command("mkdir -p ~/bin/tests_e2e/tests; touch ~/bin/agent-env")


Now we prepare a tarball on the orchestrator machine that is already structured in the way it needs to be setup on the test VMs, instead of using SSH to create/copy these files. This should make easier to include additional files that need to be copied to the test VMs.

narrieta · 2023-03-30T16:14:54Z

tests_e2e/orchestrator/lib/agent_test_suite.py

@@ -447,15 +469,15 @@ def _execute_test_suite(self, suite: TestSuiteInfo) -> bool:
                        test_start_time: datetime.datetime = datetime.datetime.now()

                        log.info("******** Executing %s", test_name)
-                        self.context.lisa_log.info("******** Executing %s", test_full_name)
+                        self.context.lisa_log.info("Executing test %s", test_full_name)


Minor formatting change. We used to output "********" to the LISA log to make those messages stand out. My previous PR created a log for each environment, so now the LISA log is not too crowded and this marker is not needed anymore.

narrieta · 2023-03-30T16:16:49Z

tests_e2e/orchestrator/lib/agent_test_suite.py

@@ -528,7 +550,7 @@ def _check_agent_log(self) -> bool:
            self.context.lisa_log.info("Checking agent log on the test node")
            log.info("Checking agent log on the test node")

-            output = self.context.ssh_client.run_command("check-agent-log.py -j")
+            output = self.context.ssh_client.run_command("$(get-agent-python) ~/bin/check-agent-log.py -j")


get_distro() in version.py is not returning the expected values in some distros when using Pypy. I need to follow up that, and in the meanwhile I am using the system's python to execute this script.

narrieta · 2023-03-30T16:18:02Z

tests_e2e/orchestrator/lib/agent_test_suite.py

@@ -551,21 +573,17 @@ def _check_agent_log(self) -> bool:
                log.info("There are no errors in the agent log")
                return True

-            log_path: Path = self.context.log_path/f"CheckAgentLog-{self.context.environment_name}.log"


My previous PR create a log for each environment, so now we do not need to create this extra CheckAgentLog-*.log, we can simply log those messages to the environment log.

narrieta · 2023-03-30T16:20:34Z

tests_e2e/orchestrator/scripts/get-agent-modules-path

 #
 set -euo pipefail

-# python3 is available on most distros


Somehow git captures this as a rename + changes. I actually removed find-python and created a totally unrelated script, get-agent-modules-path

narrieta · 2023-03-30T16:24:10Z

tests_e2e/orchestrator/scripts/uncompress.py

@@ -1,5 +1,3 @@
-#!/usr/bin/env python3


This script runs using whatever Python is installed on the test VM, and that is not necessarily Python 3.

narrieta · 2023-03-30T16:26:04Z

tests_e2e/orchestrator/lib/agent_test_suite.py

        """
-        Renames the given log to prefix it with "_".
+        Adds a message to indicate the log contains errors.


Marking failed logs by renaming them makes developer runs a little harder, since one may be watching the progress in a test log while the file is renamed. Instead, we add the message below and do not change the name.

nagworld9 · 2023-03-30T23:58:46Z

tests_e2e/orchestrator/scripts/get-agent-python

+# Returns the PYTHONPATH on which the azurelinuxagent and associated modules are located.
+#
+# To do this, the script tries to find the python command used to start the agent and then
+# returns the value of site.getsitepackages().


Need to change the comment. Now this script returns agent python executable path

nagworld9 · 2023-03-31T00:05:15Z

tests_e2e/orchestrator/scripts/install-tools

+#      ln -s "$python" ~/bin/python3
+#    fi
+#  fi
+#fi


This is commented, are we still figuring out for flatcar?

codecov · 2023-03-31T21:36:56Z

Codecov Report

Merging #2793 (ea332fd) into develop (08f86db) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #2793   +/-   ##
========================================
  Coverage    72.01%   72.01%           
========================================
  Files          104      104           
  Lines        15870    15870           
  Branches      2273     2273           
========================================
  Hits         11429    11429           
  Misses        3916     3916           
  Partials       525      525

see 2 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

maddieford · 2023-04-04T16:57:37Z

tests_e2e/pipeline/scripts/execute_tests.sh

@@ -86,18 +86,27 @@ sudo find "$LOGS_DIRECTORY" -exec chown "$USER" {} \;
 #
 # Move the relevant logs to the staging directory
 #
+# Move the logs for failed tests to a temporary location


Why do we move failed tests to a temporary location here?

we move them back on line 105

the temp location makes it easy to separate them from other logs

narrieta added 2 commits March 30, 2023 08:36

Python configuration for tests

35b0c23

.

ddb208a

narrieta requested review from ZhidongPeng, nagworld9 and maddieford as code owners March 30, 2023 16:07

narrieta commented Mar 30, 2023

View reviewed changes

nagworld9 reviewed Mar 31, 2023

View reviewed changes

fix distro and platform modules

693eb5c

narrieta added 4 commits April 3, 2023 06:38

debug info

7ea2660

.

62303dc

.

6ade9bc

.

ea332fd

maddieford reviewed Apr 4, 2023

View reviewed changes

nagworld9 approved these changes Apr 4, 2023

View reviewed changes

maddieford approved these changes Apr 4, 2023

View reviewed changes

narrieta merged commit 1601e88 into Azure:develop Apr 4, 2023

narrieta deleted the python branch April 4, 2023 19:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python configuration for tests #2793

Python configuration for tests #2793

narrieta commented Mar 30, 2023

narrieta Mar 30, 2023

narrieta Mar 30, 2023

narrieta Mar 30, 2023

narrieta Mar 30, 2023

narrieta Mar 30, 2023

narrieta Mar 30, 2023

narrieta Mar 30, 2023

nagworld9 Mar 30, 2023

nagworld9 Mar 31, 2023

codecov bot commented Mar 31, 2023 •

edited

Loading

maddieford Apr 4, 2023

narrieta Apr 4, 2023

Python configuration for tests #2793

Python configuration for tests #2793

Conversation

narrieta commented Mar 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Mar 31, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Mar 31, 2023 •

edited

Loading