Run tests using a container #2704
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| # | ||
|
There was a problem hiding this comment. This is basically the previous install_dependencies.sh now as a Dockerfile |
||
| # * Sample command to build the image: | ||
| # | ||
| # docker build -t waagenttests . | ||
| # | ||
| # * Sample command to execute a container interactively: | ||
| # | ||
| # docker run --rm -it -v /home/nam/src/WALinuxAgent:/home/waagent/WALinuxAgent waagenttests bash --login | ||
| # | ||
| FROM ubuntu:latest | ||
| LABEL description="Test environment for WALinuxAgent" | ||
|
|
||
| SHELL ["/bin/bash", "-c"] | ||
|
|
||
| # | ||
| # Install the required packages as root | ||
| # | ||
| USER root | ||
|
|
||
| RUN \ | ||
| apt-get update && \ | ||
| \ | ||
| # \ | ||
| # Install basic dependencies \ | ||
| # \ | ||
| apt-get install -y git python3.10 python3.10-dev curl && \ | ||
| ln /usr/bin/python3.10 /usr/bin/python3 && \ | ||
| \ | ||
| # \ | ||
| # Install LISA dependencies \ | ||
| # \ | ||
| apt-get install -y git gcc libgirepository1.0-dev libcairo2-dev qemu-utils libvirt-dev python3-venv && \ | ||
| \ | ||
| # \ | ||
| # Create user waagent, which is used to execute the tests \ | ||
| # \ | ||
| groupadd waagent && \ | ||
| useradd --shell /bin/bash --create-home -g waagent waagent && \ | ||
| : | ||
|
|
||
| # | ||
| # Do the Poetry and LISA setup as waagent | ||
| # | ||
| USER waagent | ||
|
|
||
| RUN \ | ||
| # \ | ||
| # Install Poetry \ | ||
| # \ | ||
| curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python3 - && \ | ||
| export PATH="$HOME/.local/bin:$PATH" && \ | ||
| \ | ||
| # \ | ||
| # Install LISA \ | ||
| # \ | ||
| cd $HOME && \ | ||
| git clone https://github.com/microsoft/lisa.git && \ | ||
| cd lisa && \ | ||
| poetry config virtualenvs.in-project true && \ | ||
| make setup && \ | ||
| \ | ||
| # \ | ||
| # Install additional test dependencies \ | ||
| # \ | ||
| poetry add msrestazure && \ | ||
| \ | ||
| # \ | ||
| # The setup for the tests depends on a couple of paths; add those to the profile \ | ||
| # \ | ||
| echo 'export PYTHONPATH="$HOME/WALinuxAgent"' >> $HOME/.bash_profile && \ | ||
| echo 'export PATH="$HOME/.local/bin:$PATH"' >> $HOME/.bash_profile && \ | ||
| : | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| # Create a base class | ||
|
There was a problem hiding this comment. When I fixed the dependencies, I realized I missed this file in my previous PR. This file comes from the current Azure Pipeline and needs to be rewritten |
||
| import logging | ||
|
|
||
|
|
||
| def get_logger(name): | ||
| return LoggingHandler(name).log | ||
|
|
||
|
|
||
| class LoggingHandler: | ||
| """ | ||
| Base class for Logging | ||
| """ | ||
| def __init__(self, name=None): | ||
| self.log = self.__setup_and_get_logger(name) | ||
|
|
||
| def __setup_and_get_logger(self, name): | ||
| logger = logging.getLogger(name if name is not None else self.__class__.__name__) | ||
| if logger.hasHandlers(): | ||
| # Logging module inherits from base loggers if already setup, if a base logger found, reuse that | ||
| return logger | ||
|
|
||
| # No handlers found for logger, set it up | ||
| # This logging format is easier to read on the DevOps UI - | ||
| # https://docs.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash#formatting-commands | ||
| log_formatter = logging.Formatter("##[%(levelname)s] [%(asctime)s] [%(module)s] {%(pathname)s:%(lineno)d} %(message)s", | ||
| datefmt="%Y-%m-%dT%H:%M:%S%z") | ||
| console_handler = logging.StreamHandler() | ||
| console_handler.setFormatter(log_formatter) | ||
| logger.addHandler(console_handler) | ||
| logger.setLevel(logging.INFO) | ||
|
|
||
| return logger | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| set -euxo pipefail | ||
|
|
||
| az login --service-principal --username "$AZURE_CLIENT_ID" --password "$AZURE_CLIENT_SECRET" --tenant "$AZURE_TENANT_ID" > /dev/null | ||
|
|
||
| az acr login --name waagenttests | ||
|
|
||
| docker pull waagenttests.azurecr.io/waagenttests:latest | ||
|
|
||
| # Logs will be placed in this location. Make waagent (UID 1000 in the container) the owner. | ||
| mkdir "$HOME/logs" | ||
| sudo chown 1000 "$HOME/logs" | ||
|
|
||
| docker run --rm \ | ||
| --volume "$BUILD_SOURCESDIRECTORY:/home/waagent/WALinuxAgent" \ | ||
|
There was a problem hiding this comment. "--volume <host_path>:<container_path>" shares the local host_path with the container as container-path. we are sharing the agent's code and the the ssh key used to connect to the test VMs (both of which Azure Pipelines downloaded to the VM); also, we are specifying a location where the container will drop the logs and then the host VM will pick them up from there to report results. |
||
| --volume "$DOWNLOADSSHKEY_SECUREFILEPATH:/home/waagent/id_rsa" \ | ||
|
There was a problem hiding this comment. I don't know if it's possible, why don't we directly provide /.ssh folder in the container path that way we could avoid doing extra steps copying from one place to other when lisa script run There was a problem hiding this comment. $BUILD_SOURCESDIRECTORY and $DOWNLOADSSHKEY_SECUREFILEPATH are inputs to the container, while $HOME/logs is its output. We should not modify the input directories, since the may be coming from a developer's machine while debugging, writing a test, etc (the script that executes the tests creates a file under '/home/waagent' and changes the mode of '/home/waagent/id_rsa') There was a problem hiding this comment. I see your point. If the file semantics changes in container path would change the input path too. Makes sense. |
||
| --volume "$HOME/logs:/home/waagent/logs" \ | ||
| --env SUBSCRIPTION_ID \ | ||
| --env AZURE_CLIENT_ID \ | ||
| --env AZURE_CLIENT_SECRET \ | ||
| --env AZURE_TENANT_ID \ | ||
| waagenttests.azurecr.io/waagenttests \ | ||
| bash --login -c '~/WALinuxAgent/tests_e2e/scripts/execute_tests.sh' | ||
|
|
||
| ls -lR "$HOME/logs" | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I renamed the connection and made the actual subscription ID a parameter.