Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installation assistant test and tier workflow migration #46

Merged
merged 48 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
c817edd
Changed automation reference input parameter
davidcr01 Aug 27, 2024
aadf487
Fixed runs-on parameter
davidcr01 Aug 27, 2024
058150a
Added view parameters step
davidcr01 Aug 27, 2024
b719d82
Added composite name variable set
davidcr01 Aug 27, 2024
1d84840
Added Ansible installation step
davidcr01 Aug 27, 2024
f2f10bd
Added AWS credentials setup and automation checkout steps
davidcr01 Aug 27, 2024
7321b14
Added allocator instance provision
davidcr01 Aug 27, 2024
81033b9
Added allocator instance deletion
davidcr01 Aug 27, 2024
549f9b3
Added provision playbook execution
davidcr01 Aug 28, 2024
509a003
Changed boolean variables in provision playbook
davidcr01 Aug 28, 2024
d8e5c9b
Removed ansible_pkg_mgr variable in provision playbook
davidcr01 Aug 28, 2024
249f9a3
Removed DNF block
davidcr01 Aug 28, 2024
622e328
Changed apt/yum for package
davidcr01 Aug 28, 2024
cc55350
Adding modules directly
davidcr01 Aug 28, 2024
a338d17
Adding ansible dependencies
davidcr01 Aug 28, 2024
2be9970
Debugging ansible version
davidcr01 Aug 28, 2024
1e37109
Changing YUM to DNF
davidcr01 Aug 28, 2024
d4c26b0
Using package module
davidcr01 Aug 28, 2024
08b193e
Installing Ansible with pip
davidcr01 Aug 28, 2024
eae9a6c
Adding Python install in CentOS 8
davidcr01 Aug 29, 2024
529489b
Moved variables to env variables
davidcr01 Aug 29, 2024
b701e90
Moved variables to env variables
davidcr01 Aug 29, 2024
5ac37db
Added AIO installation playbook
davidcr01 Aug 30, 2024
e3126bb
Added assistant pytest playbook
davidcr01 Aug 30, 2024
38ac911
Replaced debug input by verbosity input
davidcr01 Aug 30, 2024
9455831
Allocator VM is deleted conditionally
davidcr01 Sep 2, 2024
128845d
Deleted conditionals in Debian pip3 installation
davidcr01 Sep 2, 2024
4bbb703
Added async and poll parameters to AIO task
davidcr01 Sep 2, 2024
43fb084
Added allocator instance directory upload
davidcr01 Sep 2, 2024
dcc5de6
Deleted debug tasks and improved task names
davidcr01 Sep 2, 2024
898f53d
Python 3.9 is only installed in Ubuntu 22
davidcr01 Sep 3, 2024
3484f59
Installed Python3.6 in Ubuntu Xenial at early stage
davidcr01 Sep 3, 2024
17b124f
Fixed pip installation in Ubuntu Xenial
davidcr01 Sep 3, 2024
9c8b5ca
Added python3-apt installation in Ubuntu Xenial
davidcr01 Sep 3, 2024
60d2c28
Fixed Python install in Xenial
davidcr01 Sep 3, 2024
00cf2ca
Revert testing changes
davidcr01 Sep 3, 2024
7f37280
Added matrix strategy to implement tier
davidcr01 Sep 4, 2024
6188513
Added assistant reference to form
davidcr01 Sep 5, 2024
40e80c3
Added runID to the display name
davidcr01 Sep 5, 2024
f51e9db
Artifacts have unique name
davidcr01 Sep 5, 2024
d4afdaf
Deleted tier workflow
davidcr01 Sep 5, 2024
1d46ba6
Removed logs save logic
davidcr01 Sep 16, 2024
8c43c11
Changed repository reference to the workflow branch
davidcr01 Sep 16, 2024
e6bda9c
Changed default OSs
davidcr01 Sep 16, 2024
daaf1fc
Updated CHANGELOG for #46
davidcr01 Sep 17, 2024
faa1d25
Changed references of unattended
davidcr01 Sep 17, 2024
959d1e5
Merge branch '4.10.0' into enhancement/20-assisntant-workflow-develop…
davidcr01 Sep 17, 2024
c5a7b50
Changed order in CHANGELOG
davidcr01 Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
194 changes: 169 additions & 25 deletions .github/workflows/Test_installation_assistant.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
run-name: Test installation assistant - System ${{ inputs.SYSTEM }} - Launched by @${{ github.actor }}
run-name: Test installation assistant - ${{ github.run_id }} - ${{ inputs.SYSTEMS }} - Launched by @${{ github.actor }}
name: Test installation assistant

on:
Expand All @@ -21,42 +21,186 @@ on:
- staging
- pre-release
AUTOMATION_REFERENCE:
description: 'wazuh-automation reference'
description: 'Branch or tag of the wazuh-automation repository'
required: true
default: 'v4.10.0'
SYSTEM:
description: 'Operating System'
default: '4.10.0'
SYSTEMS:
description: 'Operating Systems (list of comma-separated quoted strings enclosed in square brackets)'
required: true
default: 'CentOS 8'
default: '["CentOS_8", "AmazonLinux_2", "Ubuntu_22", "RHEL8"]'
type: string
VERBOSITY:
description: 'Verbosity level on playbooks execution'
required: true
default: '-v'
type: choice
options:
- CentOS 7
- CentOS 8
- Amazon Linux 2
- Ubuntu 16
- Ubuntu 18
- Ubuntu 20
- Ubuntu 22
- RHEL7
- RHEL8
DEBUG:
description: 'Debug mode'
required: true
default: false
type: boolean
- -v
- -vv
- -vvv
- -vvvv
DESTROY:
description: 'Destroy instances after run'
required: true
default: true
type: boolean

env:
LABEL: ubuntu-latest
COMPOSITE_NAME: "linux-SUBNAME-amd64"
SESSION_NAME: "Installation-Assistant-Test"
REGION: "us-east-1"
TMP_PATH: "/tmp/test"
LOGS_PATH: "${{ github.workspace }}/assistant_logs"
PKG_REPOSITORY: "${{ inputs.REPOSITORY }}"
TEST_NAME: "test_assistant"
REPOSITORY_URL: "${{ github.server_url }}/${{ github.repository }}.git"
ALLOCATOR_PATH: "/tmp/allocator_instance"

permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout

jobs:
initialize-environment:
runs-on: $LABEL
run-test:
runs-on: ubuntu-latest
strategy:
fail-fast: false # If a job fails, the rest of jobs will not be canceled
matrix:
system: ${{ fromJson(inputs.SYSTEMS) }}

steps:
- name: Set up Git
uses: actions/checkout@v3
- name: Checkout code
uses: actions/checkout@v4

- name: View parameters
run: echo "${{ toJson(inputs) }}"

- name: Set COMPOSITE_NAME variable
run: |
case "${{ matrix.system }}" in
"CentOS_7")
SUBNAME="centos-7"
;;
"CentOS_8")
SUBNAME="centos-8"
;;
"AmazonLinux_2")
SUBNAME="amazon-2"
;;
"Ubuntu_16")
SUBNAME="ubuntu-16.04"
;;
"Ubuntu_18")
SUBNAME="ubuntu-18.04"
;;
"Ubuntu_20")
SUBNAME="ubuntu-20.04"
;;
"Ubuntu_22")
SUBNAME="ubuntu-22.04"
;;
"RHEL7")
SUBNAME="redhat-7"
;;
"RHEL8")
SUBNAME="redhat-8"
;;
*)
echo "Invalid SYSTEM selection" >&2
exit 1
;;
esac
COMPOSITE_NAME="${COMPOSITE_NAME/SUBNAME/$SUBNAME}"
echo "COMPOSITE_NAME=$COMPOSITE_NAME" >> $GITHUB_ENV

- name: Install Ansible
run: sudo apt-get update && sudo apt install -y python3 && python3 -m pip install --user ansible-core==2.16

- name: Set up AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_IAM_ROLE }}
role-session-name: ${{ env.SESSION_NAME }}
aws-region: ${{ env.REGION }}

- name: Checkout wazuh/wazuh-automation repository
uses: actions/checkout@v4
with:
repository: wazuh/wazuh-automation
ref: ${{ inputs.AUTOMATION_REFERENCE }}
token: ${{ secrets.GH_CLONE_TOKEN }}
path: wazuh-automation

- name: Install and set allocator requirements
run: pip3 install -r wazuh-automation/deployability/deps/requirements.txt

- name: Allocate instance test and set SSH variables
id: allocator_instance
run: |
python3 wazuh-automation/deployability/modules/allocation/main.py --action create --provider aws --size large --composite-name ${{ env.COMPOSITE_NAME }} --working-dir $ALLOCATOR_PATH \
--track-output $ALLOCATOR_PATH/track.yml --inventory-output $ALLOCATOR_PATH/inventory.yml --instance-name gha_${{ github.run_id }}_assistant_test \
--label-team devops --label-termination-date 1d

sed 's/: */=/g' $ALLOCATOR_PATH/inventory.yml > $ALLOCATOR_PATH/inventory_mod.yml
sed -i 's/-o StrictHostKeyChecking=no/\"-o StrictHostKeyChecking=no\"/g' $ALLOCATOR_PATH/inventory_mod.yml
source $ALLOCATOR_PATH/inventory_mod.yml

echo "[gha_instance]" > $ALLOCATOR_PATH/inventory
echo "$ansible_host ansible_port=$ansible_port ansible_user=$ansible_user ansible_ssh_private_key_file=$ansible_ssh_private_key_file ansible_ssh_common_args='$ansible_ssh_common_args'" >> $ALLOCATOR_PATH/inventory

- name: Execute provision playbook
run: |
INSTALL_DEPS=true
INSTALL_PYTHON=true
INSTALL_PIP_DEPS=true

ansible-playbook .github/workflows/ansible-playbooks/provision.yml \
-i $ALLOCATOR_PATH/inventory \
-l all \
-e "repository=$REPOSITORY_URL" \
-e "reference=${{ github.ref_name }}" \
-e "tmp_path=$TMP_PATH" \
-e "pkg_repository=$PKG_REPOSITORY" \
-e "install_deps=$INSTALL_DEPS" \
-e "install_python=$INSTALL_PYTHON" \
-e "install_pip_deps=$INSTALL_PIP_DEPS" \
"${{ inputs.VERBOSITY }}"

- name: Execute AIO installation playbook
run: |
ansible-playbook .github/workflows/ansible-playbooks/aio.yml \
-i $ALLOCATOR_PATH/inventory \
-l all \
-e "tmp_path=$TMP_PATH" \
-e "logs_path=$LOGS_PATH" \
-e "test_name=$TEST_NAME" \
"${{ inputs.VERBOSITY }}"

- name: Execute Python test playbook
run: |
TEST_NAME="test_installation_assistant"
ansible-playbook .github/workflows/ansible-playbooks/aio_tests.yml \
-i $ALLOCATOR_PATH/inventory \
-l all \
-e "tmp_path=$TMP_PATH" \
-e "logs_path=$LOGS_PATH" \
-e "test_name=$TEST_NAME" \
"${{ inputs.VERBOSITY }}"

- name: Compress Allocator VM directory
id: compress_allocator_files
if: always() && steps.allocator_instance.outcome == 'success' && inputs.DESTROY == false
run: |
zip -P "${{ secrets.ZIP_ARTIFACTS_PASSWORD }}" -r $ALLOCATOR_PATH.zip $ALLOCATOR_PATH

- name: Upload Allocator VM directory as artifact
if: always() && steps.compress_allocator_files.outcome == 'success' && inputs.DESTROY == false
uses: actions/upload-artifact@v4
with:
name: allocator-instance-${{ matrix.system }}
path: ${{ env.ALLOCATOR_PATH }}.zip

- name: Delete allocated VM
if: always() && steps.allocator_instance.outcome == 'success' && inputs.DESTROY == true
run: python3 wazuh-automation/deployability/modules/allocation/main.py --action delete --track-output $ALLOCATOR_PATH/track.yml

84 changes: 0 additions & 84 deletions .github/workflows/Test_installation_assistant_tier.yml

This file was deleted.

16 changes: 16 additions & 0 deletions .github/workflows/ansible-playbooks/aio.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@

- hosts: all
become: true

vars:
script_path: "{{ tmp_path }}"
script_name: "wazuh-install.sh"

tasks:
- name: Test assistant AIO install
command: "bash {{ script_name }} -a -v"
args:
chdir: "{{ script_path }}"
register: install_results
async: 500
poll: 5
14 changes: 14 additions & 0 deletions .github/workflows/ansible-playbooks/aio_tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@

- hosts: all
become: true

vars:
script_path: "{{ tmp_path }}/tests/install"
script_name: "{{ test_name }}.py"

tasks:
- name: Test unattended AIO install
teddytpc1 marked this conversation as resolved.
Show resolved Hide resolved
command: "python3 -m pytest --tb=long {{ script_name }} -v -m \"wazuh or wazuh_worker or indexer or dashboard\""
args:
chdir: "{{ script_path }}"
register: test_results
Loading