Change OVA system due to EOL #1575
Assignees
Labels
Comments
rauldpm
changed the title
Update ReportResearchFirst, it is necessary to list all the available or possible operative systems to adapt the OVA:
After talking with the team, Ubuntu systems, and RHEL systems seems to be not good for the OVA as they tend to have problems or complex systems to deploy the OVA. With this, it seems that currently, the best option to deploy the OVA is in Amazon Linux 2. It would not have many problems due to it is RPM-based, as CentOS 7 is. DevelopmentAs a first approach, the following workaround is to change the OS specified in the Vagrantfile, changing The provision.sh 🟢
postProvision.sh 🔴This stage fails because it tries to remove the `/root/anaconda-ks.cfg` and `/root/original-ks.cfg` files that do not exist in Amazon Linux 2. These files are the kickstart files installation. Deleting the related lines no errors are generated.After importing the OVA, the system crashes when login into it. This is an unexpected behavior that must be investigated and solved. This problem does not occur if the OVA is generated by using CentOS 7 as the system base. |
Update ReportDevelopment and changesA summary of the realized changes are:
TestingThe system crash was not related to the OVA itself, it seems that my machine was having problems with the import of the OVA in VirtualBox, but another member of the team was able to generate and import the OVA successfully in VirtualBox. ✔️ The generation of the OVA finished successfully. The complete log is: Show logThe Wazuh logo is displayed correctly after login into the VM. Show logo
VMWare Wazuh indexer status
Wazuh manager status
Filebeat status
Wazuh dashboard status
Wazuh app working
VirtualBox Wazuh indexer status
Wazuh manager status
Filebeat
Wazuh dashboard status
Wazuh app working
|
Update ReportMeetingAfter talking with the team and discussing the current progress, we thought it is not a good idea to use a foreign Vagrant box to perform this task. Generally, it is not recommended to use third-party software that is not maintainable or unofficial. Due to this reasons, two alternatives are available using the official image of Amazon Linux 2 :
|
Update Report - OVA and BOX generationAfter talking with the team about the alternatives, we conclude that the best option is to create the Vagrant box from the VM. Instead of uploading it to the Vagrant cloud, we can store it in S3. Here is an example: The steps to follow this process is:
The process is described in this documentation: https://docs.aws.amazon.com/en_us/AWSEC2/latest/UserGuide/amazon-linux-2-virtual-machine.html Creating the VMAmazon officially provides some [virtual disks ] (https://cdn.amazonlinux.com/os-images/2.0.20230307.0/) of Amazon Linux 2. The steps are:
The The
Install the GuestAdditionsGuestAdditions are mandatory in the Vagrant configuration. It allows some features such as shared folders. The steps to perform this task are:
In these steps, some warnings can be displayed. Clean the VMWhen we use a Vagrant box, it should be as clean as possible, without history, ssh keys, logs, and unnecessary packages. The clean-up commands are: Create the Vagrant boxIn the host machine, execute the following commands: This will generate the Amazon Linux 2 Vagrant box in the current path. Create the OVAWith the box generated, the OVA can be generated easily by changing the vagrant base box that the builder script uses. With this change, the OVA is generated successfully and works as expected. Display screenshot
Upload to S3The Vagrant box and the OVA have been uploaded to S3. The files are stored in https://packages-dev.wazuh.com/ provisionally, in the folder |
Update ReportAfter talking with the team about the current progress, we decided to perform some changes in the process. Next stepsThe next steps are to re-create the Vagrant box with the mentioned configuration and create the AMI from that box. |
Update ReportDevelopmentTo perform the steps given above, I will follow the steps explained in the documentation above. With this, the And the rest of the file would be the same. As the With this, we have created a Vagrant box:
|
Update ReportDevelopmentProblem with the OVA functionality❌ With the new Vagrant box, the generation of the OVA finished successfully without executing the
To investigate these errors, some tests have been done.
It seems that some of the steps performed in the postProvision stage are necessary for the correct functionality of the Wazuh installation. This steps are: The part of removing everything related to Vagrant is not necessary anymore, as the created Vagrant box does not have anything related to Vagrant. Hence, the result steps of the postProvision stage are: ✔️ With these steps added to the Problem with the sync foldersDue to the pre-configuration of the Vagrant box (can not be accessed by the SSH configuration of Vagrant as it does not have the In the Vagrantfile, the connection with the VM is configured via password. When Vagrant executes the sync commands, the following output is generated: ❌ With this, the script is stopped waiting for the password, and this may produce problems in the automatic scripts to generate the OVA. Some tests have been done to insert the password automatically, with no success:
✔️ The only alternative that works and avoids creating the With this, the access to the Vagrant machine is done via password, but it inserts the insecure Vagrant key into it. This change makes Vagrant not to ask for the password to sync the folders. In the This file only contains the insecure key of Vagrant, so it can be performed safely.
|
Update ReportAMI baseOnce the OVA and the Vagrant box are created and uploaded to S3 (packages-dev.wazuh.com), we can create the related AMI. To perform this, is necessary to have the AWS account configured in the system, and the AWS CLI installed. To generate the AMI, I executed the following command: Where To check the status of the AMI generation, I used the following command: And the previous command returns the following content: Once the AMI is generated, its information can be consulted in the AWS console, and launch an instance using the generated AMI. The OVA is created by default in
The instance has been created with:
These features are specified in https://github.com/wazuh/wazuh-jenkins/blob/master/src/org/wazuh/TFInstance.groovy and https://github.com/wazuh/wazuh-jenkins/blob/master/jenkins-files/packages/Packages_builder_OVA.groovy |
Update ReportAMI workaroundOnce the AMI is created, it's necessary to perform some steps to clean up the AMI with the configuration that Amazon adds to it.
After this, a new AMI will be generated, and this AMI will be used to build the OVA through the automatic process. AMI developmentWith this, the commands that have to be executed in the AMI are: After this, in AWS console, I clicked on Actions -> Images and Templates -> Create image, add it a name (Amazon-Linux2-for-OVA-wp1575) and a description (AMI created from AL2_OVA_base_wp1575 after clean up). |
Update ReportI had to rebuild the Vagrant box and the OVA due to they did not have the Besides, it is necessary to rebuild the AMIs. Summarizing, repeat the process.
|
Update ReportAfter all the steps mentioned above, a strange behavior has been found in the Packages_builder_OVA pipeline. It seems that the The error is: But, it has been proved that this script is being executed. If an error is produced in the Is necessary to investigate this behavior and finish the OVA generation development. |
Update reportI was working on the tests and I was able to validate that the ova is built correctly locally, I am working on the Jenkins build process
|
Update reportI was adapting the branches pointing to master since the destination of this development was changed to 4.6.0, for this reason I had to generate new packages in staging, to be able to build the OVA |
Update reportI am debugging an error when trying to use the provision.sh script, at the moment exists in the path where it is searched for but I cannot find why it is failing, I keep validating options. 16:24:54 TASK [Clean history] ***********************************************************
16:24:54 task path: /home/ec2-user/workspace/Packages_Builder_OVA/ansible-playbooks/wazuh_ova_generation.yml:34
16:24:54 changed: [Packages_Builder_OVA_B235_20230703192249] => {
16:24:54 "changed": true,
16:24:54 "cmd": "ls -la \"/var/provision/wazuh-packages/ova\"",
16:24:54 "delta": "0:00:00.003868",
16:24:54 "end": "2023-07-03 19:24:53.993063",
16:24:54 "invocation": {
16:24:54 "module_args": {
16:24:54 "_raw_params": "ls -la \"/var/provision/wazuh-packages/ova\"",
16:24:54 "_uses_shell": true,
16:24:54 "argv": null,
16:24:54 "chdir": null,
16:24:54 "creates": null,
16:24:54 "executable": null,
16:24:54 "removes": null,
16:24:54 "stdin": null,
16:24:54 "stdin_add_newline": true,
16:24:54 "strip_empty_ends": true,
16:24:54 "warn": true
16:24:54 }
16:24:54 },
16:24:54 "rc": 0,
16:24:54 "start": "2023-07-03 19:24:53.989195"
16:24:54 }
16:24:54
16:24:54 STDOUT:
16:24:54
16:24:54 total 44
16:24:54 drwxr-xr-x 3 root root 185 Jul 3 19:24 .
16:24:54 drwxr-xr-x 22 root root 4096 Jul 3 19:24 ..
16:24:54 drwxr-xr-x 3 root root 36 Jul 3 19:24 assets
16:24:54 -rwxr-xr-x 1 root root 6630 Jul 3 19:24 generate_ova.sh
16:24:54 -rw-r--r-- 1 root root 27 Jul 3 19:24 .gitignore
16:24:54 -rwxr-xr-x 1 root root 2020 Jul 3 19:24 Ova2Ovf.py
16:24:54 -rwxr-xr-x 1 root root 1109 Jul 3 19:24 provision.sh
16:24:54 -rw-r--r-- 1 root root 1205 Jul 3 19:24 README.md
16:24:54 -rwxr-xr-x 1 root root 1480 Jul 3 19:24 setOVADefault.sh
16:24:54 -rwxr-xr-x 1 root root 756 Jul 3 19:24 Vagrantfile
16:24:54 -rw-r--r-- 1 root root 5543 Jul 3 19:24 wazuh_ovf_template16:33:20 TASK [Run provision script] ****************************************************
16:33:20 task path: /home/ec2-user/workspace/Packages_Builder_OVA/ansible-playbooks/wazuh_ova_generation.yml:37
16:33:20 fatal: [Packages_Builder_OVA_B235_20230703192249]: FAILED! => {
16:33:20 "changed": true,
16:33:20 "cmd": [
16:33:20 "sh",
16:33:20 "provision.sh",
16:33:20 "staging",
16:33:20 "yes"
16:33:20 ],
16:33:20 "delta": "0:00:00.002836",
16:33:20 "end": "2023-07-03 19:33:20.346639",
16:33:20 "invocation": {
16:33:20 "module_args": {
16:33:20 "_raw_params": "sh provision.sh staging yes",
16:33:20 "_uses_shell": false,
16:33:20 "argv": null,
16:33:20 "chdir": "/var/provision/wazuh-packages/ova",
16:33:20 "creates": null,
16:33:20 "executable": null,
16:33:20 "removes": null,
16:33:20 "stdin": null,
16:33:20 "stdin_add_newline": true,
16:33:20 "strip_empty_ends": true,
16:33:20 "warn": true
16:33:20 }
16:33:20 },
16:33:20 "rc": 127,
16:33:20 "start": "2023-07-03 19:33:20.343803"
16:33:20 }
16:33:20
16:33:20 STDERR:
16:33:20
16:33:20 sh: provision.sh: No such file or directory |
Update report:I found that the error is possibly occurring in the shutdown of the instance that occurs in the "clean" stage in the steps.sh script that is part of provision.sh. Removing this step, the construction of the OVA is successful, although I have encountered some performance problems, which I am investigating if they are due to this. https://ci.wazuh.info/view/Packages/job/Packages_Builder_OVA/246/console The error that occurs is that after starting the OVA, the virtual machine stops responding, it freezes, I'm investigating what could be the reason |
|
On Hold by release protocol |
Update reportAdapt the branches pointing to master in both wazuh-packages and wazuh-jenkins, test the creation of the OVAs and both locally and through the pipeline, the OVAs are built correctly. I find an error when running it on Virtualbox, after a moment the Virtualbox terminal is frozen. The VM continues to work, Wazuh dashboard as per ssh connection, I don't know if this is due to the version of Virtualbox I have, I'm going to request that someone else test it locally. This does not happen when running the OVA in VMware Player, the OVA works correctly here
|
|
The modifications made for the change of the operating system of the OVA on branch 4.4.5 were applied, a creation test was carried out and it finished correctly: https://ci.wazuh.info/job/Packages_Builder_OVA/264/console It remains to carry out a test on the OVA generated to verify that all the Wazuh functionalities have been installed correctly |
1 task
|
The changes made on the https://ci.wazuh.info/job/Packages_Builder_OVA/266/console All possible causes were analyzed (memory, VirtualBox version, CPU, network, etc) and it was found that the error is generated when the OVA VM is started in Virtualbox using the
The video driver was modified for
After solving this problem, we proceeded to verify that the Wazuh stack has been deployed correctly and that FIPS is enabled on the server:
|
30 tasks
7 tasks
Update4.4.5 OVA testing done in wazuh/wazuh#18115 |
|
Testing has finished. The PR wazuh/wazuh-documentation#6287 will be merged as part of wazuh/wazuh#18190. |
This was referenced Aug 22, 2023
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It is necessary to research and choose a new operating system to use in the OVA package once CentOS 7 reaches its EOL, so that we can make the pertinent changes and carry out the necessary testing.
Currently CentOS 7 EOL is set for June 30, 2024.
Regards, Raúl.
Resolution (edit)
Research
The research of this issue is in #1575 (comment).
Extra configuration
FIPS mode should be enabled for the OVA following this documentation:
https://aws.amazon.com/blogs/publicsector/enabling-fips-mode-amazon-linux-2/
This was manually tested with an EC2 instance with Amazon Linux 2. FIPS mode was configured and Wazuh installed with the assistant. Everything works fine.
Testing
The testing of this issue is in #1575 (comment).
Generation of OVA
The generation of the OVA is described in #1575 (comment). It includes the generation of the base Vagrant box for the OVA.
The text was updated successfully, but these errors were encountered: