Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix integrity sync test and config to avoid flaky behavior #4406

Merged
merged 2 commits into from
Aug 14, 2023

Conversation

EduLeon12
Copy link
Contributor

@EduLeon12 EduLeon12 commented Aug 8, 2023

Related issue
Closes #4368

Description

Added

As explained in #4368 to fix the integrity sync test a change in the test had to be made to ensure the timeout occurs and expected and the cancel_task signals were properly received.

Updated

  • tests/system/test_cluster/test_integrity_sync/test_integrity_sync.py : changed number of files created from 5 to 10
  • tests/system/test_cluster/test_integrity_sync/data/cluster_json.yml: changed max zip size config from 50 MB to 100 MB.

Testing performed

Failing System test were ran multiple times to confirm behavior.

(system-test-env) eduardoleon@pop-os:~/git/wazuh-qa/tests/system/test_cluster$ pytest test_integrity_sync/
==================================== test session starts =====================================
platform linux -- Python 3.9.16, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/eduardoleon/git/wazuh-qa/tests/system, configfile: pytest.ini
plugins: testinfra-5.0.0, metadata-2.0.4, html-3.1.1
collected 1 item

test_integrity_sync/test_integrity_sync.py .                                           [100%]

=============================== 1 passed in 244.11s (0:04:04) ================================
(system-test-env) eduardoleon@pop-os:~/git/wazuh-qa/tests/system/test_cluster$ pytest test_integrity_sync/
==================================== test session starts =====================================
platform linux -- Python 3.9.16, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/eduardoleon/git/wazuh-qa/tests/system, configfile: pytest.ini
plugins: testinfra-5.0.0, metadata-2.0.4, html-3.1.1
collected 1 item

test_integrity_sync/test_integrity_sync.py .                                           [100%]

=============================== 1 passed in 260.82s (0:04:20) ================================
(system-test-env) eduardoleon@pop-os:~/git/wazuh-qa/tests/system/test_cluster$ pytest test_integrity_sync/
==================================== test session starts =====================================
platform linux -- Python 3.9.16, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/eduardoleon/git/wazuh-qa/tests/system, configfile: pytest.ini
plugins: testinfra-5.0.0, metadata-2.0.4, html-3.1.1
collected 1 item

test_integrity_sync/test_integrity_sync.py .                                           [100%]

=============================== 1 passed in 250.99s (0:04:10) ================================

@EduLeon12 EduLeon12 self-assigned this Aug 8, 2023
@EduLeon12 EduLeon12 force-pushed the fix/4368-fix-integrity-sync-system-test branch 3 times, most recently from 5e2e127 to 856b892 Compare August 8, 2023 12:37
EduLeon12 pushed a commit that referenced this pull request Aug 8, 2023
@EduLeon12 EduLeon12 force-pushed the fix/4368-fix-integrity-sync-system-test branch from 856b892 to c899393 Compare August 8, 2023 12:41
@EduLeon12 EduLeon12 changed the title Increased max zip size limit and number of files created for integrit… Fix integrity sync test and config to avoid flaky behavior Aug 8, 2023
@EduLeon12 EduLeon12 linked an issue Aug 8, 2023 that may be closed by this pull request
Copy link
Contributor

@Selutario Selutario left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM, the test still works in my env after the changes (unrelated tests were commented out to speed up the process):

$ pytest -vv test_integrity_sync
============================================================================================ test session starts =============================================================================================
platform linux -- Python 3.10.12, pytest-7.1.2, pluggy-1.0.0 -- /home/selu/venv/wazuh-qa/bin/python
cachedir: .pytest_cache
metadata: {'Python': '3.10.12', 'Platform': 'Linux-5.15.0-78-generic-x86_64-with-glibc2.35', 'Packages': {'pytest': '7.1.2', 'pluggy': '1.0.0'}, 'Plugins': {'html': '3.1.1', 'testinfra': '5.0.0', 'metadata': '2.0.4'}}
rootdir: /home/selu/Git/wazuh-qa/tests/system, configfile: pytest.ini
plugins: html-3.1.1, testinfra-5.0.0, metadata-2.0.4
collected 1 item                                                                                                                                                                                             

test_integrity_sync/test_integrity_sync.py::test_zip_size_limit PASSED                                                                                                                                 [100%]

======================================================================================= 1 passed in 364.79s (0:06:04) ========================================================================================

However, updating the changelog is required.

@EduLeon12 EduLeon12 force-pushed the fix/4368-fix-integrity-sync-system-test branch from f4a951c to 6563b6a Compare August 9, 2023 10:38
@EduLeon12 EduLeon12 force-pushed the fix/4368-fix-integrity-sync-system-test branch from 592a044 to 749fb46 Compare August 9, 2023 10:41
@Rebits
Copy link
Member

Rebits commented Aug 14, 2023

LGTM

Report: 4406.zip

==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-0.13.1
rootdir: /home/rebits/Wazuh/system_testing_2/wazuh-qa/tests/system, configfile: pytest.ini
plugins: html-3.1.1, testinfra-5.0.0, json-report-1.5.0, metadata-2.0.1, asyncio-0.20.3, docgen-1.3.0
asyncio: mode=strict
collected 4 items                                                                                                                                                                            

test_cluster/test_integrity_sync/test_integrity_sync.py ....                                                                                                                           [100%]

----------------------------------------------- generated html file: file:///home/rebits/Wazuh/system_testing_2/wazuh-qa/tests/system/r1.html ------------------------------------------------
=============================================================================== 4 passed in 821.80s (0:13:41) ================================================================================

@Rebits Rebits merged commit c83e1c2 into 4.6.0 Aug 14, 2023
@Rebits Rebits deleted the fix/4368-fix-integrity-sync-system-test branch August 14, 2023 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrity sync test flaky behavior
3 participants