IT Tests: FIM - Refactor limits test of test report changes #2417

CamiRomero · 2021-12-28T18:33:50Z

Related issue
#2345

Description

Changes applied in this PR:

We add the auto-configuration of the test in order to not set manually the local internal options.
We changed the import in order to only import the functions that use in the test and not all the module
We changed YAML, now this file only contains one configuration, we note that when we put more than one, the test that does not apply with the configuration will be automatically skipped and this is wrong, if not applied the
test just not run. When this changes the test only runs nine times and there isn't a skipped test.
We modified callback_generater function in order to apply this function and not a specific function per test.
We update the documentation.

Important

In this PR we only work on tests:

tests/integration/test_fim/test_files/test_report_changes/test_diff_size_limit_configured.py
tests/integration/test_fim/test_files/test_report_changes/test_diff_size_limit_default.py

but we apply a small change to use other Callback in order to not affect the tests:

tests/integration/test_fim/test_registry/test_registry_report_changes/test_registry_file_size_default.py
tests/integration/test_fim/test_synchronization/test_sync_disabled_win32.py
tests/integration/test_fim/test_synchronization/test_sync_registry_disabled_win32.py
tests/integration/test_fim/test_synchronization/test_sync_registry_enabled_win32.py

Configuration options

Type	Format	Tag	File name
Agent - Windows	msi	4.3.0	wazuh-agent-4.3.0-0.40301.20211108.msi
Agent - MacOS	msi	4.3.0	wazuh-agent-4.3.0-0.40301.20211108.pkg
Agent - Centos	msi	4.3.0	wazuh-agent-4.3.0-0.40301.20211108.x86_64.rpm
Manager - Centos	msi	4.3.0	wazuh-manager-4.3.0-0.40301.20211108.x86_64.rpm

Configuration options

syscheck.debug=2
agent.debug=2
monitord.rotate_log=0
windows.debug=2

test_fim/test_files/test_report_changes/test_diff_size_limit_configured.py

Test path	Execution	OS	Type	Version	Date	Status	Executed by
test_fim/test_files/test_report_changes/test_diff_size_limit_configured.py	Local	Windows	Agent	4.3.0	10/01/2022	🟢	Camila
test_diff_size_limit_configured.py	Local	Windows	Agent	4.3.0	10/01/2022	🟢	Camila
test_diff_size_limit_configured.py	Local	Windows	Agent	4.3.0	10/01/2022	🟢	Camila

tests/integration/test_fim/test_files/test_report_changes/test_diff_size_limit_default.py

Test path	Execution	OS	Type	Version	Date	Status	Executed by
tests/integration/test_fim/test_files/test_report_changes/test_diff_size_limit_default.py	Local	Windows	Agent	4.3.0	10/01/2022	🟢	Camila
test_diff_size_limit_default.py	Local	Windows	Agent	4.3.0	10/01/2022	🟢	Camila
test_diff_size_limit_default.py	Local	Windows	Agent	4.3.0	10/01/2022	🟢	Camila

test_fim/

Test path	Execution	OS	Type	Version	Date	Status	Executed by
test_fim/	Jenkins	Windows	Agent	4.3.0	28/12/2021	🟢	Camila
test_fim/	Jenkins	Windows	Agent	4.3.0	28/12/2021	🟢	Camila
test_fim/	Jenkins	Windows	Agent	4.3.0	28/12/2021	🟢	Camila

Test path	Execution	OS	Type	Version	Date	Status	Executed by
test_fim/	Jenkins	MacOS	Agent	4.3.0	28/12/2021	🟢	Camila
test_fim/	Jenkins	MacOS	Agent	4.3.0	28/12/2021	🟢	Camila
test_fim/	Jenkins	MacOS	Agent	4.3.0	28/12/2021	🟢	Camila

Test path	Execution	OS	Type	Version	Date	Status	Executed by
test_fim/	Jenkins	Centos	Agent	4.3.0	28/12/2021	🟢	Camila
test_fim/	Jenkins	Centos	Agent	4.3.0	28/12/2021	🟢	Camila
test_fim/	Jenkins	Centos	Agent	4.3.0	28/12/2021	🟢	Camila

Test path	Execution	OS	Type	Version	Date	Status	Executed by
test_fim/	Jenkins	Centos	Manager	4.3.0	28/12/2021	🟢	Camila
test_fim/	Jenkins	Centos	Manager	4.3.0	28/12/2021	🟢	Camila
test_fim/	Jenkins	Centos	Manager	4.3.0	28/12/2021	🟢	Camila

Test path	Execution	OS	Type	Version	Date	Status	Executed by
test_fim/	Jenkins	Solaris	Manager	4.3.0	10/01/2022	🟢	Camila
test_fim/	Jenkins	Solaris	Manager	4.3.0	10/01/2022	🟢	Camila
test_fim/	Jenkins	Solaris	Manager	4.3.0	10/01/2022	🟢	Camila

Tests

Proven that tests pass when they have to pass.
Proven that tests fail when they have to fail.

Python codebase satisfies PEP-8 style style guide. pycodestyle --max-line-length=120 --show-source --show-pep8 file.py.

…ndard

Deblintrake09 · 2022-01-04T21:18:06Z

deps/wazuh_testing/wazuh_testing/fim_module/fim_variables.py

 # FIM modules
 SCHEDULE_MODE = 'scheduled'

 # Yaml Configuration
 YAML_CONF_REGISTRY_RESPONSE = 'wazuh_conf_registry_responses_win32.yaml'
 YAML_CONF_SYNC_WIN32 = 'wazuh_sync_conf_win32.yaml'
+YAML_CONF_DIFF = 'wazuh_conf_diff.yaml'


Why add config yaml files here, if they will be used only on a specific test?

Currently, this configuration is only used in one test but the idea is to reuse it in the next refactors

Deblintrake09 · 2022-01-04T21:21:09Z

tests/integration/test_fim/test_files/test_report_changes/test_diff_size_limit_configured.py

-configurations_path = os.path.join(test_data_path, 'wazuh_conf.yaml')
-testdir1 = test_directories[0]
-DIFF_LIMIT_VALUE = 2
+test_directories = [os.path.join(PREFIX, TEST_DIR_1)]


Why use test_directories as an array, if only One test path will be used? And it won't be passed as an iterable on configuration, but it's hardcoded to only use the first in the array?
Better change it to a simple string instead of an array for legibility.

Done in commit 543b1

Deblintrake09 · 2022-01-04T22:23:54Z

Test path	Execution	OS	Type	Version	Date	Status	Executed by
test_fim/	Jenkins	Windows	Agent	4.3.0	2021/01/04	🟢	@Deblintrake09
test_fim/	Jenkins	Windows	Agent	4.3.0	2021/01/05	🟢	@Deblintrake09
test_fim/	Jenkins	Windows	Agent	4.3.0	2021/01/05	🟢	@Deblintrake09
test_fim/	Jenkins	MacOs	Agent	4.3.0	2021/01/04	🟡	@Deblintrake09
test_fim/	Jenkins	MacOs	Agent	4.3.0	2021/01/05	🟡	@Deblintrake09
test_fim/	Jenkins	Centos	Agent	4.3.0	2021/01/04	🟢	@Deblintrake09
test_fim/	Jenkins	Centos	Agent	4.3.0	2021/01/05	🟢	@Deblintrake09
test_fim/	Jenkins	Centos	Agent	4.3.0	2021/01/05	🟢	@Deblintrake09
test_fim/	Jenkins	Centos	Manager	4.3.0	2021/01/04	🟢	@Deblintrake09
test_fim/	Jenkins	Centos	Manager	4.3.0	2021/01/05	🟢	@Deblintrake09
test_fim/	Jenkins	Centos	Manager	4.3.0	2021/01/05	🟢	@Deblintrake09

Macos Skipped all tests

Deblintrake09

All works correctly. Will need to run tests on local environments when we are done with Issue #2421.

…rt-changes

CamiRomero added 11 commits December 23, 2021 12:41

rm: remove callback and used a generic one

723d092

add: Add variables for FIM module

49289e5

rf: refactor test_diff_size_limit_configured according to the new sta…

8aa43e5

…ndard

rf: adapt test to new callback

08fc220

rf: refactor wazuh conf yaml

f12300b

Fix import error

fdaf830

rf: refactor test diff size limit default according to new standard

88d79a6

Fix import errors

26ef5e9

style: change code to adapt pep-8 code style

68c4330

Changed name of variables

31f6c04

Add new variables - error messages

a72d765

CamiRomero requested a review from Deblintrake09 December 28, 2021 18:34

Update Changelog

bd400ed

Deblintrake09 reviewed Jan 4, 2022

View reviewed changes

Rf: refactor variable

543b135

damarisg linked an issue Jan 10, 2022 that may be closed by this pull request

Refactor FIM limits tests of 'test_report_changes' according to new standard (2) #2345

Closed

damarisg assigned CamiRomero Jan 10, 2022

Deblintrake09 previously approved these changes Jan 10, 2022

View reviewed changes

Merge branch 'master' into 2345-refactor-fim-limits-test-of-test-repo…

24030a8

…rt-changes

CamiRomero added the subteam/qa-storm label Jan 18, 2022

damarisg changed the title ~~2345 refactor fim limits test of test report changes~~ IT Tests: FIM - Refactor limits test of test report changes Jan 23, 2022

Merge branch 'master' into 2345-refactor-fim-limits-test-of-test-repo…

2309191

…rt-changes

CamiRomero dismissed Deblintrake09’s stale review via 2309191 January 27, 2022 18:08

snaow approved these changes Jan 31, 2022

View reviewed changes

snaow approved these changes Feb 1, 2022

View reviewed changes

snaow merged commit c20d2dc into master Feb 1, 2022

snaow deleted the 2345-refactor-fim-limits-test-of-test-report-changes branch February 1, 2022 09:16

juliamagan mentioned this pull request Feb 8, 2022

Change generate_monitoring_callback for system tests #2509

Closed

juliamagan mentioned this pull request Feb 8, 2022

Change callback function used for system tests #2557

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IT Tests: FIM - Refactor limits test of test report changes #2417

IT Tests: FIM - Refactor limits test of test report changes #2417

CamiRomero commented Dec 28, 2021 •

edited

Loading

Deblintrake09 Jan 4, 2022

CamiRomero Jan 10, 2022

Deblintrake09 Jan 4, 2022

CamiRomero Jan 10, 2022

Deblintrake09 commented Jan 4, 2022 •

edited

Loading

Deblintrake09 left a comment

IT Tests: FIM - Refactor limits test of test report changes #2417

IT Tests: FIM - Refactor limits test of test report changes #2417

Conversation

CamiRomero commented Dec 28, 2021 • edited Loading

Description

Configuration options

Configuration options

test_fim/test_files/test_report_changes/test_diff_size_limit_configured.py

tests/integration/test_fim/test_files/test_report_changes/test_diff_size_limit_default.py

test_fim/

Tests

Deblintrake09 Jan 4, 2022

Choose a reason for hiding this comment

CamiRomero Jan 10, 2022

Choose a reason for hiding this comment

Deblintrake09 Jan 4, 2022

Choose a reason for hiding this comment

CamiRomero Jan 10, 2022

Choose a reason for hiding this comment

Deblintrake09 commented Jan 4, 2022 • edited Loading

Deblintrake09 left a comment

Choose a reason for hiding this comment

CamiRomero commented Dec 28, 2021 •

edited

Loading

Deblintrake09 commented Jan 4, 2022 •

edited

Loading