Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test documentation validation when running qa-ctl in automatic mode #2158

Merged
merged 3 commits into from
Nov 2, 2021

Conversation

jmv74211
Copy link
Contributor

@jmv74211 jmv74211 commented Nov 2, 2021

Related issue
close #2023

Description

The aim of this PR is to validate if the specified tests in the qa-ctl run are documented or not according to the current qa-docs schema. This is a necessary validation before proceeding with the execution of the tests automatically.

This PR makes the following changes:

  • Add new parameter in qa-docs to check the test documentation
  • Validate if the tests are well documented before running the qa-ctl build.

Checks

Test does not exist
qa-ctl -r test_x --qa-branch  2023-qa-ctl-doc-validation --dry-run
2021-11-02 12:42:58,263 - INFO - Validating input parameters
2021-11-02 12:42:59,718 - ERROR - test_x does not exist in /tmp/wazuh_qa_ctl/wazuh-qa/tests
wazuh_testing.tools.exceptions.QAValueError: test_x does not exist in /tmp/wazuh_qa_ctl/wazuh-qa/tests

Test is not documented
qa-ctl -r test_general_settings_enabled --qa-branch  2023-qa-ctl-doc-validation --dry-run
2021-11-02 12:30:46,038 - INFO - Validating input parameters
2021-11-02 12:30:47,700 - INFO - Input parameters validation has passed successfully
2021-11-02 12:30:48,333 - INFO - Run as dry-run mode. Configuration file saved in /tmp/wazuh_qa_ctl/config_1635852647.700738.yaml
Check with multiple tests
qa-ctl -r test_general_settings_enabled test_ambiguous_complex --qa-branch  2023-qa-ctl-doc-validation --dry-run
2021-11-02 12:40:32,084 - INFO - Validating input parameters
2021-11-02 12:40:35,049 - ERROR - test_ambiguous_complex is not documented using qa-docs current schema
wazuh_testing.tools.exceptions.QAValueError: test_ambiguous_complex is not documented using qa-docs current schema

@jmv74211 jmv74211 self-assigned this Nov 2, 2021
@jmv74211 jmv74211 merged commit 337738c into 2021-qa-ctl-v2 Nov 2, 2021
@jmv74211 jmv74211 deleted the 2023-qa-ctl-doc-validation branch November 2, 2021 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants