Custom qa-docs logger use
#1896
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 13 commits
September 14, 2021 13:12
11 tasks
mdengra
reviewed
Sep 15, 2021
|
|
||
| try: | ||
| DocGenerator.LOGGER.debug(f"Writing {doc_path}.json") | ||
| with open(doc_path + ".json", "w+") as outfile: |
There was a problem hiding this comment.
Perhaps out_file would be better (improve readability to separate the words that form the variable name with underscores).
Suggested change
| with open(doc_path + ".json", "w+") as outfile: | |
| with open(doc_path + ".json", "w+") as out_file: |
mdengra
reviewed
Sep 15, 2021
| for regex in self.ignore_regex: | ||
| if regex.match(file): | ||
| DocGenerator.LOGGER.warning(f"File validation: {regex} not matchin with {file}") |
There was a problem hiding this comment.
Suggested change
| DocGenerator.LOGGER.warning(f"File validation: {regex} not matchin with {file}") | |
| DocGenerator.LOGGER.warning(f"File validation: {regex} not matching with {file}") |
mdengra
reviewed
Sep 15, 2021
| for regex in self.include_regex: | ||
| if regex.match(file): | ||
| DocGenerator.LOGGER.warning(f"File validation: {regex} not matchin with {file}") |
There was a problem hiding this comment.
Suggested change
| DocGenerator.LOGGER.warning(f"File validation: {regex} not matchin with {file}") | |
| DocGenerator.LOGGER.warning(f"File validation: {regex} not matching with {file}") |
mdengra
reviewed
Sep 15, 2021
| @@ -44,7 +46,9 @@ def is_valid_folder(self, path): | |||
| """ | |||
| for regex in self.ignore_regex: | |||
| if regex.match(path): | |||
| DocGenerator.LOGGER.warning(f"Folder validation: {regex} not matchin with {path}") | |||
There was a problem hiding this comment.
Suggested change
| DocGenerator.LOGGER.warning(f"Folder validation: {regex} not matchin with {path}") | |
| DocGenerator.LOGGER.warning(f"Folder validation: {regex} not matching with {path}") |
mdengra
reviewed
Sep 15, 2021
|
|
||
| try: | ||
| DocGenerator.LOGGER.debug(f"Writing {doc_path}.yaml") | ||
| with open(doc_path + ".yaml", "w+") as outfile: |
mdengra
reviewed
Sep 15, 2021
| Error: {inst}", stacklevel=2) | ||
| logging.warning(f"Failed to parse comment of function '{function.name}'' from module {self.scan_file}. \ | ||
| Error: {inst}") | ||
| CodeParser.LOGGER.warning(f"Failed to parse comment of function {function.name} " |
There was a problem hiding this comment.
As the comment that is parsed is the documentation, how about this:
Suggested change
| CodeParser.LOGGER.warning(f"Failed to parse comment of function {function.name} " | |
| CodeParser.LOGGER.warning(f"Failed to parse test documentation in {function.name} " |
mdengra
reviewed
Sep 15, 2021
| else: | ||
| warnings.warn(f"Failed to parse comment of module {self.scan_file}. Error: {inst}", stacklevel=2) | ||
| logging.warning(f"Failed to parse comment of module {self.scan_file}. Error: {inst}") | ||
| CodeParser.LOGGER.warning(f"Failed to parse comment of module {self.scan_file}. Error: {inst}") |
There was a problem hiding this comment.
Suggested change
| CodeParser.LOGGER.warning(f"Failed to parse comment of module {self.scan_file}. Error: {inst}") | |
| CodeParser.LOGGER.warning(f"Failed to parse module documentation in {self.scan_file}. Error: {inst}") |
mdengra
reviewed
Sep 15, 2021
| """ | ||
| brief: Reads from the config file the regexes used to identify test files. | ||
| """ | ||
| Config.LOGGER.debug('Reading the regular expressions to include files from config file') |
There was a problem hiding this comment.
Suggested change
| Config.LOGGER.debug('Reading the regular expressions to include files from config file') | |
| Config.LOGGER.debug('Reading the regular expressions from the config file to include test files') |
mdengra
reviewed
Sep 15, 2021
| if not 'Group files' in self._config_data: | ||
| logging.error("Config group files is empty") | ||
| raise Exception("Config group files is empty") | ||
| raise QAValueError('Config include paths are empty', Config.LOGGER.error) |
There was a problem hiding this comment.
Suggested change
| raise QAValueError('Config include paths are empty', Config.LOGGER.error) | |
| raise QAValueError('The include paths of the configuration file are empty', Config.LOGGER.error) |
mdengra
reviewed
Sep 15, 2021
| """ | ||
| brief: Reads from the config file the regexes used to identify a test method. | ||
| """ | ||
| Config.LOGGER.debug('Reading the regular expressions to include test methods from config file') |
There was a problem hiding this comment.
Suggested change
| Config.LOGGER.debug('Reading the regular expressions to include test methods from config file') | |
| Config.LOGGER.debug('Reading the regular expressions to include test methods from the config file') |
mdengra
reviewed
Sep 15, 2021
| """ | ||
| brief: Reads from the config file all the paths to be excluded from the parsing. | ||
| """ | ||
| Config.LOGGER.debug('Reading the paths to be ignored from config file') |
There was a problem hiding this comment.
Suggested change
| Config.LOGGER.debug('Reading the paths to be ignored from config file') | |
| Config.LOGGER.debug('Reading the paths to be ignored from the config file') |
mdengra
reviewed
Sep 15, 2021
| if not 'Module' in self._config_data['Output fields']: | ||
| logging.error("Config output module fields is missing") | ||
| raise Exception("Config output module fields is missing") | ||
| raise QAValueError('Config output module fields is missing', Config.LOGGER.error) |
There was a problem hiding this comment.
Suggested change
| raise QAValueError('Config output module fields is missing', Config.LOGGER.error) | |
| raise QAValueError('Config output module fields are missing', Config.LOGGER.error) |
mdengra
reviewed
Sep 15, 2021
| """ | ||
| brief: Reads from the config file the optional and mandatory fields for the test functions. | ||
| """ | ||
| Config.LOGGER.debug('Reading mandatory and optional test fields from config file') |
There was a problem hiding this comment.
Suggested change
| Config.LOGGER.debug('Reading mandatory and optional test fields from config file') | |
| Config.LOGGER.debug('Reading mandatory and optional test fields from the config file') |
mdengra
reviewed
Sep 15, 2021
| if not 'Test' in self._config_data['Output fields']: | ||
| logging.error("Config output test fields is missing") | ||
| raise Exception("Config output test fields is missing") | ||
| raise QAValueError('Config output test fields is missing', Config.LOGGER.error) |
There was a problem hiding this comment.
Suggested change
| raise QAValueError('Config output test fields is missing', Config.LOGGER.error) | |
| raise QAValueError('Config output test fields are missing', Config.LOGGER.error) |
mdengra
reviewed
Sep 15, 2021
| raise Exception("Config output fields is missing") | ||
| self._read_module_fields() | ||
| self._read_test_fields() | ||
| raise QAValueError('Config output fields is missing', Config.LOGGER.error) |
There was a problem hiding this comment.
Suggested change
| raise QAValueError('Config output fields is missing', Config.LOGGER.error) | |
| raise QAValueError('Config output fields are missing', Config.LOGGER.error) |
mdengra
reviewed
Sep 15, 2021
| """ | ||
| brief: Reads from the configuration file the key to identify a Test Case list. | ||
| """ | ||
| Config.LOGGER.debug('Reading Test Case key from config file') |
There was a problem hiding this comment.
Suggested change
| Config.LOGGER.debug('Reading Test Case key from config file') | |
| Config.LOGGER.debug('Reading Test Case key from the config file') |
mdengra
reviewed
Sep 15, 2021
| Test info: | ||
| - test_wazuh_min_version: wazuh_min_version | ||
| """ | ||
| Config.LOGGER.debug('Reading test info from config file') |
There was a problem hiding this comment.
Suggested change
| Config.LOGGER.debug('Reading test info from config file') | |
| Config.LOGGER.debug('Reading test info from the config file') |
mdengra
reviewed
Sep 15, 2021
| """ | ||
| brief: Reads from the config file the optional and mandatory fields for the test module. | ||
| """ | ||
| Config.LOGGER.debug('Reading mandatory and optional module fields from config file') |
There was a problem hiding this comment.
Suggested change
| Config.LOGGER.debug('Reading mandatory and optional module fields from config file') | |
| Config.LOGGER.debug('Reading mandatory and optional module fields from the config file') |
mdengra
reviewed
Sep 15, 2021
| """ | ||
| brief: Reads from the config file the file name to be identified with a group. | ||
| """ | ||
| Config.LOGGER.debug('Reading group files from config file') |
There was a problem hiding this comment.
Suggested change
| Config.LOGGER.debug('Reading group files from config file') | |
| Config.LOGGER.debug('Reading group files from the config file') |
mdengra
reviewed
Sep 15, 2021
| """ | ||
| brief: Reads from the config file all the paths to be included in the parsing. | ||
| """ | ||
| Config.LOGGER.debug('Reading include paths from config file') |
There was a problem hiding this comment.
Suggested change
| Config.LOGGER.debug('Reading include paths from config file') | |
| Config.LOGGER.debug('Reading include paths from the config file') |
mdengra
reviewed
Sep 15, 2021
| """ | ||
| if 'Project path' in self._config_data: | ||
| self.project_path = self._config_data['Project path'] | ||
| Config.LOGGER.debug('Reading module info from config file') |
There was a problem hiding this comment.
Suggested change
| Config.LOGGER.debug('Reading module info from config file') | |
| Config.LOGGER.debug('Reading module info from the config file') |
mdengra
reviewed
Sep 15, 2021
| @@ -38,7 +43,7 @@ def collect_test_cases(self, path): | |||
| - "path (string): Path of the test file to extract the test cases. | |||
| returns: "dictionary: The output of pytest parsed into a dictionary" | |||
| """ | |||
| logging.debug(f"Running pytest to collect testcases for '{path}'") | |||
| PytestWrap.LOGGER.debug(f"Running pytest to collect testcases for '{path}'") | |||
There was a problem hiding this comment.
Suggested change
| PytestWrap.LOGGER.debug(f"Running pytest to collect testcases for '{path}'") | |
| PytestWrap.LOGGER.debug(f"Running pytest to collect test cases for '{path}'") |
mdengra
reviewed
Sep 15, 2021
| @@ -55,7 +59,7 @@ def validate_fields(self, required_fields, available_fields): | |||
| for field in required_fields: | |||
| if not check_existance(available_fields, field): | |||
| self.add_report(f"Mandatory field '{field}' is missing in file {self.scan_file}") | |||
| logging.error(f"Mandatory field '{field}' is missing in file {self.scan_file}") | |||
| Sanity.LOGGER.error(f"Mandatory field '{field}' is missing in file {self.scan_file}") | |||
There was a problem hiding this comment.
Suggested change
| Sanity.LOGGER.error(f"Mandatory field '{field}' is missing in file {self.scan_file}") | |
| Sanity.LOGGER.error(f"Mandatory field '{field}' is missing in the file {self.scan_file}") |
mdengra
reviewed
Sep 15, 2021
| @@ -65,7 +69,7 @@ def validate_fields(self, required_fields, available_fields): | |||
| else: | |||
| if not check_existance(available_fields, field): | |||
| self.add_report(f"Mandatory field '{field}' is missing in file {self.scan_file}") | |||
| logging.error(f"Mandatory field '{field}' is missing in file {self.scan_file}") | |||
| Sanity.LOGGER.error(f"Mandatory field '{field}' is missing in file {self.scan_file}") | |||
There was a problem hiding this comment.
Suggested change
| Sanity.LOGGER.error(f"Mandatory field '{field}' is missing in file {self.scan_file}") | |
| Sanity.LOGGER.error(f"Mandatory field '{field}' is missing in the file {self.scan_file}") |
mdengra
suggested changes
Sep 15, 2021
32 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
qa-docs was using the logging module, so we are using a custom
qa-docslogger now.All logging is filtered by level and showed using the standard output and a log file created in
qa_docs_installation/log/todays_date-qa-docs.log.Also, exceptions, errors, and raises are using custom loggers.
Logs example
A few of them can be found in #1879.
Log file creation: