Bugfix: Fix final log output missing when instance IDs are used in process list

[georgemccabe]

Describe the Problem

Currently there is a bug when instance names in the process list are used where the final logs like "INFO: METplus has successfully finished running." and the path to the log file are not shown.

Expected Behavior

All log output, including final log info reporting success of overall METplus run and path to the log file, are shown, even when using instance identifiers in the process list.

Environment

Describe your runtime environment:
Any

To Reproduce

Describe the steps to reproduce the behavior:
1. Copy parm/use_cases/met_tool_wrapper/GridStat/GridStat.conf to another directory
2. Modify conf file to replace PROCESS_LIST line:

from:

PROCESS_LIST = GridStat

to:

PROCESS_LIST = GridStat(inst1), GridStat(inst2)

[inst1]

LEAD_SEQ = 12

[inst2]

LEAD_SEQ = 12

[config]

3. Run use case and confirm that final log output is not shown. Last log line will be success status of grid_stat run

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Review default alert labels
• Select component(s)
• Select priority
• Select requestor(s)

Milestone and Projects

• Select Milestone as the next bugfix version
• Select Coordinated METplus-X.Y Support project for support of the current coordinated release
• Select METplus-Wrappers-X.Y.Z Development project for development toward the next official release

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

Bugfix Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.

• Fork this repository or create a branch of main_<Version>.
  Branch name: bugfix_<Issue Number>_main_<Version>_<Description>

• Fix the bug and test your changes.

• Add/update log messages for easier debugging.

• Add/update unit tests.

• Add/update documentation.

• Add any new Python packages to the METplus Components Python Requirements table.

• Push local changes to GitHub.

• Submit a pull request to merge into main_<Version>.
  Pull request: bugfix <Issue Number> main_<Version> <Description>

• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next bugfix version
  Select: Coordinated METplus-X.Y Support project for support of the current coordinated release

• Iterate until the reviewer(s) accept and merge your changes.

• Delete your fork or branch.

• Complete the steps above to fix the bug on the develop branch.
  Branch name: bugfix_<Issue Number>_develop_<Description>
  Pull request: bugfix <Issue Number> develop <Description>
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next official version
  Select: METplus-Wrappers-X.Y.Z Development project for development toward the next official release

• Close this issue.

[georgemccabe]

Some notes about what I determined was happening and how it was fixed:

The config_metplus.setup function creates a METplusConfig object from the command line arguments. It also sets up the logger with handlers for terminal and file output. This logger is stored in the METplusConfig object. Issue #2245 was created because multiple runs of METplus started concurrently (and the pytests configs objects) were writing all of their logs to the same files redundantly. This was resolved by writing a __del__ function for METplusConfig to close the logger handlers when the object goes out of scope and is deleted. The METplusConfig object created by config_metplus.setup is passed to the run_util.run_metplus function to initialize the wrapper classes and persists after the wrappers are run to be passed to the run_util.post_run_cleanup function to output the final logs of the run (e.g. success/failure of METplus and log file path).

To handle Instance Names in the Process List, a new METplusConfig object is created and populated with the values from the [config] section and values from the [instance_name] section. This instance of the config object uses the same logger as the original. The issue arises when the run_util.run_metplus function ends and the CommandBuilder (METplus wrapper base class) instances go out of scope and are deleted. This closes the logger handlers before the run_util.post_run_cleanup function is called.

To handle this, I modified the METplusConfig class to track if it is the original instance or a "copy" instance used for handling instance IDs. If it is a "copy," then it does not remove the logger handlers, assuming that the original copy will do this at the end of the METplus run when it goes out of scope.

There may be a better way to handle this, such as decoupling the logger and the METplusConfig object so that the config does not clean up the logger to prevent the logger's scope to be determined by the config object's scope.