Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature dtcenter/METplus-Internal#24 test failure #1895

Merged
merged 102 commits into from
Oct 26, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
ab0dda9
clean up
georgemccabe Sep 28, 2022
04db77b
per #1842, add support for setting multiple jobs for StatAnalysis wra…
georgemccabe Sep 28, 2022
8c5983d
removed test for LOOP_ORDER
georgemccabe Sep 29, 2022
4e7db5f
skip setting env var for JOBS since it is now a list instead of a str…
georgemccabe Sep 29, 2022
5b007f9
fixed broken unit test by using wrapped StatAnalysis config file inst…
georgemccabe Sep 30, 2022
c325ed9
remove files that should be generated by tests if they already exists…
georgemccabe Sep 30, 2022
a44c866
remove LOOP_ORDER from MakePlots wrapper and tests
georgemccabe Sep 30, 2022
08d621f
Revert "remove LOOP_ORDER from MakePlots wrapper and tests"
georgemccabe Sep 30, 2022
88f1fb8
per #1772, set verbosity in stat_analysis command
georgemccabe Sep 30, 2022
3fc2aee
per #1772, added verbosity to expected command
georgemccabe Sep 30, 2022
3ec43d0
added test for value found in existing use case, clean up formatting
georgemccabe Sep 30, 2022
09d4839
moved stat_analysis tests to new pytest group wrapper_d
georgemccabe Sep 30, 2022
a8946f7
temporarily disable unit test to ensure that changes to remove MakePl…
georgemccabe Sep 30, 2022
191f811
per #1843, remove MakePlots wrapper
georgemccabe Sep 30, 2022
8fb7a6f
turn on StatAnalysis use cases to test after removing MakePlots ci-ru…
georgemccabe Sep 30, 2022
c039349
per #1843, removed plot_util tests because it has been removed
georgemccabe Sep 30, 2022
3f73c7d
removed glossary item that is no longer used
georgemccabe Sep 30, 2022
4a9e3ca
changed logic to format lists when they are read and added support fo…
georgemccabe Sep 30, 2022
e644194
per #1842, add quotation marks around name/level/units that were read…
georgemccabe Oct 3, 2022
3bc2e88
fixed unit test to match change to get_level_list function that now a…
georgemccabe Oct 3, 2022
d97ceb9
per #1842, change logic to handle padding HHMMSS values to still supp…
georgemccabe Oct 3, 2022
7739d2c
Merge branch 'develop' into feature_1842_stat_analysis_multiple_jobs
georgemccabe Oct 3, 2022
ce3a33c
clean up logic that loops over 2 lists and performs the same action o…
georgemccabe Oct 3, 2022
4647b07
reordered functions to better follow flow of logic
georgemccabe Oct 3, 2022
d350c61
moved list in function to class variable and capitalized all constant…
georgemccabe Oct 3, 2022
2097047
clean up formatting of test code
georgemccabe Oct 3, 2022
504b990
clean up formatting of test code
georgemccabe Oct 3, 2022
86ac6d9
per #1842, moved logic to set fcst/obs beg/end string template sub va…
georgemccabe Oct 3, 2022
92b2def
refactored wrapper to be more readable, use relativedelta for hour/le…
georgemccabe Oct 4, 2022
6238de4
refactored wrapper to be more readable, use relativedelta for hour/le…
georgemccabe Oct 4, 2022
f7a23fa
refactored unit tests to use parameterize and cleaned up code
georgemccabe Oct 5, 2022
3512997
merged develop and resolved conflicts
georgemccabe Oct 5, 2022
dc0e239
combine logic to process loop lists and group lists to consolidate lo…
georgemccabe Oct 5, 2022
c749660
formatting
georgemccabe Oct 5, 2022
83405d4
Added test to ensure that reading FCST_LEAD_LIST1, FCST_LEAD_LIST2, e…
georgemccabe Oct 5, 2022
37b42c5
added logic to compute init_beg/end from valid_beg/end and forecast l…
georgemccabe Oct 5, 2022
fd89fcb
fixed logic that was using date beg + 23:59:59 for the end time if no…
georgemccabe Oct 5, 2022
a7250ba
Merge branch 'develop' into feature_1842_stat_analysis_multiple_jobs
georgemccabe Oct 10, 2022
32ec575
cleanup and document new logic
georgemccabe Oct 10, 2022
3a0ac73
refactored logic to remove unnecessary arguments, changed logic that …
georgemccabe Oct 10, 2022
8eaf348
changed logic to call function that copies over relevant config list …
georgemccabe Oct 10, 2022
77f029c
more cleanup
georgemccabe Oct 10, 2022
da6c638
added info specific to StatAnalysis wrapper to documentation
georgemccabe Oct 11, 2022
8dc34e3
removed LOOP_ORDER and LEAD_SEQ from basic use case because they are …
georgemccabe Oct 11, 2022
42babcb
clean up formatting
georgemccabe Oct 11, 2022
a4bd994
improved documentation for function and more clean up
georgemccabe Oct 11, 2022
5c1de4a
per #1862, add support for using filename template tags anywhere in t…
georgemccabe Oct 11, 2022
c4b1bc8
refactor tests to be more easily expandable
georgemccabe Oct 11, 2022
12a2d9a
cleaned up logic to be more maintainable, ci-run-diff
georgemccabe Oct 11, 2022
64ad315
set INIT/VALID_BEG/END even if no corresponding HOUR list was specifi…
georgemccabe Oct 11, 2022
ee0f21c
updated use case to read from a different point_stat stat file that h…
georgemccabe Oct 12, 2022
e50acdd
clean up code, rearrange functions in order they are used, updated co…
georgemccabe Oct 12, 2022
e7f719a
moved list_to_str function to string_manip util and moved tests to ap…
georgemccabe Oct 12, 2022
f6a64fb
Changed logic to use full [INIT/VALID]_[BEG/END] times instead of tru…
georgemccabe Oct 13, 2022
f25fa86
removed logic to build filename template if not specified by the user…
georgemccabe Oct 13, 2022
39bc1a1
remove run_at_time method since it will no longer be called since LOO…
georgemccabe Oct 13, 2022
49b7a2f
moved variables to store datetime notation to set YYYYMMDD and YYYYMM…
georgemccabe Oct 13, 2022
75375b9
Removed rest of logic that reads filename type since we now require t…
georgemccabe Oct 14, 2022
d0cc48c
improved list to string logic by handling if quotes already exist whe…
georgemccabe Oct 14, 2022
3e8b832
made MODEL<n>_OBTYPE optional so users can filter without obtype, rem…
georgemccabe Oct 14, 2022
acc04d3
exit non-zero and print an error if any warnings/errors occur when ru…
georgemccabe Oct 14, 2022
9c44e40
added glossary entry for STAT_ANALYSIS_JOB<n> variable and cleaned up…
georgemccabe Oct 14, 2022
926d231
added info about setting groups of fcst leads
georgemccabe Oct 14, 2022
867783f
added info specific to StatAnalysis wrapper its Description section a…
georgemccabe Oct 14, 2022
fe3c2a0
added info about additional filename template tags that can be used i…
georgemccabe Oct 14, 2022
6b30b6b
added info to note that full day of end time will be processed if onl…
georgemccabe Oct 14, 2022
685d25e
Per #1687, removed LOOP_ORDER from all config files
georgemccabe Oct 14, 2022
e978a51
Per #1687, removed LOOP_ORDER from default config
georgemccabe Oct 14, 2022
c5061ab
added missing rst characters
georgemccabe Oct 14, 2022
6cf90ef
Merge branch 'develop' into feature_1842_stat_analysis_multiple_jobs
georgemccabe Oct 17, 2022
39e2c41
moved utilities to parse time lists from strings to time_util, cleane…
georgemccabe Oct 17, 2022
1ac8645
moved functions to handle thresholds to string_manip utility
georgemccabe Oct 17, 2022
c91e299
cleaned up logic to remove pylint complaints and be consistent
georgemccabe Oct 17, 2022
665cf09
turn off all use case tests
georgemccabe Oct 17, 2022
b563d86
check that changing logic to not set beg or end based on hour list do…
georgemccabe Oct 19, 2022
86b2e42
Merge branch 'develop' into feature_1842_stat_analysis_multiple_jobs
georgemccabe Oct 20, 2022
e871d50
change logic to never set fcst/obs_init/valid__beg/end values based o…
georgemccabe Oct 20, 2022
653b04c
pass runtime settings to set_environment_variables function so that s…
georgemccabe Oct 20, 2022
5f25e89
added logic to substitute string template tags when reading explicit …
georgemccabe Oct 20, 2022
2462db3
started adding tests to ensure fcst/obs_init/valid_beg/end values are…
georgemccabe Oct 20, 2022
6c9a627
added more tests for setting beg/end values in wrapped MET config
georgemccabe Oct 20, 2022
437e6d6
removed setting of legacy environment variables that were set to supp…
georgemccabe Oct 20, 2022
a7ef6ef
updating docs with info about new settings
georgemccabe Oct 20, 2022
9d74886
added rest of new config variables to glossary and fixed formatting t…
georgemccabe Oct 20, 2022
339e31a
remove env vars that are no longer needed
georgemccabe Oct 21, 2022
beedf87
use latest release of MET for tests
georgemccabe Oct 21, 2022
5057f23
fixed path to generate code coverage report
georgemccabe Oct 21, 2022
404f034
Changed directory to write test output to a directory called test_out…
georgemccabe Oct 21, 2022
47b5a2f
per dtcenter/METplus-Internal#24, added tests to ensure that run_metp…
georgemccabe Oct 21, 2022
5547117
save changes
georgemccabe Oct 21, 2022
a306188
Merge branch 'feature_1842_stat_analysis_multiple_jobs' of github.com…
georgemccabe Oct 21, 2022
3c760f8
Modified pytest fixture logic so that tests that create a minimum con…
georgemccabe Oct 21, 2022
33ae44f
fix tests that create more than 1 METplusConfig object in the test
georgemccabe Oct 21, 2022
1646010
fixed another test that broke
georgemccabe Oct 21, 2022
d89f218
updated tests that use a config file to set the values in the test fi…
georgemccabe Oct 21, 2022
f1a6892
allow now and today keywords in string template substitution for Stat…
georgemccabe Oct 21, 2022
2aea590
Merge branch 'feature_1842_stat_analysis_multiple_jobs' into feature_…
georgemccabe Oct 21, 2022
61c6d12
updated tests to avoid using config files so they can be auto-scrubbe…
georgemccabe Oct 21, 2022
4c0a024
merged develop and resolved conflicts
georgemccabe Oct 24, 2022
949e809
Merge branch 'develop' into feature_24i_test_failure
georgemccabe Oct 25, 2022
46a8d64
simplified logic
georgemccabe Oct 25, 2022
2f461c7
merged develop and resolved conflicts
georgemccabe Oct 25, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/run_tests/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ if [[ "$INPUT_CATEGORIES" == pytests* ]]; then
for x in `cat $PYTESTS_GROUPS_FILEPATH`; do
marker="${x//_or_/ or }"
marker="${marker//not_/not }"
command+="/usr/local/envs/pytest/bin/pytest -vv --cov=../../metplus -m \"$marker\""
command+="/usr/local/envs/pytest/bin/pytest -vv --cov=../../../metplus -m \"$marker\""
command+=";if [ \$? != 0 ]; then status=1; fi;"
done
command+="if [ \$status != 0 ]; then echo ERROR: Some pytests failed. Search for FAILED to review; false; fi"
Expand Down
1 change: 1 addition & 0 deletions .github/parm/pytest_groups.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
run_metplus
util
wrapper
wrapper_a
Expand Down
76 changes: 64 additions & 12 deletions internal/tests/pytests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import subprocess
import pytest
import getpass
import shutil
from pathlib import Path

# add METplus directory to path so the wrappers and utilities can be found
Expand All @@ -19,7 +20,8 @@
if pytest_host is None:
import socket
pytest_host = socket.gethostname()
print(f"No hostname provided with METPLUS_PYTEST_HOST, using {pytest_host}")
print("No hostname provided with METPLUS_PYTEST_HOST, "
f"using {pytest_host}")
else:
print(f"METPLUS_PYTEST_HOST = {pytest_host}")

Expand All @@ -33,7 +35,8 @@

# source minimum_pytest.<pytest_host>.sh script
current_user = getpass.getuser()
command = shlex.split(f"env -i bash -c 'export USER={current_user} && source {minimum_pytest_file} && env'")
command = shlex.split(f"env -i bash -c 'export USER={current_user} && "
f"source {minimum_pytest_file} && env'")
proc = subprocess.Popen(command, stdout=subprocess.PIPE)

for line in proc.stdout:
Expand All @@ -43,21 +46,70 @@

proc.communicate()

output_base = os.environ['METPLUS_TEST_OUTPUT_BASE']
if not output_base:
print('ERROR: METPLUS_TEST_OUTPUT_BASE must be set to a path to write')
sys.exit(1)

test_output_dir = os.path.join(output_base, 'test_output')
if os.path.exists(test_output_dir):
print(f'Removing test output dir: {test_output_dir}')
shutil.rmtree(test_output_dir)


@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
"""! This is used to capture the status of a test so the metplus_config
fixture can remove output data from tests that pass.
"""
# execute all other hooks to obtain the report object
outcome = yield
rep = outcome.get_result()

# set a report attribute for each phase of a call, which can
# be "setup", "call", "teardown"

setattr(item, "rep_" + rep.when, rep)


@pytest.fixture()
def metplus_config(request):
"""! Create a METplus configuration object using only the minimum required
settings found in minimum_pytest.conf. This fixture checks the result of
the test it is used in and automatically removes the output that is
generated by it unless the test fails. This makes it much easier to review
the failed tests. To use this fixture, add metplus_config to the test
function arguments and set a variable called config to metplus_config, e.g.
config = metplus_config.
"""
script_dir = os.path.dirname(__file__)
args = [os.path.join(script_dir, 'minimum_pytest.conf')]
config = config_metplus.setup(args)
yield config

# don't remove output base if test fails
if request.node.rep_call.failed:
return
config_output_base = config.getdir('OUTPUT_BASE')
if config_output_base and os.path.exists(config_output_base):
shutil.rmtree(config_output_base)


@pytest.fixture(scope='function')
def metplus_config():
"""! Create a METplus configuration object that can be
manipulated/modified to
reflect different paths, directories, values, etc. for individual
tests.
def metplus_config_files():
"""! Create a METplus configuration object using minimum_pytest.conf
settings and any list of config files.The metplus_config fixture is
preferred because it automatically cleans up the output files generated
by the use case unless the test fails. To use this in a test, add
metplus_config_files as an argument to the test function and pass in a list
of config files to it. Example: config = metplus_config_files([my_file])
"""
def read_configs(extra_configs=[]):
def read_configs(extra_configs):
# Read in minimum pytest config file and any other extra configs
script_dir = os.path.dirname(__file__)
minimum_conf = os.path.join(script_dir, 'minimum_pytest.conf')
args = [minimum_conf]
if extra_configs:
args.extend(extra_configs)

args = extra_configs.copy()
args.append(minimum_conf)
config = config_metplus.setup(args)
return config

Expand Down
2 changes: 1 addition & 1 deletion internal/tests/pytests/minimum_pytest.conf
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[config]
INPUT_BASE = {ENV[METPLUS_TEST_INPUT_BASE]}
OUTPUT_BASE = {ENV[METPLUS_TEST_OUTPUT_BASE]}
OUTPUT_BASE = {ENV[METPLUS_TEST_OUTPUT_BASE]}/test_output/{RUN_ID}
MET_INSTALL_DIR = {ENV[METPLUS_TEST_MET_INSTALL_DIR]}
TMP_DIR = {ENV[METPLUS_TEST_TMP_DIR]}

Expand Down
13 changes: 0 additions & 13 deletions internal/tests/pytests/minimum_pytest.corrinado.sh

This file was deleted.

9 changes: 0 additions & 9 deletions internal/tests/pytests/minimum_pytest.dakota.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,3 @@ export METPLUS_TEST_INPUT_BASE=/d3/projects/MET/METplus_Data
export METPLUS_TEST_OUTPUT_BASE=/d3/personal/${USER}/pytest
export METPLUS_TEST_MET_INSTALL_DIR=/d3/projects/MET/MET_releases/met-9.1_beta3
export METPLUS_TEST_TMP_DIR=${METPLUS_TEST_OUTPUT_BASE}/tmp

export METPLUS_TEST_EXE_WGRIB2=/usr/local/bin/wgrib2
export METPLUS_TEST_EXE_CUT=/usr/bin/cut
export METPLUS_TEST_EXE_TR=/usr/bin/tr
export METPLUS_TEST_EXE_RM=/bin/rm
export METPLUS_TEST_EXE_NCAP2=/usr/local/nco/bin/ncap2
export METPLUS_TEST_EXE_CONVERT=/usr/bin/convert
export METPLUS_TEST_EXE_NCDUMP=/usr/local/bin/ncdump
export METPLUS_TEST_EXE_EGREP=/bin/egrep
9 changes: 0 additions & 9 deletions internal/tests/pytests/minimum_pytest.eyewall.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,3 @@ export METPLUS_TEST_OUTPUT_BASE=/d1/${USER}/pytest
export METPLUS_TEST_MET_INSTALL_DIR=/usr/local/met-9.0
#export METPLUS_TEST_MET_INSTALL_DIR=/d1/CODE/MET/MET_releases/met-9.0_beta4
export METPLUS_TEST_TMP_DIR=${METPLUS_TEST_OUTPUT_BASE}/tmp

export METPLUS_TEST_EXE_WGRIB2=/usr/local/bin/wgrib2
export METPLUS_TEST_EXE_CUT=/usr/bin/cut
export METPLUS_TEST_EXE_TR=/usr/bin/tr
export METPLUS_TEST_EXE_RM=/bin/rm
export METPLUS_TEST_EXE_NCAP2=/usr/local/nco/bin/ncap2
export METPLUS_TEST_EXE_CONVERT=/usr/bin/convert
export METPLUS_TEST_EXE_NCDUMP=/usr/local/bin/ncdump
export METPLUS_TEST_EXE_EGREP=/bin/egrep
9 changes: 0 additions & 9 deletions internal/tests/pytests/minimum_pytest.hera.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,3 @@ export METPLUS_TEST_INPUT_BASE=/home/${USER}/metplus_pytests
export METPLUS_TEST_OUTPUT_BASE=/home/${USER}/metplus_pytests/out
export METPLUS_TEST_MET_INSTALL_DIR=/contrib/met/8.1
export METPLUS_TEST_TMP_DIR=/tmp

export METPLUS_TEST_EXE_WGRIB2=/apps/wgrib2/2.0.8/intel/18.0.3.222/bin/wgrib2
export METPLUS_TEST_EXE_CUT=/usr/bin/cut
export METPLUS_TEST_EXE_TR=/usr/bin/tr
export METPLUS_TEST_EXE_RM=/usr/bin/rm
export METPLUS_TEST_EXE_NCAP2=/apps/nco/4.7.0/intel/18.0.3.051/bin/ncap2
export METPLUS_TEST_EXE_CONVERT=/usr/bin/convert
export METPLUS_TEST_EXE_NCDUMP=/apps/netcdf/4.7.0/intel/18.0.5.274/bin/ncdump
export METPLUS_TEST_EXE_EGREP=/usr/bin/grep
11 changes: 1 addition & 10 deletions internal/tests/pytests/minimum_pytest.kiowa.sh
Original file line number Diff line number Diff line change
@@ -1,14 +1,5 @@
export METPLUS_TEST_INPUT_BASE=/d1/projects/METplus/METplus_Data
export METPLUS_TEST_OUTPUT_BASE=/d1/personal/${USER}/pytest
export METPLUS_TEST_MET_INSTALL_DIR=/usr/local/met-9.0
export METPLUS_TEST_MET_INSTALL_DIR=/usr/local/met
#export METPLUS_TEST_MET_INSTALL_DIR=/d1/projects/MET/MET_releases/met-9.0_beta4
export METPLUS_TEST_TMP_DIR=${METPLUS_TEST_OUTPUT_BASE}/tmp
#export METPLUS_TEST_TMP_DIR=/tmp
export METPLUS_TEST_EXE_WGRIB2=/usr/local/bin/wgrib2
export METPLUS_TEST_EXE_CUT=/usr/bin/cut
export METPLUS_TEST_EXE_TR=/usr/bin/tr
export METPLUS_TEST_EXE_RM=/bin/rm
export METPLUS_TEST_EXE_NCAP2=/usr/local/nco/bin/ncap2
export METPLUS_TEST_EXE_CONVERT=/usr/bin/convert
export METPLUS_TEST_EXE_NCDUMP=/usr/local/bin/ncdump
export METPLUS_TEST_EXE_EGREP=/bin/egrep
9 changes: 0 additions & 9 deletions internal/tests/pytests/minimum_pytest.venus.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,3 @@ export METPLUS_TEST_INPUT_BASE=/gpfs/dell2/emc/verification/noscrub/$USER/METplu
export METPLUS_TEST_OUTPUT_BASE=/gpfs/dell2/emc/verification/noscrub/$USER/metplus_test
export METPLUS_TEST_MET_INSTALL_DIR=/gpfs/dell2/emc/verification/noscrub/$USER/met/9.0_beta4
export METPLUS_TEST_TMP_DIR=${METPLUS_TEST_OUTPUT_BASE}/tmp

export METPLUS_TEST_EXE_WGRIB2=$WGRIB2
export METPLUS_TEST_EXE_CUT=cut
export METPLUS_TEST_EXE_TR=tr
export METPLUS_TEST_EXE_RM=rm
export METPLUS_TEST_EXE_NCAP2=ncap2
export METPLUS_TEST_EXE_CONVERT=convert
export METPLUS_TEST_EXE_NCDUMP=ncdump
export METPLUS_TEST_EXE_EGREP=egrep
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ def set_minimum_config_settings(config):
)
@pytest.mark.plotting
def test_read_loop_info(metplus_config, config_overrides, expected_loop_args):
config = metplus_config()
config = metplus_config

set_minimum_config_settings(config)

Expand Down Expand Up @@ -181,7 +181,7 @@ def test_read_loop_info(metplus_config, config_overrides, expected_loop_args):
@pytest.mark.plotting
def test_tcmpr_plotter_loop(metplus_config, config_overrides,
expected_strings):
config = metplus_config()
config = metplus_config

set_minimum_config_settings(config)

Expand Down Expand Up @@ -278,7 +278,7 @@ def test_tcmpr_plotter(metplus_config, config_overrides, expected_string):
expected_string = f' {expected_string}'

for single_file in [True, False]:
config = metplus_config()
config = metplus_config

set_minimum_config_settings(config)

Expand Down
1 change: 1 addition & 0 deletions internal/tests/pytests/pytest.ini
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
[pytest]
markers =
run_metplus: custom marker for testing run_metplus.py script
util: custom marker for testing metplus/util logic
wrapper_a: custom marker for testing metplus/wrapper logic - A group
wrapper_b: custom marker for testing metplus/wrapper logic - B group
Expand Down
53 changes: 53 additions & 0 deletions internal/tests/pytests/run_metplus/test_run_metplus.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/usr/bin/env python3

import pytest

from pathlib import Path
import os
import shutil
from subprocess import run

# get METplus directory relative to this file
# from this script's directory, go up 4 directories
METPLUS_DIR = str(Path(__file__).parents[4])
RUN_METPLUS = os.path.join(METPLUS_DIR, 'ush', 'run_metplus.py')
EXAMPLE_CONF = os.path.join(METPLUS_DIR, 'parm', 'use_cases',
'met_tool_wrapper', 'Example', 'Example.conf')
MINIMUM_CONF = os.path.join(METPLUS_DIR, 'internal', 'tests', 'pytests',
'minimum_pytest.conf')
TEST_OUTPUT_DIR = os.path.join(os.environ['METPLUS_TEST_OUTPUT_BASE'],
'test_output')
NEW_OUTPUT_BASE = os.path.join(TEST_OUTPUT_DIR, 'run_metplus')
OUTPUT_BASE_OVERRIDE = f"config.OUTPUT_BASE={NEW_OUTPUT_BASE}"

@pytest.mark.run_metplus
def test_run_metplus_exists():
"""! Check that run_metplus.py script exists """
assert os.path.exists(RUN_METPLUS)


@pytest.mark.parametrize(
'command, expected_return_code', [
([RUN_METPLUS], 2),
([RUN_METPLUS, EXAMPLE_CONF], 2),
([RUN_METPLUS, EXAMPLE_CONF, MINIMUM_CONF, OUTPUT_BASE_OVERRIDE], 0),
]
)
@pytest.mark.run_metplus
def test_run_metplus_check_return_code(command, expected_return_code):
"""! Call run_metplus.py without various arguments and check that the
expected value is returned by the script. A successful run should return
0 and a failed run should return a non-zero return code, typically 2.
"""
process = run(command)
assert process.returncode == expected_return_code

if os.path.exists(NEW_OUTPUT_BASE):
shutil.rmtree(NEW_OUTPUT_BASE)


@pytest.mark.run_metplus
def test_output_dir_is_created():
"""! Check that the test output directory was created after running tests
"""
assert os.path.exists(TEST_OUTPUT_DIR)
Loading