Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature 1842 StatAnalysis - multiple issues #1869

Merged
merged 88 commits into from
Oct 24, 2022
Merged

Feature 1842 StatAnalysis - multiple issues #1869

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
ab0dda9
clean up
georgemccabe Sep 28, 2022
04db77b
per #1842, add support for setting multiple jobs for StatAnalysis wra…
georgemccabe Sep 28, 2022
8c5983d
removed test for LOOP_ORDER
georgemccabe Sep 29, 2022
4e7db5f
skip setting env var for JOBS since it is now a list instead of a str…
georgemccabe Sep 29, 2022
5b007f9
fixed broken unit test by using wrapped StatAnalysis config file inst…
georgemccabe Sep 30, 2022
c325ed9
remove files that should be generated by tests if they already exists…
georgemccabe Sep 30, 2022
a44c866
remove LOOP_ORDER from MakePlots wrapper and tests
georgemccabe Sep 30, 2022
08d621f
Revert "remove LOOP_ORDER from MakePlots wrapper and tests"
georgemccabe Sep 30, 2022
88f1fb8
per #1772, set verbosity in stat_analysis command
georgemccabe Sep 30, 2022
3fc2aee
per #1772, added verbosity to expected command
georgemccabe Sep 30, 2022
3ec43d0
added test for value found in existing use case, clean up formatting
georgemccabe Sep 30, 2022
09d4839
moved stat_analysis tests to new pytest group wrapper_d
georgemccabe Sep 30, 2022
a8946f7
temporarily disable unit test to ensure that changes to remove MakePl…
georgemccabe Sep 30, 2022
191f811
per #1843, remove MakePlots wrapper
georgemccabe Sep 30, 2022
8fb7a6f
turn on StatAnalysis use cases to test after removing MakePlots ci-ru…
georgemccabe Sep 30, 2022
c039349
per #1843, removed plot_util tests because it has been removed
georgemccabe Sep 30, 2022
3f73c7d
removed glossary item that is no longer used
georgemccabe Sep 30, 2022
4a9e3ca
changed logic to format lists when they are read and added support fo…
georgemccabe Sep 30, 2022
e644194
per #1842, add quotation marks around name/level/units that were read…
georgemccabe Oct 3, 2022
3bc2e88
fixed unit test to match change to get_level_list function that now a…
georgemccabe Oct 3, 2022
d97ceb9
per #1842, change logic to handle padding HHMMSS values to still supp…
georgemccabe Oct 3, 2022
7739d2c
Merge branch 'develop' into feature_1842_stat_analysis_multiple_jobs
georgemccabe Oct 3, 2022
ce3a33c
clean up logic that loops over 2 lists and performs the same action o…
georgemccabe Oct 3, 2022
4647b07
reordered functions to better follow flow of logic
georgemccabe Oct 3, 2022
d350c61
moved list in function to class variable and capitalized all constant…
georgemccabe Oct 3, 2022
2097047
clean up formatting of test code
georgemccabe Oct 3, 2022
504b990
clean up formatting of test code
georgemccabe Oct 3, 2022
86ac6d9
per #1842, moved logic to set fcst/obs beg/end string template sub va…
georgemccabe Oct 3, 2022
92b2def
refactored wrapper to be more readable, use relativedelta for hour/le…
georgemccabe Oct 4, 2022
6238de4
refactored wrapper to be more readable, use relativedelta for hour/le…
georgemccabe Oct 4, 2022
f7a23fa
refactored unit tests to use parameterize and cleaned up code
georgemccabe Oct 5, 2022
3512997
merged develop and resolved conflicts
georgemccabe Oct 5, 2022
dc0e239
combine logic to process loop lists and group lists to consolidate lo…
georgemccabe Oct 5, 2022
c749660
formatting
georgemccabe Oct 5, 2022
83405d4
Added test to ensure that reading FCST_LEAD_LIST1, FCST_LEAD_LIST2, e…
georgemccabe Oct 5, 2022
37b42c5
added logic to compute init_beg/end from valid_beg/end and forecast l…
georgemccabe Oct 5, 2022
fd89fcb
fixed logic that was using date beg + 23:59:59 for the end time if no…
georgemccabe Oct 5, 2022
a7250ba
Merge branch 'develop' into feature_1842_stat_analysis_multiple_jobs
georgemccabe Oct 10, 2022
32ec575
cleanup and document new logic
georgemccabe Oct 10, 2022
3a0ac73
refactored logic to remove unnecessary arguments, changed logic that …
georgemccabe Oct 10, 2022
8eaf348
changed logic to call function that copies over relevant config list …
georgemccabe Oct 10, 2022
77f029c
more cleanup
georgemccabe Oct 10, 2022
da6c638
added info specific to StatAnalysis wrapper to documentation
georgemccabe Oct 11, 2022
8dc34e3
removed LOOP_ORDER and LEAD_SEQ from basic use case because they are …
georgemccabe Oct 11, 2022
42babcb
clean up formatting
georgemccabe Oct 11, 2022
a4bd994
improved documentation for function and more clean up
georgemccabe Oct 11, 2022
5c1de4a
per #1862, add support for using filename template tags anywhere in t…
georgemccabe Oct 11, 2022
c4b1bc8
refactor tests to be more easily expandable
georgemccabe Oct 11, 2022
12a2d9a
cleaned up logic to be more maintainable, ci-run-diff
georgemccabe Oct 11, 2022
64ad315
set INIT/VALID_BEG/END even if no corresponding HOUR list was specifi…
georgemccabe Oct 11, 2022
ee0f21c
updated use case to read from a different point_stat stat file that h…
georgemccabe Oct 12, 2022
e50acdd
clean up code, rearrange functions in order they are used, updated co…
georgemccabe Oct 12, 2022
e7f719a
moved list_to_str function to string_manip util and moved tests to ap…
georgemccabe Oct 12, 2022
f6a64fb
Changed logic to use full [INIT/VALID]_[BEG/END] times instead of tru…
georgemccabe Oct 13, 2022
f25fa86
removed logic to build filename template if not specified by the user…
georgemccabe Oct 13, 2022
39bc1a1
remove run_at_time method since it will no longer be called since LOO…
georgemccabe Oct 13, 2022
49b7a2f
moved variables to store datetime notation to set YYYYMMDD and YYYYMM…
georgemccabe Oct 13, 2022
75375b9
Removed rest of logic that reads filename type since we now require t…
georgemccabe Oct 14, 2022
d0cc48c
improved list to string logic by handling if quotes already exist whe…
georgemccabe Oct 14, 2022
3e8b832
made MODEL<n>_OBTYPE optional so users can filter without obtype, rem…
georgemccabe Oct 14, 2022
acc04d3
exit non-zero and print an error if any warnings/errors occur when ru…
georgemccabe Oct 14, 2022
9c44e40
added glossary entry for STAT_ANALYSIS_JOB<n> variable and cleaned up…
georgemccabe Oct 14, 2022
926d231
added info about setting groups of fcst leads
georgemccabe Oct 14, 2022
867783f
added info specific to StatAnalysis wrapper its Description section a…
georgemccabe Oct 14, 2022
fe3c2a0
added info about additional filename template tags that can be used i…
georgemccabe Oct 14, 2022
6b30b6b
added info to note that full day of end time will be processed if onl…
georgemccabe Oct 14, 2022
685d25e
Per #1687, removed LOOP_ORDER from all config files
georgemccabe Oct 14, 2022
e978a51
Per #1687, removed LOOP_ORDER from default config
georgemccabe Oct 14, 2022
c5061ab
added missing rst characters
georgemccabe Oct 14, 2022
6cf90ef
Merge branch 'develop' into feature_1842_stat_analysis_multiple_jobs
georgemccabe Oct 17, 2022
39e2c41
moved utilities to parse time lists from strings to time_util, cleane…
georgemccabe Oct 17, 2022
1ac8645
moved functions to handle thresholds to string_manip utility
georgemccabe Oct 17, 2022
c91e299
cleaned up logic to remove pylint complaints and be consistent
georgemccabe Oct 17, 2022
665cf09
turn off all use case tests
georgemccabe Oct 17, 2022
b563d86
check that changing logic to not set beg or end based on hour list do…
georgemccabe Oct 19, 2022
86b2e42
Merge branch 'develop' into feature_1842_stat_analysis_multiple_jobs
georgemccabe Oct 20, 2022
e871d50
change logic to never set fcst/obs_init/valid__beg/end values based o…
georgemccabe Oct 20, 2022
653b04c
pass runtime settings to set_environment_variables function so that s…
georgemccabe Oct 20, 2022
5f25e89
added logic to substitute string template tags when reading explicit …
georgemccabe Oct 20, 2022
2462db3
started adding tests to ensure fcst/obs_init/valid_beg/end values are…
georgemccabe Oct 20, 2022
6c9a627
added more tests for setting beg/end values in wrapped MET config
georgemccabe Oct 20, 2022
437e6d6
removed setting of legacy environment variables that were set to supp…
georgemccabe Oct 20, 2022
a7ef6ef
updating docs with info about new settings
georgemccabe Oct 20, 2022
9d74886
added rest of new config variables to glossary and fixed formatting t…
georgemccabe Oct 20, 2022
f1a6892
allow now and today keywords in string template substitution for Stat…
georgemccabe Oct 21, 2022
1521c09
Refactored the logic to build the string substitution dictionary and …
georgemccabe Oct 24, 2022
e7f79e2
set default value of variables that may not be set and removed variab…
georgemccabe Oct 24, 2022
b9a1aa6
apply shift to relativedelta objects
georgemccabe Oct 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/parm/pytest_groups.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,5 @@ wrapper
wrapper_a
wrapper_b
wrapper_c
wrapper_d
plotting_or_long
403 changes: 272 additions & 131 deletions docs/Users_Guide/glossary.rst
View file
Open in desktop

Large diffs are not rendered by default.

5 changes: 0 additions & 5 deletions docs/Users_Guide/installation.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,11 +113,6 @@ to run.

- netCDF4 (1.5.4)

- MakePlots wrapper

- cartopy (0.20.3)
- pandas (1.4.3)

- CyclonePlotter wrapper

- cartopy (0.20.3)
Expand Down
2 changes: 1 addition & 1 deletion docs/Users_Guide/systemconfiguration.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1116,7 +1116,7 @@ paths, and more. The value of each list item can be referenced in the
METplus configuration variables by using {custom?fmt=%s}. The variable
CUSTOM_LOOP_LIST will apply the values to each wrapper in the PROCESS_LIST
unless the wrapper does not support this functionality. CyclonePlotter,
MakePlots, SeriesByInit, SeriesByLead, StatAnalysis, TCStat, and
StatAnalysis, TCStat, and
TCMPRPlotter wrappers are not supported. If the variable is not set or set
to an empty string, the wrapper will execute as normal without additional
runs. The name of the wrapper-specific variables contain the name of the
Expand Down
Loading