Update Develop-ref after #1710 #1714
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… User's Guide instead of downloading it and attaching it to the release
…rnal repository such as MET to test to ensure that changes from that repo will break anything in METplus
…if triggered by an external repository such as MET
…commit hash of the push event that triggered in the other repo
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Mrinal Biswas <biswas@ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
* moved logic to handle climo_mean/stdev dictionaries into met_config utility * moved constant variable to constants and made name uppercase * per #1599, added logic to read VAR<n> variables for climo_mean/stdev (i.e. GRID_STAT_CLIMO_MEAN_VAR1_NAME). Added function to easily format field info for a single field or list of fields with fewer arguments * added function to format field info that is easier to call than get_field_info and can eventually replace it * added line breaks to match pep8 standards, ci-run-all-diff * per 1599, added documentation for new config variables * replaced calls to get_field_info with format_field_info since it is much simpler * fixed bug where field list is never returned from function call * removed whitespace * modified logic to report an error if no commands were run at all * only report an error if the process list contains a wrapper that should generate at least 1 command. Example and CyclonePlotter wrappers do not run any shell commands, so they are excluded * per feedback in PR #1658 review, added glossary entry that is wrapper independent and provide reference to that entry instead of duplicating information for each wrapper * per #1653, add unit tests to check that day_interval value of climo_mean/stdev dictionary can be NA or an integer * per #1653, modified function to read climo variables to support NA instead of just integers * per #1653, updated unit tests to test both climo_mean and climo_stdev variables, added tests for all other climo variables
…er/met-dev if the tag is develop, otherwise use dtcenter/met
…. After restructuring the repo, remove the 'met/' prefix from some paths. Also, remove the attach_release_tarfile.rst instructions since that step is no longer required. Technically, its still needed for version 10 bugfix releases, but just removing it now is simpler.
… release was created
* added documentation * added pic, config file, more documentation * modified config file, added to internal tests * attempt to correct docs error * added new tool-specific probability feature * Updated run option in json file
* first pass at documentation, config, and python scripts * 2nd addition of files to use case * updated config file for dir path * Update use_case_groups.json * Removed unused config options
…or METdataio (renamed from METdatadb) to use in automated tests
* changing from 2nd to 3rd person and fixing some spacing #770 * missed we and corrected #770 * typos, spacing etc #770 * spelling, italics and bolding #770 * removing you changing to 3rd person #770 * changing to 3rd person #770 * 3rd person already done. Fixing formatting, typos etc. #770 * 3rd person already done. Fixing formatting #770 * changing to 3rd person, fixing typos, spacing, etc. #770 * changing to 3rd person, fixing typos, grammar and punctuation. #770 * google doc corrections. #770 * clean up #770 * removing | #770 * typo #770 * Fixing TOC and coding block #770 * fixing note #770 * one 3rd person fix. Updating bolding and italics for file names and directories #770 * Modified Python Code Analyis Tools * Modified Python Code Analyis Tools * Removed link for PROCESS_LIST. If we can't have an internal link, we do not want one in this case. * fixing space #770 * fixing conflict #2 #770 * fixing typos #770 * fixing typos #2 #770 * directory formatting #2 #770 * Changed sub process to subprocess * Modified usage of user because this text is for developers. Made other minor changes. * first attempt at grammar, bolding, italics and period clean up #770 * under 79 characters #770 * changing to 3rd person #770 * fixing typos #770 * 3rd person and typos #770 * fixing formatting #770 * typo cleanup #770 * more typo cleanup #770 * adding periods to most bullet lists #770 * adding periods to most bullet lists #770 * changing web address to italics instead of bold, not linking because it is an example. * creating links attempt. * creating links attempt #2. * creating links attempt #3. * fixing typo for add_use_categories * removing warning block per George M. * changing sentence * no screenshots to be added * adding note box * fixing bolding * Made some minor tweaks Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
* per #685, added custom pytest markers for many tests so we can run groups of tests in automation to speed things up. removed unused imports from tests and removed parenthesis from assert statements when they are not needed * per #685, change logic to check if test category is not equal to 'pytest' to does not start with 'pytest' to allow groups of pytests * per #685, run pytests with markers to subset tests into groups * fixed check if string starts with pytests * added missing pytest marker name * added logic to support running all pytests that do not match a given marker with the 'not <marker>' syntax * change pytest group to wrapper because the test expects another test to have run prior to running * fix 'not' logic by adding quotation marks around value * another approach to fixing not functionality for tests * added util marker to more tests * fixed typo in not logic * added util marker to more tests again * fixed logic to split string * marked rest of util tests with util marker * fixed another typo in string splitting logic * tested change that should properly split strings * moved wrapper tests into wrapper directory * changed marker for plotting tests * added plotting marker * improved logic for removing underscore after 'not' and around 'or' to specify more complex marker strings * test running group of 3 markers * fixed path the broke when test file was moved into a lower directory * Changed StatAnalysis tests to use plotting marker because some of the tests involve plotting but the other StatAnalysis tests produce output that are used in the plotting tests * changed some tests from marker 'wrapper' to 'wrapper_a' to split up some of these tests into separate runs * test to see if running pytests in single job but split into groups by marker will improve the timing enough * fixed typos in new logic * removed code that is no longer needed (added comment in issue #685 if this logic is desired in the future) * per #685, divided pytests into smaller groups * added a test that will fail to test that the entire pytest job will fail as expected * add error message if any pytests failed to help reviewer search for failed tests * removed failing test after confirming that entire pytest job properly reports an error if any test fails * turn on single use case group to make sure logic to build matrix of test jobs to run still works as expected * turn off use case after confirming tests were created properly * added documentation to contributor's guide to describe changes to unit test logic * added note about adding new pytest markers
* Per #1402, added additional comments to config file including links to documentation. Added old format version of config file that contain many comments with the _DIR/_TEMPLATE variables grouped together for comparison * rearrange variables and added link to user's guide * updated comment to note that variables in the section are optional * added URL for use case documentation * per #1402, rearrange and clean up config file * added missing config variables from CyclonePlotter to glossary and wrappers doc pages * per #1402, clean up use case conf files, ci-run-all-diff * per #1402, clean up Point2Grid conf file * per #1402, clean up rest of met_tool_wrapper use case conf files, ci-run-all-diff * removed conf file that is no longer needed
* per #1694, moved 4 use cases from s2s to s2s_mjo, updated paths accordingly, and turned on all s2s use cases to test that they all run successfully after the changes * per #1694, fixed paths to s2s_mjo conf files * updated documentation for use cases that were moved from s2s to s2s_mjo * attempt to free up unused disk space in GHA runner environment * moved 4 s2s use cases into s2s_mid_lat * added new model application categories to contrib guide for adding new use cases * per #947, changed convection_allowing_models use cases to short_range * changed which use case tests run to the ones that are failing and added other METdbLoad use case to see if that fails as well * test to determine which files are preventing MySQL database from being created properly * test 2 to determine which files are preventing MySQL database from being created properly * test 3 to see if removing these files is not the cause of the METdbLoad failure * updated references to METdatadb to METdataio since the repository was renamed * fixed typo * changed path to sql file needed to create database because it was moved from METviewer to METdataio * fixed path to sql file that was moved from METviewer to METdataio * removed temporary fix because metdataio conda env was created in the dtcenter/metplus-envs:metdataio Docker image * added note to update path when METviewer Dockerfile changes to reflect METdatadb rename to METdataio, ci-skip-unit-tests * fixed path to METdataio repo * add back commands to free up disk space because issue with METdbLoad use case was likely not related, ci-skip-unit-tests * run all tests with ci-run-all-diff * remove use case group added for testing, ci-skip-all * changed exit code for diff tests to 2 so it is easier to see if a use case test job failed due to an actual failure or due to differences in the output * changed grouping of s2s mid lat use cases to original grouping to prevent warning that artifact contains more than 10,000 files. The 2 WeatherRegime use cases produce a lot of output files, so splitting them up should resolve this warning * per #1694, changed all references to convection allowing models to short range in the Verification Datasets section of the documentation * changed URLs to develop version of documentation to a URL relative to the current version of the documentation to match the quick search links from the METplus User's Guide * per #947, changed references to convection_allowing_model (without the s) to short_range that were missed * updated use case test scripts to rename convection_allowing_models to short_range and added note to alert developers that the list of use cases in the script is not maintained and therefore not complete
* adding new questions section * changing table config #650 * odd section bullets #650 * adding in the rest of the text but not formatted #650 * first attempt at bullet formatting #650 * fixing errors #650 * fixing errors take 2 #650 * fixing errors take 3 #650 * fixing errors take 4 #650 * fixing errors take 5 #650 * Update systemconfiguration.rst Cleaning up "you" and changing to 3rd person. Except where "you" is referenced by a program example #650 * Update systemconfiguration.rst Taking another stab at making the bullets consistent #650 * creating new getting_started file to house Questions to Consider that was removed from the systemconfiguration file. * moving Running METplus into getting_started file and removing it from the systemconfiguration. #650 * updating index to include new Getting Started chapter #650. * updating index to include new Getting Started chapter take 2 #650. * shortening the chapter name to Installation and Configuration #650 * using updated questions #650 * Made a few formatting/wording changes * Removed text * adding flowchart question and adding periods #650 * bolding files #650 * adding changes John O. recommended. #650 * small John O. change #650 * removing duplicate question #650 * fixing question order #650 * fixing typos Tara requested * answering question 2.1.1.7 * adding Use Case Rules link to 2.1.1.7 * adding proper link to the Add Use Cases in the Contributors Guide * trying to get link to work * trying to get link to work take 2 * trying to get link to work take 3 * trying to get link to work take 4 * removing unnecessary link * fixing punctuation Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
* changes needed to create v5.0 version of metplus_base docker/conda environment * update metplotpy conda env to use python 3.8 and new package list, then test to see if use cases that use that env still work using the .v5 version * turn on use cases that use metplotpy_env to test new conda env * add .v5 to metplotpy conda env to test that environment * removed cartopy from requirements because cartopy may no longer be needed * changed conda environment used for use case * update conda before building environments * updated instructions and scripts to build python environments using Python 3.8.6 for METplus v5 * updated conda env to include .v5 * updated automated test logic to use new .v5 Docker images that use Conda environments built from Python 3.8.6 * rebuilt gempak and gfdl-tracker docker images to use .v5 extension to be consistent in the automation logic and avoid special behavior for those environments * added note to update OS packages to prevent wget certificate errors * renamed variables to be more clear which path they are actually storing * Updated logic to no longer handle gempak and gfdl-tracker envs differently by excluding the version number. Cleaned up logic to add version number extension to conda environment names as needed * updated scripts to create Docker image containing Conda environment for METdataio (renamed from METdatadb) to use in automated tests for v5 * turn off all use cases * turn on use cases that previously failed * turn off use cases tests that passed * turn off all use cases * per #1566, added explicit version numbers for each Python package that is installed in the Docker Conda environments used for the automated tests to ensure that the same environment will be generated even if the default version of the package differs from the env used to create them
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR #1710 changed the version of Python used for the use cases to 3.8.6, which caused slight differences in a few use cases. These differences were approved by a use case contact for each.