Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal: Enhance automated testing to properly test main branches #2336

Closed
22 tasks
georgemccabe opened this issue Sep 5, 2023 · 1 comment
Closed
22 tasks
Assignees
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue component: CI/CD Continuous integration and deployment issues component: testing Software testing issue priority: blocker Blocker requestor: METplus Team METplus Development Team required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone type: enhancement Improve something that it is currently doing
Milestone

Comments

@georgemccabe
Copy link
Collaborator

Currently testing is robust for the develop branches. Changes to the develop branch of METplus components such as MET, METdataio, METplotpy, and METcalcpy trigger a METplus use case test run for its develop branch.

Describe the Enhancement

Enhance the testing logic to trigger the default main_vX.Y METplus branch when changes to the components' corresponding main branches occur. It would also be nice to improve the Docker build logic to automatically update the METplus image with any bugfixes applied to the components (or at least MET).

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

  • Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Repository and/or Organization level Project(s) or add alert: NEED CYCLE ASSIGNMENT label
  • Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Add any new Python packages to the METplus Components Python Requirements table.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issues
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@georgemccabe georgemccabe added type: enhancement Improve something that it is currently doing alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Sep 5, 2023
@georgemccabe georgemccabe added this to the Consider for Next Release milestone Sep 5, 2023
@georgemccabe georgemccabe moved this from 🩺 Needs Triage to 🏗 In progress in METplus-Wrappers-6.0.0 Development Nov 15, 2024
@georgemccabe georgemccabe added component: testing Software testing issue priority: blocker Blocker component: CI/CD Continuous integration and deployment issues requestor: METplus Team METplus Development Team required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone and removed alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Nov 15, 2024
@georgemccabe georgemccabe self-assigned this Nov 15, 2024
georgemccabe added a commit to dtcenter/METplotpy that referenced this issue Nov 15, 2024
…e tests when changes are pushed to main_vX.Y and develop branches
georgemccabe added a commit to dtcenter/METcalcpy that referenced this issue Nov 15, 2024
…e tests when changes are pushed to main_vX.Y and develop branches
georgemccabe added a commit to dtcenter/METcalcpy that referenced this issue Nov 15, 2024
…e tests when changes are pushed to main_vX.Y and develop branches
georgemccabe added a commit to dtcenter/METdataio that referenced this issue Nov 15, 2024
…e tests when changes are pushed to main_vX.Y and develop branches
georgemccabe added a commit to dtcenter/METdataio that referenced this issue Nov 15, 2024
…e tests when changes are pushed to main_vX.Y and develop branches
georgemccabe added a commit to dtcenter/METplotpy that referenced this issue Nov 15, 2024
…e tests when changes are pushed to main_vX.Y and develop branches
@georgemccabe
Copy link
Collaborator Author

This was completed by creating a custom GitHub Action called metplus-action-trigger-use-cases that properly handles triggering of METplus use case tests for both main_vX.Y and develop branches. The above commits show where this was added for the main_vX.Y and develop branches for METdataio, METcalcpy, and METplotpy.

This was added for MET in commits 4e6a889 (develop) and 30bf4fb (main_v12.0).

@github-project-automation github-project-automation bot moved this from 🏗 In progress to 🏁 Done in METplus-Wrappers-6.0.0 Development Nov 15, 2024
bikegeek added a commit to dtcenter/METdataio that referenced this issue Dec 13, 2024
* Update version

* Per dtcenter/METplus#2336, call custom GHA to trigger METplus use case tests when changes are pushed to main_vX.Y and develop branches

* Update keep_all_mpr_col to keep_all_cols now that DMAP linetype is also supported

* Added test for reformatting the DMAP (grid_stat) linetype for supporting scatter plots

* Test config file for reformatted DMAP data used for scatter plot

* Replace keep_all_mpr_cols with keep_all_cols now that DMAP linetype can be used for scatter plots, line plots, and contour plots

* update keep_all_mpr_cols to keep_all_cols

* update the comment

* Removed debug print statements

* Added test for reformatted DMAP output for line and contour plots

* DMAP sample data

* Add values for DMAP reformatting

* Config file for testing reformatting DMAP for line and contour plot support

* Example data and reformatted data for stat_name/stat_value/confidence level reformatting

* Refactored version of original.  Content needed for matched pair data, TCMPR data, and aggregation statistics data

* Added content for ranked data values, by all common stat+linetype-specific columns. Made some edits.

* New example data

* Clean up formatting and wording, added content for the TCDIAG reformatting

* Examples for TCDIAG reformatting

* Delete METreformat/test/data/ensemble_stat/ensemble_stat_20100101_120000V.stat

no longer relevant due to additional columns introduced in one of the beta releases

* Updated the data to the latest version of ECNT (includes additional columns)

* Accidentally removed this input file

* Added DMAP support

* Updated for new ensemble stat ECNT

* DMAP reformatting support

* change aggregation status in config file

* Update the ECNT/ensemble stat tests with latest data and make testing less reliant on hard-coding.

* Add better explanation for the input_stats_aggregated setting

* update counts for the ensemble stat data directory data: orank from 1426 to 1921 in new data, phist 18 from 9.

* updated values for line_data_phist_bin from 180 to 340 and line_data_rhist_rank from 84 to 102

* Delete METreformat/test/data/ensemble_stat/ensemble_stat_20100101_120000V.stat

no longer valid/out of date

* fix numbe of line_data_phist_bin to 360

* update line_data_rhist_rank

* Update ECNT aggregation reformatting test to not use hard-coded values

* Remove use of named tuple now that test has been simplified

* update the yaml file to use keep_all_cols and updated DMAP line type support in comment

* Added reformatting information for ECNT

* Example reformatted output for ECNT

* Cleaned up format in 6.3.1 for easier reading.

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested text

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested reformatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested formatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested reformatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested format

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested format/reformatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested formatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested formatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

applying suggested change

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update reformat_stat_data.rst

updated MET tool names from lower case to camel case with '-' in between the two words

* Update reformat_stat_data.rst

Replace text version of table references to links to the MET User's Guide

* Address duplicate explicit target name in links for table 11.1, 12.1, and 13.1

* Capitalizing MET tool names

* Per #348, updated spacing to attempt to fix formatting and update capitalization of MET tools

* Per #348, updated spacing to attempt to fix formatting and update capitalization of MET tools

* Per #348, removing additional space before literal include directive so it will render correctly in RTD

* Per #348, removing additional space before literal include directive so it will render correctly in RTD

* Per #348, adjusting alignment for appropriate alignment in rendering

* Per #348, modified spacing and capitalization

* Fixed some ambiguous text that was meant to be removed or reworded from the previous version of documentation

---------

Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
bikegeek added a commit to dtcenter/METdataio that referenced this issue Dec 13, 2024
* Update version

* Per dtcenter/METplus#2336, call custom GHA to trigger METplus use case tests when changes are pushed to main_vX.Y and develop branches

* Update keep_all_mpr_col to keep_all_cols now that DMAP linetype is also supported

* Added test for reformatting the DMAP (grid_stat) linetype for supporting scatter plots

* Test config file for reformatted DMAP data used for scatter plot

* Replace keep_all_mpr_cols with keep_all_cols now that DMAP linetype can be used for scatter plots, line plots, and contour plots

* update keep_all_mpr_cols to keep_all_cols

* update the comment

* Removed debug print statements

* Added test for reformatted DMAP output for line and contour plots

* DMAP sample data

* Add values for DMAP reformatting

* Config file for testing reformatting DMAP for line and contour plot support

* Example data and reformatted data for stat_name/stat_value/confidence level reformatting

* Refactored version of original.  Content needed for matched pair data, TCMPR data, and aggregation statistics data

* Added content for ranked data values, by all common stat+linetype-specific columns. Made some edits.

* New example data

* Clean up formatting and wording, added content for the TCDIAG reformatting

* Examples for TCDIAG reformatting

* Delete METreformat/test/data/ensemble_stat/ensemble_stat_20100101_120000V.stat

no longer relevant due to additional columns introduced in one of the beta releases

* Updated the data to the latest version of ECNT (includes additional columns)

* Accidentally removed this input file

* Added DMAP support

* Updated for new ensemble stat ECNT

* DMAP reformatting support

* change aggregation status in config file

* Update the ECNT/ensemble stat tests with latest data and make testing less reliant on hard-coding.

* Add better explanation for the input_stats_aggregated setting

* update counts for the ensemble stat data directory data: orank from 1426 to 1921 in new data, phist 18 from 9.

* updated values for line_data_phist_bin from 180 to 340 and line_data_rhist_rank from 84 to 102

* Delete METreformat/test/data/ensemble_stat/ensemble_stat_20100101_120000V.stat

no longer valid/out of date

* fix numbe of line_data_phist_bin to 360

* update line_data_rhist_rank

* Update ECNT aggregation reformatting test to not use hard-coded values

* Remove use of named tuple now that test has been simplified

* update the yaml file to use keep_all_cols and updated DMAP line type support in comment

* Added reformatting information for ECNT

* Example reformatted output for ECNT

* Cleaned up format in 6.3.1 for easier reading.

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested text

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested reformatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested formatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested reformatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested format

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

use suggested format/reformatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested formatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested formatting

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested fix

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

apply suggested

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

applying suggested change

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update docs/Users_Guide/reformat_stat_data.rst

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update reformat_stat_data.rst

updated MET tool names from lower case to camel case with '-' in between the two words

* Update reformat_stat_data.rst

Replace text version of table references to links to the MET User's Guide

* Address duplicate explicit target name in links for table 11.1, 12.1, and 13.1

* Capitalizing MET tool names

* Per #348, updated spacing to attempt to fix formatting and update capitalization of MET tools

* Per #348, updated spacing to attempt to fix formatting and update capitalization of MET tools

* Per #348, removing additional space before literal include directive so it will render correctly in RTD

* Per #348, removing additional space before literal include directive so it will render correctly in RTD

* Per #348, adjusting alignment for appropriate alignment in rendering

* Per #348, modified spacing and capitalization

* Fixed some ambiguous text that was meant to be removed or reworded from the previous version of documentation

---------

Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
bikegeek added a commit to dtcenter/METplotpy that referenced this issue Jan 23, 2025
#483)

* Update conf.py

* Per dtcenter/METplus#2336, call custom GHA to trigger METplus use case tests when changes are pushed to main_vX.Y and develop branches

* Bugfix #475 main_v3.0 - make default configs available in package (#477)

* fix version number for rc1

* get version number from verison file

* per #475, add default config files to python package

* prevent GPU failure running in containers by running kaleido with single process option

* Adding In Memoriam

* Removing, superfluous since Github automatically checks

* Feature 480 v3.0.0 (#481)

* updates for coordinated release: version updates, dates, rotating authors

* Clean up text: fix typos, remove duplicates

* Update vulnerabilities.yaml

update gh-action-pip-audit from 1.0.5 to 1.1.0

* Apply suggestions from code review

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Adding space

* Removing, superfluous since vulnerability checks are automatically done

---------

Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>

* Update conf.py

added the version number

* Update conf.py

fixed version number

* Update conf.py

version number needs quotes

* Modified default log level to ERROR to reduce logging messages captured in METviewer

* Revert "Modified default log level to ERROR to reduce logging messages captured in METviewer"

This reverts commit b273d4b.

* Create 2.1.0_casper

* update documentation requirements to include all sphinx packages and version numbers (#411)

* Absolute import instead of relative

for files in metplotpy/contributed/fv3_physics_tend

* Not v 2 anymore

* Hi

* hi

* a

* Update _version.py

* Update conf.py

fixed version

* Delete metplotpy/_version.py

no such file

* Accidentally deleted

---------

Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: bikegeek <minnawin@ucar.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue component: CI/CD Continuous integration and deployment issues component: testing Software testing issue priority: blocker Blocker requestor: METplus Team METplus Development Team required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone type: enhancement Improve something that it is currently doing
Projects
No open projects
Status: 🏁 Done
Development

No branches or pull requests

1 participant