Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for explicitly setting file list file paths in wrappers that support multiple input files #1289

Closed
3 of 21 tasks
Tracked by #1285
hertneky opened this issue Nov 30, 2021 · 2 comments · Fixed by #1387
Closed
3 of 21 tasks
Tracked by #1285
Assignees
Labels
component: use case wrapper MET: Object Verification Object-based feature Verification priority: high High Priority required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone type: enhancement Improve something that it is currently doing
Milestone

Comments

@hertneky
Copy link
Contributor

hertneky commented Nov 30, 2021

Describe the Enhancement

Support the use of file_lists in the METplus wrappers for MET tools that already have that functionality. MET tools that currently accept a file_list as input look to be: ensemble_stat, grid_diag, gsidens2orank, MTD, rmw_analysis, series_analysis, tc_rmw.

Depending on the tool, users should be able to specify either a single file_list and/or a fcst_/obs_file_list to be used within the METplus configuration file . For example the MET MTD tool has either option:

Usage: mtd
-fcst file_1 ... file_n | file_list
-obs file_1 ... file_n | file_list
-single file_1 ... file_n | file_list

This functionality will allow users to run over a specified set of cases that may not be at continuous time intervals. When a file list is provided, the time controls in the configuration would presumably be ignored.

Highest Priority:
series_analysis - file_list overrides timing control
mtd - file_list overrides timing controls
tc_rmw and rmw_analysis - file_list just passes in data - need to check if it overrides timing controls
ensemble_stat - file_list is used to define members of ensemble - will still need timing controls

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

  • Add a checkbox for each sub-issue here.

Relevant Deadlines

4.1.0-beta5

Funding Source

2700042

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
  • Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Linked issues
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@hertneky hertneky added type: enhancement Improve something that it is currently doing component: use case wrapper alert: NEED ACCOUNT KEY Need to assign an account key to this issue MET: Object Verification Object-based feature Verification labels Nov 30, 2021
@hertneky hertneky added this to the METplus-4.1.0 milestone Nov 30, 2021
@hertneky hertneky self-assigned this Nov 30, 2021
@TaraJensen TaraJensen added priority: high High Priority required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project and removed alert: NEED ACCOUNT KEY Need to assign an account key to this issue labels Dec 1, 2021
@georgemccabe georgemccabe added required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone and removed required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project labels Dec 7, 2021
@TaraJensen
Copy link
Contributor

series_analysis
-fcst file_1 ... file_n | fcst_file_list
-obs file_1 ... file_n | obs_file_list
[-both file_1 ... file_n | both_file_list]

tc_rmw
-data file_1 ... file_n | data_file_list

georgemccabe added a commit that referenced this issue Jan 31, 2022
…tlyt set file list file path for SeriesAnalysis
georgemccabe added a commit that referenced this issue Jan 31, 2022
georgemccabe added a commit that referenced this issue Jan 31, 2022
…or TCMRW wrapper -- updated logic to write a file list file since the tc_rmw tool now supports reading a ascii file containing a list of input files
georgemccabe added a commit that referenced this issue Jan 31, 2022
georgemccabe added a commit that referenced this issue Jan 31, 2022
…nsemble members (EnsembleStat and GenEnsProd)
georgemccabe added a commit that referenced this issue Jan 31, 2022
georgemccabe added a commit that referenced this issue Jan 31, 2022
…icit file list to read ensembles for GenEnsProd and EnsembleStat
@georgemccabe georgemccabe linked a pull request Feb 4, 2022 that will close this issue
12 tasks
georgemccabe added a commit that referenced this issue Feb 4, 2022
…t functionality so that output template can reference the forecast lead if looping by lead
@hertneky
Copy link
Contributor Author

hertneky commented Feb 7, 2022

@georgemccabe I tested this enhanced functionality with the series-analysis tool and it worked and provided identical results to using the timing info from the conf.

@georgemccabe georgemccabe changed the title File_lists within METplus wrappers Add support for explicitly setting file lists within METplus wrappers Feb 15, 2022
@georgemccabe georgemccabe changed the title Add support for explicitly setting file lists within METplus wrappers Add support for explicitly setting file list file paths in wrappers that support multiple input files Feb 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: use case wrapper MET: Object Verification Object-based feature Verification priority: high High Priority required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone type: enhancement Improve something that it is currently doing
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants