Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] Split EnsembleStat vx tasks into GenEnsProd and EnsembleStat #809

Merged
merged 45 commits into from
Jun 5, 2023

Commits on May 12, 2023

  1. Configuration menu
    Copy the full SHA
    a46a5af View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f77a8a6 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4298da0 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    627c94f View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    6b2e934 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    203020a View commit details
    Browse the repository at this point in the history
  7. Remove lines in EnsembleStat conf files that are for actions that are…

    … now done in GenEnsProd (to avoid duplication).
    gsketefian committed May 12, 2023
    Configuration menu
    Copy the full SHA
    8926622 View commit details
    Browse the repository at this point in the history
  8. Change input to ensemble mean and probabilistic tasks to come from th…

    …e output of GenEnsProd (instead of EnsembleStat, which no longer generates the necessary outputs).
    gsketefian committed May 12, 2023
    Configuration menu
    Copy the full SHA
    9d9c2c0 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2023

  1. Rename metatasks/tasks for clarity; create new metatasks/tasks to mak…

    …e desired dependencies possible; remove unneeded variables; change vx WE2E test config files to be compatible with aforementioned changes.
    gsketefian committed May 17, 2023
    Configuration menu
    Copy the full SHA
    822b633 View commit details
    Browse the repository at this point in the history
  2. Add config variable VX_APCP_ACCUMS_HH to make the accumulation hours …

    …for which to perform vx user-specifiable.
    gsketefian committed May 17, 2023
    Configuration menu
    Copy the full SHA
    8fb28e7 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2023

  1. Add the experiment configuration parameter VX_FIELDS that specifies w…

    …hich fields or group of fields on which to perform verification.
    gsketefian committed May 18, 2023
    Configuration menu
    Copy the full SHA
    54db250 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d31dac6 View commit details
    Browse the repository at this point in the history
  3. Bug fix in metatask name.

    gsketefian committed May 18, 2023
    Configuration menu
    Copy the full SHA
    d6dfb44 View commit details
    Browse the repository at this point in the history
  4. Move CCPA/APCP metatask to before MRMS and NDAS once since the former…

    … field(s) are usually come first in vx.
    gsketefian committed May 18, 2023
    Configuration menu
    Copy the full SHA
    255a2fc View commit details
    Browse the repository at this point in the history
  5. Remove rocoto verification metatasks from workflow when there are no …

    …fields for those metatasks.
    gsketefian committed May 18, 2023
    Configuration menu
    Copy the full SHA
    b17871b View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    a4db400 View commit details
    Browse the repository at this point in the history
  7. Add new experiment flags RUN_VX_DET and RUN_VX_ENS that specify wheth…

    …er or not to run deterministic and ensemble verification, respectively.
    gsketefian committed May 18, 2023
    Configuration menu
    Copy the full SHA
    a095d42 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    d4941be View commit details
    Browse the repository at this point in the history

Commits on May 19, 2023

  1. Configuration menu
    Copy the full SHA
    27f1fa2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f314366 View commit details
    Browse the repository at this point in the history
  3. Modifications to vx WE2E tests that don't use staged forecasts to get…

    … them to work properly with latest changes.
    gsketefian committed May 19, 2023
    Configuration menu
    Copy the full SHA
    7da364d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    e99d3ba View commit details
    Browse the repository at this point in the history
  5. For consistency with the way other variables are passed into the METp…

    …lus config files (and in order for the final config files to have the correct values), change the way the variable ENSMEM_INDX is passed into METplus config files from an environment variable to a jinja variable.
    gsketefian committed May 19, 2023
    Configuration menu
    Copy the full SHA
    a920213 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    abdd609 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2023

  1. Changes to make vx tests work in NCO mode:

    1) Replace "mem${ensmem_indx}" with new variable "${ensmem_name}".
    2) Remove DOT_ENSMEM_OR_NULL since ensmem_name can now be used instead.
    3) Replace SLASH_ENSMEM_SUBDIR_OR_NULL with slash_ensmem_subdir_or_null.
    4) Remove unneeded code from exregional_run_met_pb2nc_obs.sh.
    gsketefian committed May 23, 2023
    Configuration menu
    Copy the full SHA
    3c2af8d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b534d73 View commit details
    Browse the repository at this point in the history
  3. Add code to allow each element of a list variable to be checked for v…

    …alid values; specify valid sets of values for VX_FIELDS and VX_APCP_ACCUMS_HRS.
    gsketefian committed May 23, 2023
    Configuration menu
    Copy the full SHA
    e6bbad3 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2023

  1. Change argument list to function set_vx_fhr_list() for clarity and to…

    … allow separate max number of missing files for obs vs. forecast.
    gsketefian committed May 25, 2023
    Configuration menu
    Copy the full SHA
    017bc5c View commit details
    Browse the repository at this point in the history
  2. Minor spacing edits.

    gsketefian committed May 25, 2023
    Configuration menu
    Copy the full SHA
    1e908c7 View commit details
    Browse the repository at this point in the history
  3. Split out vx preprocessing tasks that are needed by both deterministi…

    …c and ensemble vx tasks into a new workflow task group file (verify_pre.yaml); add new vx preprocessing metatask (check_post_output) that checks that all post-processed forecast output files exist; fix up vx task dependencies.
    gsketefian committed May 25, 2023
    Configuration menu
    Copy the full SHA
    cf7bff0 View commit details
    Browse the repository at this point in the history
  4. Remove experiment variables RUN_VX_DET and RUN_VX_ENS since their fun…

    …ctionality is replaced by inclusion/exclusion of workflow taskgroup yaml configuration files.
    gsketefian committed May 25, 2023
    Configuration menu
    Copy the full SHA
    72440f5 View commit details
    Browse the repository at this point in the history
  5. If no vx fields are specified, remove the metatask "check_post_output…

    …_all_mems" from the workflow.
    gsketefian committed May 25, 2023
    Configuration menu
    Copy the full SHA
    74975f0 View commit details
    Browse the repository at this point in the history
  6. Clean up check of consistency between DO_ENSEMBLE flag and inclusion …

    …of ensemble vx tasks in the workflow (by making sure that the names of the ensemble vx tasks are read in from the appropriate yaml file instead of being hard-coded).
    gsketefian committed May 25, 2023
    Configuration menu
    Copy the full SHA
    71db846 View commit details
    Browse the repository at this point in the history
  7. Fix the way the time-lag array ENS_TIME_LAG_HRS is set (for both ense…

    …mble and deterministic forecasts) and the way the time lag is calculated for a given ensemble member (or the deterministic forecast).
    gsketefian committed May 25, 2023
    Configuration menu
    Copy the full SHA
    e0b46a8 View commit details
    Browse the repository at this point in the history

Commits on May 26, 2023

  1. Add missing files.

    gsketefian committed May 26, 2023
    Configuration menu
    Copy the full SHA
    c02dfe4 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a304f50 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b8dd481 View commit details
    Browse the repository at this point in the history

Commits on May 31, 2023

  1. Change the test configuration for the WE2E test "MET_ensemble_verific…

    …ation_only_vx_time_lag" to look for the staged forecast data from the default location for the machine; also, change the predefined grid name to the one on which this staged data is provided (this is necessary to form the correct path to the data).
    gsketefian committed May 31, 2023
    Configuration menu
    Copy the full SHA
    77056d2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    76420d6 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    38d7102 View commit details
    Browse the repository at this point in the history
  4. Add forecast lead time (based on nominal cycle start time disregardin…

    …g any time-lagging) to names of GenEnsProd and EnsembleStat output files.
    gsketefian committed May 31, 2023
    Configuration menu
    Copy the full SHA
    1a6bb6d View commit details
    Browse the repository at this point in the history
  5. Bug fix: Change dependence of ensmean and ensprob tasks to be on the …

    …GenEnsProd task, not the EnsembleStat task. Also, change order of metatasks so that the one for MRMS comes before the one for NDAS.
    gsketefian committed May 31, 2023
    Configuration menu
    Copy the full SHA
    3824437 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2023

  1. Configuration menu
    Copy the full SHA
    7d86881 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    141f9e7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    6b163fc View commit details
    Browse the repository at this point in the history