Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 2708 mvmode merge flag default #2713

Merged
merged 5 commits into from
Oct 25, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions data/config/MODEMultivarConfig_default
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@ quilt = FALSE;
//
multivar_logic = "#1 && #2 && #3";

//
// MODE Multivar intensity computation flag
//
multivar_intensity_flag = [FALSE, TRUE, TRUE];

//
// Forecast and observation fields to be verified
//
Expand Down Expand Up @@ -84,6 +89,8 @@ fcst = {
filter_attr_thresh = [];
merge_thresh = >=3.5;
merge_flag = NONE;
multivar_name = "ALPHA_BETA_GAMMA";
multivar_level = "LO";
}
obs = fcst;

Expand Down
4 changes: 3 additions & 1 deletion docs/Users_Guide/mode.rst
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,8 @@ mode configuration file

The default configuration file for the MODE tool, **MODEConfig_default**, can be found in the installed *share/met/config* directory. Another version of the configuration file is provided in *scripts/config*. We encourage users to make a copy of the configuration files prior to modifying their contents. Descriptions of **MODEConfig_default** and the required variables for any MODE configuration file are also provided below. While the configuration file contains many entries, most users will only need to change a few for their use. Specific options are described in the following subsections.

A second default configuration file for the multivar MODE option, **MODEMultivarConfig_default**, is also found in the installed *share/met/config* directory. We encourage users to make a copy of this default configuration file when setting up a multivar configuration prior to modifying content. The two default config files **MODEConfig_default** and **MODEMultivarConfig_default** are similar, with **MODEMultivarConfig_default** having example multivar specific content.

Note that environment variables may be used when editing configuration files, as described in the :numref:`config_env_vars`.

_____________________
Expand Down Expand Up @@ -324,7 +326,7 @@ The **merge_flag** entry controls what type of merging techniques will be applie

• **BOTH** indicates that both techniques should be used.

By default, the double thresholding merging technique is applied.
By default, the double thresholding **THRESH** merging technique is applied in single variable mode. The merging defaults to **NONE** with multivariate mode.

_____________________

Expand Down
83 changes: 70 additions & 13 deletions src/libcode/vx_shapedata/mode_field_info.cc
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

////////////////////////////////////////////////////////////////////////

#include <string.h>

#include "vx_config.h"
#include "vx_data2d.h"
Expand All @@ -20,6 +21,7 @@

#include "mode_field_info.h"

using std::string;

////////////////////////////////////////////////////////////////////////

Expand Down Expand Up @@ -219,32 +221,87 @@ if ( dict->lookup(conf_key_conv_thresh) ) {

}

if ( dict->lookup(conf_key_merge_thresh) ) {

merge_thresh_array = dict->lookup_thresh_array(conf_key_merge_thresh);

}

if ( dict->lookup(conf_key_vld_thresh) ) {

vld_thresh = dict->lookup_double(conf_key_vld_thresh);

}

// for the multivar case, go without parent, and error out if merge flag is
// not set for the individual field
// For the multivar case, more complex logic regarding merge_flag and merge_thresh
// If individual entry has a merge_flag, it must have a merge_thresh (unless merge_flag=NONE)
// If individual entry has no merge_flag, check the parent and go with that flag and thresh
// If individual entry has no merge_flag, but individual entry has a merge_thresh, it's an error

if ( _multivar ) {
// set defaults to no merging
merge_thresh.clear();
merge_flag = MergeType_None;

// pull out the name
string name = var_info->name();

if ( dict->lookup(conf_key_merge_flag, false)) {
merge_flag = int_to_mergetype(dict->lookup_int(conf_key_merge_flag));
// this individual entry has merge_flag
merge_flag = int_to_mergetype(dict->lookup_int(conf_key_merge_flag,
default_dictionary_error_out,
default_dictionary_print_warning,
false));
string merge_name = mergetype_to_string(merge_flag);
if (dict->lookup(conf_key_merge_thresh, false)) {
// the individual entry also has a merge_thresh, this is good
merge_thresh_array = dict->lookup_thresh_array(conf_key_merge_thresh,
default_dictionary_error_out,
default_dictionary_print_warning,
false);
} else {
// get the parent's merge_thresh, just to have something. Error out if the merge_flag is not none
// becAuse that is inconsistent
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// becAuse that is inconsistent
// because that is inconsistent

merge_thresh_array = dict->lookup_thresh_array(conf_key_merge_thresh);

if (merge_flag != MergeType_None) {
mlog << Error << "\nMode_Field_Info::set() -> "
<< "Field:" << name << ". "
<< " When 'merge_flag' is explicitly set, 'merge_thresh' must be explicitly set for multivariate mode\n\n";
exit ( 1 );
}

}
} else {
mlog << Error << "\nMode_Field_Info::set() -> "
<< "'merge_flag' must be explicitly set for all fields with multivariate mode\n\n";
exit ( 1 );
// individual entry does not have a merge_flag, try parent
if ( dict->lookup(conf_key_merge_flag, true)) {
// the parent does have a merge flag
merge_flag = int_to_mergetype(dict->lookup_int(conf_key_merge_flag));
string merge_name = mergetype_to_string(merge_flag);
if (dict->lookup(conf_key_merge_thresh, false)) {
// individual entry has a merge_thresh but no merge_flag, this is not good
mlog << Error << "\nMode_Field_Info::set() -> "
<< "Field:" << name << ". "
<< "When 'merge_flag' is not explicitly set, 'merge_thresh' cannot explicitly set for multivariate mode\n\n";
exit ( 1 );
} else {
// individual entry doesn't have a merge_thresh, parent has a merge_flag
// expect parent to have a merge_thresh
merge_thresh_array = dict->lookup_thresh_array(conf_key_merge_thresh);
if (merge_thresh_array.n() == 0 && merge_flag != MergeType_None) {
// parent has a merge_flag but no merge_thresh
mlog << Error << "\nMode_Field_Info::set() -> "
<< "Field:" << name << ". using parent merge_flag: " << merge_name
<< " Parent has no 'merge_thresh', not allowed in multivariate mode\n\n";
exit ( 1 );
} else {
string thresh_str = merge_thresh_array.thresh()->get_str();
mlog << Debug(2) << "Field:" << name << ". Using parent merge_flag: "
<< merge_name << " and parent merge_thresh:" << thresh_str
<< "\n";
}
}
}
}
} else {
} else {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
} else {
} else {

Minor point. Just aligning braces.

if ( dict->lookup(conf_key_merge_flag, true)) {
merge_flag = int_to_mergetype(dict->lookup_int(conf_key_merge_flag));
}
merge_thresh_array = dict->lookup_thresh_array(conf_key_merge_thresh);
}

filter_attr_map = parse_conf_filter_attr_map(dict);
Expand Down
7 changes: 7 additions & 0 deletions src/tools/core/mode/mode.cc
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ const char * const program_name = "mode";


static const char default_config_filename [] = "MET_BASE/config/MODEConfig_default";
static const char default_multivar_config_filename [] = "MET_BASE/config/MODEMultivarConfig_default";


///////////////////////////////////////////////////////////////////////
Expand Down Expand Up @@ -169,6 +170,12 @@ int met_main(int argc, char * argv [])

if ( config.is_multivar() ) {


// read again, this time with the mvmode defaults, which are slightly different

config.clear();
config.read_config(default_multivar_config_filename, user_config_filename);

// exit inside this method if something is not implemented

config.check_multivar_not_implemented();
Expand Down
11 changes: 8 additions & 3 deletions src/tools/core/mode/mode_exec.cc
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ static const char * cts_str[n_cts] = {"RAW", "OBJECT"};
static const char program_name [] = "mode";

static const char * default_config_filename = "MET_BASE/config/MODEConfig_default";
static const char * default_multivar_config_filename = "MET_BASE/config/MODEMultivarConfig_default";

// took this out of the do_conv_thresh() method
static int local_r_index = -1;
Expand Down Expand Up @@ -143,7 +144,7 @@ void ModeExecutive::clear()
///////////////////////////////////////////////////////////////////////


void ModeExecutive::init(int n_files)
void ModeExecutive::init(int n_files, bool isMultivar)

{

Expand All @@ -152,7 +153,11 @@ void ModeExecutive::init(int n_files)
R_index = T_index = 0;

// Create the default config file name
default_config_file = replace_path(default_config_filename);
if (isMultivar) {
default_config_file = replace_path(default_multivar_config_filename);
} else {
default_config_file = replace_path(default_config_filename);
}

// If the merge config file was not set, use the match config file
if(merge_config_file.length() == 0)
Expand Down Expand Up @@ -226,7 +231,7 @@ void ModeExecutive::init_multivar(GrdFileType ftype, GrdFileType otype)
R_index = T_index = 0;

// Create the default config file name
default_config_file = replace_path(default_config_filename);
default_config_file = replace_path(default_multivar_config_filename);

// If the merge config file was not set, use the match config file
if(merge_config_file.length() == 0)
Expand Down
2 changes: 1 addition & 1 deletion src/tools/core/mode/mode_exec.h
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ class ModeExecutive {

void clear();

void init(int n_files);
void init(int n_files, bool isMultivar);
void init_multivar(GrdFileType ftype, GrdFileType otype);


Expand Down
2 changes: 1 addition & 1 deletion src/tools/core/mode/mode_frontend.cc
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ int ModeFrontEnd::run(const StringArray & Argv, Processing_t ptype, int field_in

process_command_line(Argv, false);

mode_exec->init(n_files);
mode_exec->init(n_files, ptype != SINGLE_VAR);

ModeConfInfo & conf = mode_exec->engine.conf_info;
if ( field_index >= 0 ) conf.set_field_index(field_index);
Expand Down
3 changes: 2 additions & 1 deletion src/tools/core/mode/multivar_frontend.cc
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@
////////////////////////////////////////////////////////////////////////


static const char mode_default_config [] = "MET_BASE/config/MODEConfig_default";
// for multivar mode, this is the default file
static const char mode_default_config [] = "MET_BASE/config/MODEMultivarConfig_default";

static const int dir_creation_mode = 0755;

Expand Down