Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 1809 gen prob #1967

Merged
merged 28 commits into from
Nov 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
2a159e1
Per #1809, define the ProbGenInfo class and update the Makefile to co…
JohnHalleyGotway Nov 5, 2021
8b8fc96
Per #1809, update the ProbInfoArray class to store arrays of both Pro…
JohnHalleyGotway Nov 5, 2021
661b2e7
Per #1809, since tc_pairs processes probabilities of RIRW and not gen…
JohnHalleyGotway Nov 5, 2021
79371ca
Per #1809, this is definitely a work in progress. TC-Gen parses the g…
JohnHalleyGotway Nov 5, 2021
0c52234
Per #1809, store the distance to land in the ProbInfoBase class and u…
JohnHalleyGotway Nov 5, 2021
88ec38e
Per #1809...
JohnHalleyGotway Nov 5, 2021
78c930a
Per #1809, work in progress renaming tc_gen functions in a more consi…
JohnHalleyGotway Nov 9, 2021
9929924
Per #1809, make the genesis probability thresholds configurable.
JohnHalleyGotway Nov 9, 2021
475c96a
Per #1809, add ProbInfoArray::n_technique() member function to count …
JohnHalleyGotway Nov 11, 2021
c52c899
Per #1809, update PCTInfo to add the set_fthresh() function and expan…
JohnHalleyGotway Nov 11, 2021
ca8506c
Per #1809, update TC-Gen to add options for probabilistic outputs whe…
JohnHalleyGotway Nov 11, 2021
5236918
Per #1809, switch to consistently naming classes related to the proba…
JohnHalleyGotway Nov 11, 2021
aa858f3
Per #1809, saving off the current state of development before proceed…
JohnHalleyGotway Nov 12, 2021
4141adb
Per #1809, keep track of the min/max BEST track genesis times, and we…
JohnHalleyGotway Nov 12, 2021
3b0a2df
Per #1809, make access to ProbGenInfoArray elements non-constant so t…
JohnHalleyGotway Nov 12, 2021
42d6fad
Per #1809, Update the ProbGenInfo class to store a pointer to the mat…
JohnHalleyGotway Nov 12, 2021
5c2575e
Per #1809, this is a good stopping point. I have tc-gen accurately ve…
JohnHalleyGotway Nov 12, 2021
bfbf570
Per #1809, have to link to vx_nav to call gc_dist().
JohnHalleyGotway Nov 12, 2021
d9f4382
Per #1809, add PROB_LEAD and PROB_VAL to the GENMPR line type.
JohnHalleyGotway Nov 13, 2021
17b1323
Per #1809, add PROB_LEAD and PROB_VAL to the GENMPR line type.
JohnHalleyGotway Nov 13, 2021
9f834d1
Per #1809, add GenesisLead to the ProbGenInfo class to track the lead…
JohnHalleyGotway Nov 13, 2021
9a722e1
Per #1809, update tc_gen to write PROB_GENESIS matched pair output to…
JohnHalleyGotway Nov 13, 2021
b9dd9f0
Per #1809, stop storing a pointer to the BEST genesis info object in …
JohnHalleyGotway Nov 13, 2021
ac1045c
Per #1809, use ProbGenPCTInfo to store pointers to the individual mat…
JohnHalleyGotway Nov 13, 2021
540a4f4
Per #1809, add a call to tc_gen in unit_tc_gen.xml.
JohnHalleyGotway Nov 13, 2021
effa629
Per #1809, update the tc_gen chapter of the user's guide.
JohnHalleyGotway Nov 15, 2021
7bc3d70
Per #1809, fix typo in docs.
JohnHalleyGotway Nov 15, 2021
a18b069
Per #1809, doc updates.
JohnHalleyGotway Nov 15, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion met/data/config/TCGenConfig_default
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,11 @@ ops_method_flag = TRUE;
//
////////////////////////////////////////////////////////////////////////////////

//
// Probability of genesis thresholds
//
prob_genesis_thresh = ==0.25;

//
// Confidence interval alpha value
//
Expand All @@ -222,8 +227,12 @@ ci_alpha = 0.05;
//
output_flag = {
fho = NONE;
ctc = BOTH;
ctc = BOTH;
cts = BOTH;
pct = NONE;
pstd = NONE;
pjc = NONE;
prc = NONE;
genmpr = NONE;
}

Expand Down
2 changes: 1 addition & 1 deletion met/data/table_files/met_header_columns_V10.1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ V10.1 : STAT : SSVAR : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID
V10.1 : STAT : VAL1L2 : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL UFABAR VFABAR UOABAR VOABAR UVFOABAR UVFFABAR UVOOABAR
V10.1 : STAT : VL1L2 : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL UFBAR VFBAR UOBAR VOBAR UVFOBAR UVFFBAR UVOOBAR F_SPEED_BAR O_SPEED_BAR
V10.1 : STAT : VCNT : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL FBAR FBAR_BCL FBAR_BCU OBAR OBAR_BCL OBAR_BCU FS_RMS FS_RMS_BCL FS_RMS_BCU OS_RMS OS_RMS_BCL OS_RMS_BCU MSVE MSVE_BCL MSVE_BCU RMSVE RMSVE_BCL RMSVE_BCU FSTDEV FSTDEV_BCL FSTDEV_BCU OSTDEV OSTDEV_BCL OSTDEV_BCU FDIR FDIR_BCL FDIR_BCU ODIR ODIR_BCL ODIR_BCU FBAR_SPEED FBAR_SPEED_BCL FBAR_SPEED_BCU OBAR_SPEED OBAR_SPEED_BCL OBAR_SPEED_BCU VDIFF_SPEED VDIFF_SPEED_BCL VDIFF_SPEED_BCU VDIFF_DIR VDIFF_DIR_BCL VDIFF_DIR_BCU SPEED_ERR SPEED_ERR_BCL SPEED_ERR_BCU SPEED_ABSERR SPEED_ABSERR_BCL SPEED_ABSERR_BCU DIR_ERR DIR_ERR_BCL DIR_ERR_BCU DIR_ABSERR DIR_ABSERR_BCL DIR_ABSERR_BCU
V10.1 : STAT : GENMPR : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL INDEX STORM_ID AGEN_INIT AGEN_FHR AGEN_LAT AGEN_LON AGEN_DLAND BGEN_LAT BGEN_LON BGEN_DLAND GEN_DIST GEN_TDIFF INIT_TDIFF DEV_CAT OPS_CAT
V10.1 : STAT : GENMPR : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL INDEX STORM_ID PROB_LEAD PROB_VAL AGEN_INIT AGEN_FHR AGEN_LAT AGEN_LON AGEN_DLAND BGEN_LAT BGEN_LON BGEN_DLAND GEN_DIST GEN_TDIFF INIT_TDIFF DEV_CAT OPS_CAT
V10.1 : STAT : SSIDX : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE FCST_MODEL REF_MODEL N_INIT N_TERM N_VLD SS_INDEX

V10.1 : MODE : OBJ : VERSION MODEL N_VALID GRID_RES DESC FCST_LEAD FCST_VALID FCST_ACCUM OBS_LEAD OBS_VALID OBS_ACCUM FCST_RAD FCST_THR OBS_RAD OBS_THR FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE OBJECT_ID OBJECT_CAT CENTROID_X CENTROID_Y CENTROID_LAT CENTROID_LON AXIS_ANG LENGTH WIDTH AREA AREA_THRESH CURVATURE CURVATURE_X CURVATURE_Y COMPLEXITY INTENSITY_10 INTENSITY_25 INTENSITY_50 INTENSITY_75 INTENSITY_90 INTENSITY_USER INTENSITY_SUM CENTROID_DIST BOUNDARY_DIST CONVEX_HULL_DIST ANGLE_DIFF ASPECT_DIFF AREA_RATIO INTERSECTION_AREA UNION_AREA SYMMETRIC_DIFF INTERSECTION_OVER_AREA CURVATURE_RATIO COMPLEXITY_RATIO PERCENTILE_INTENSITY_RATIO INTEREST
Expand Down
143 changes: 96 additions & 47 deletions met/docs/Users_Guide/tc-gen.rst

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions met/internal_tests/libcode/vx_tc_util/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ test_read_LDADD = -lvx_stat_out \
-lvx_config \
-lvx_gsl_prob \
-lvx_cal \
-lvx_nav \
-lvx_util \
-lvx_math \
-lvx_color \
Expand Down Expand Up @@ -75,6 +76,7 @@ test_read_prob_LDADD = -lvx_stat_out \
-lvx_config \
-lvx_gsl_prob \
-lvx_cal \
-lvx_nav \
-lvx_util \
-lvx_math \
-lvx_color \
Expand Down
2 changes: 1 addition & 1 deletion met/internal_tests/libcode/vx_tc_util/test_read_prob.cc
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ int main(int argc, char *argv[]) {
while(f >> p_line) {

// Add the current line to the array of probs
probs.add(p_line);
probs.add(p_line, bad_data_double, false);

// Increment the line count
count++;
Expand Down
1 change: 1 addition & 0 deletions met/src/basic/vx_config/config_constants.h
Original file line number Diff line number Diff line change
Expand Up @@ -1097,6 +1097,7 @@ static const char conf_key_ops_hit_window[] = "ops_hit_window";
static const char conf_key_discard_init_post_genesis_flag[] = "discard_init_post_genesis_flag";
static const char conf_key_dev_method_flag[] = "dev_method_flag";
static const char conf_key_ops_method_flag[] = "ops_method_flag";
static const char conf_key_prob_genesis_thresh[] = "prob_genesis_thresh";
static const char conf_key_fcst_fy_oy[] = "fcst_fy_oy";
static const char conf_key_fcst_fy_on[] = "fcst_fy_on";
static const char conf_key_fcst_tracks[] = "fcst_tracks";
Expand Down
1 change: 1 addition & 0 deletions met/src/basic/vx_util/stat_column_defs.h
Original file line number Diff line number Diff line change
Expand Up @@ -320,6 +320,7 @@ static const char * ssidx_columns [] = {

static const char * genmpr_columns [] = {
"TOTAL", "INDEX", "STORM_ID",
"PROB_LEAD", "PROB_VAL",
"AGEN_INIT", "AGEN_FHR",
"AGEN_LAT", "AGEN_LON", "AGEN_DLAND",
"BGEN_LAT", "BGEN_LON", "BGEN_DLAND",
Expand Down
21 changes: 21 additions & 0 deletions met/src/libcode/vx_statistics/met_stats.cc
Original file line number Diff line number Diff line change
Expand Up @@ -2424,6 +2424,27 @@ void PCTInfo::allocate_n_alpha(int i) {

////////////////////////////////////////////////////////////////////////

void PCTInfo::set_fthresh(const ThreshArray &ta) {

// Expand the probability thresholds, as needed
fthresh = string_to_prob_thresh(ta.get_str().c_str());

// Validate the threshold settings
check_prob_thresh(fthresh, true);

// Store the values in an array
NumArray prob_vals;
for(int i=0; i<fthresh.n(); i++) prob_vals.add(fthresh[i].get_value());

// Set the PCT size and thresholds
pct.set_size(prob_vals.n() - 1);
pct.set_thresholds(prob_vals.vals());

return;
}

////////////////////////////////////////////////////////////////////////

void PCTInfo::compute_stats() {

total = pct.n();
Expand Down
1 change: 1 addition & 0 deletions met/src/libcode/vx_statistics/met_stats.h
Original file line number Diff line number Diff line change
Expand Up @@ -546,6 +546,7 @@ class PCTInfo {

void clear();
void allocate_n_alpha(int);
void set_fthresh(const ThreshArray &);
void compute_stats();
void compute_ci();
};
Expand Down
1 change: 1 addition & 0 deletions met/src/libcode/vx_tc_util/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ libvx_tc_util_a_SOURCES = \
prob_info_base.h prob_info_base.cc \
prob_info_array.h prob_info_array.cc \
prob_rirw_info.h prob_rirw_info.cc \
prob_gen_info.h prob_gen_info.cc \
prob_rirw_pair_info.h prob_rirw_pair_info.cc \
genesis_info.cc genesis_info.h \
pair_data_genesis.cc pair_data_genesis.h \
Expand Down
15 changes: 13 additions & 2 deletions met/src/libcode/vx_tc_util/atcf_offsets.h
Original file line number Diff line number Diff line change
Expand Up @@ -100,24 +100,35 @@ static const int GenMax700VortOffset = 30;
//

static const int ProbOffset = 8; // probability of event (0-100)
static const int ProbItemOffset = 9; // intensity change for event
static const int ProbItemOffset = 9; // RI: intensity change, GN: time window

//
// Offsets specific to ATCF RIRW line type
// Offsets specific to ATCF EDECK RI line type
//

static const int ProbRIRWValueOffset = 10; // final intensity
static const int ProbRIRWInitialsOffset = 11; // forecaster initials
static const int ProbRIRWBegOffset = 12; // RIRW start time
static const int ProbRIRWEndOffset = 13; // RIRW stop time

//
// Offsets specific to ATCF EDECK GN line type
//

static const int ProbGenInitialsOffset = 10; // forecaster initials
static const int ProbGenOrDisOffset = 11; // genesis or dissipation:
// invest, genFcst, genesis,
// disFcst, dissipate
static const int ProbGenTimeOffset = 12; // forecast genesis time

//
// Minimum number of required elements
//

static const int MinATCFTrackElements = 8;
static const int MinATCFGenTrackElements = 9;
static const int MinATCFProbRIRWElements = 14;
static const int MinATCFProbGNElements = 13;

////////////////////////////////////////////////////////////////////////

Expand Down
6 changes: 5 additions & 1 deletion met/src/libcode/vx_tc_util/atcf_prob_line.cc
Original file line number Diff line number Diff line change
Expand Up @@ -124,8 +124,12 @@ int ATCFProbLine::read_line(LineDataFile * ldf) {
n_expect = MinATCFProbRIRWElements;
break;

case ATCFLineType_ProbGN:
n_expect = MinATCFProbGNElements;
break;

default:
mlog << Debug(4)
mlog << Debug(10)
<< "ATCFProbLine::read_line(LineDataFile * ldf) -> "
<< "skipping ATCF line type ("
<< atcflinetype_to_string(Type) << ")\n";
Expand Down
Loading