pb2nc generating many exactly 5000 m PBL heights #1140
-
|
Hi, |
Beta Was this translation helpful? Give feedback.
Replies: 6 comments 13 replies
-
|
The pb2nc resets PBL values to 5000 if it's bigger than 5000. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @hsoh-u, here's the line of code that does that check with MAX_PBL = 5000 being the upper limit: @gsketefian, as we discussed in an email thread with @jwolff-ncar, I pulled down some sample data and did more digging. Using an input NDAS file from Hera (prepbufr.ndas.2019053000) and deriving PBL in PB2NC results in 54 of 136 PBL values being computed as 5000. Here's a histogram showing this result: Running pb2nc at verbosity level 7 (-v 7) produces lots of useful info about the derivation of PBL in this PBL derivation results in two types of warning messages:
While values too low are discarded, values too high are just reset to 5000. I recommend that we modify this logic and also discard the large values. I tested with a modified version of PB2NC and here's a much more reasonable histogram of computed PBL values: However, I'd like @PerryShafran-NOAA to weigh in. The PBL derivation logic was ported over from NOAA/EMC verification code. @PerryShafran-NOAA is resetting large values of PBL to 5000 a useful feature? Or should we instead just discard those obs? Note that the vast majority of the values that are reset are ridiculous and certainly not meaningful: Does this indicate some fault in the PBL derivation algorithm? Or is this expected behavior? @gsketefian while we sort out the details, there is an easy workaround you can use. In your Point-Stat configuration file, you can censor the PBL observations using: This replaces all instances of 5000 with MET's internal bad data value (-9999). And those values will no longer be included in the verification. Unfortunately this does require rerunning Point-Stat. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @JohnHalleyGotway and @hsoh-u for following up. Appreciate the work-around while this behavior is discussed further with @PerryShafran-NOAA to decide on a final fix. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, thanks @JohnHalleyGotway and @hsoh-u for the answers. Let's see what @PerryShafran-NOAA says and go from there. |
Beta Was this translation helpful? Give feedback.
-
|
I think I might want to check with the team to see if there should be a max level of PBL height. 5000 m is a real physical value, and in some very stable regions, PBL height can get even above that. I'm not entirely sure if there is a max value in the models, but I can also check that out. I'm not certain of why there was a 5000 m max in the VSDB code, I wasn't the author of that, I just took that as face value. Let me confirm to see if there is a current max and what the code should do with those. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for taking a look @PerryShafran-NOAA. Please let me know what you find. @hsoh-u from a software perspective, I'm concerned about some pretty clearly bad data values being computed. Listed below is some debug level 8 log output from PB2NC. For station number 72274, the computed PBL value, 3.68935e+19, is rejected because its greater than 5000. But we see that that value comes from the first input level... and is clearly not physical. I wonder where these ridiculous values are coming from, and wonder if there's any additional checks we should add to interpret them as bad data? |
Beta Was this translation helpful? Give feedback.
-
|
I agree that 5000 m is too low to safely claim anything at that height or higher is invalid. I'd feel comfortable with ~7000 m, but again, this comes back to how the astronomically high PBL values are getting created in the first place. @JohnHalleyGotway, does MET need to be rebuilt to change MAX_PBL, or can that be set at run time? |
Beta Was this translation helpful? Give feedback.
-
|
@JeffBeck-NOAA the MAX_PBL value is hard-coded on this line: Changing it from 5000 to some other value, like 7000, would require that MET be recompiled. At this point, I see three relevant questions that need to be addressed: (1) @hsoh-u and @PerryShafran-NOAA: How can we detect and handle bad input values (like level 0 height = 3.68935e+19, in the example above)? Am I right in assuming the resulting PBL should be bad data rather than 5000? (2) @JeffBeck-NOAA and @PerryShafran-NOAA: Should the max PBL value (currently 5000) be modified? And should values above the max be reset to that max value (current behavior) or reset to bad data? (3) Should we make the definition of that max PBL a configurable option instead of hard coding it? If so, consider using the existing censor_thresh and censor_val options to define how to censor the output observations. Something like this? Defining it in this general way would give users a lot of flexibility in censoring the observation data. If this is worthwhile that'd require a new GitHub issue. |
Beta Was this translation helpful? Give feedback.
-
|
@JohnHalleyGotway, regarding 2) Yes, 5000 m should not be the max PBL value. If bad values are always 3e19 or something huge, then setting a max PBL of, say, 10000m would be fine, and would capture all valid values of PBL. Then values above that should be thrown out. 5000 m is just too low of a threshold. Regarding 3), I like the idea of allowing users flexibility to set their PBL height max; however, if the difference between a good and "bad" PBL height value is huge, then there's no need to do this. |
Beta Was this translation helpful? Give feedback.
-
|
I am asking the verification team thoughts about max PBL height. We are waiting for the relevant people to return from vacation to help give us an answer. But I think that there are realistic PBL heights above 5000 in certain regimes. We would also be limited by a max value in the UPP, which I'm not entirely sure if there is a threshold there. We will have to conform to what would be in our UPP has in order to be consistent. |
Beta Was this translation helpful? Give feedback.
-
|
@PerryShafran-NOAA @JohnHalleyGotway @hsoh-u We (I, @jwolff-ncar, and @JeffBeck-NOAA, and others who are part of the DTC's R2O project) are looking to rerun our verification tasks with a modification to account for the 5000m HPBL bug. Since we need to do that pretty soon, we are trying to decide on how to proceed. The options are:
@PerryShafran-NOAA, just wondering if you have an estimate on when the relevant people will be able to take a look at the code. Thanks. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, there, it seems to me that our validation folks seem to agree that 5000 m is too low a cap, but I'm trying to figure out whether they want some sort of cap at all, or just to throw out the unrealistic values. I'll try to get an answer ASAP. For the unrealistic values, it would be useful to plot out all the PBL components at every level and find the unrealistic value in there. I would be willing to assist. It should be noted that EMC does not agree that we should allow users to select their own level to throw out PBL data. |
Beta Was this translation helpful? Give feedback.
-
|
Confirmed: set the max level to 10,000 m, and that should capture the physical PBL heights. Anything about that should be flagged and thrown out. |
Beta Was this translation helpful? Give feedback.
-
|
@PerryShafran-NOAA thanks for the direction. Two questions for you... (1) @hsoh-u noticed in this example, that the first level has a pressure of 0.00000. We assume that we should treat any input levels with a pressure value of 0 as being bad data. Can you confirm that's correct? And if so, @hsoh-u please write up a GitHub issue for this. (2) For the maximum value of 10,000, and resetting anything above that to bad data, do you think we should hard-code that upper limit? Or would you prefer that we handle it via the PB2NC configuration file by adding a new option for censoring point obsrevations values: I suspect that we're likely to encounter issues similar to this in the future. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, @JohnHalleyGotway ,
Thanks! Perry |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for clarifying Perry. I migrated this direction over to a new METplus issue #1913. @hsoh-u will work on this for MET version 10.1.0. I also assigned Perry to that issue for him to test the result. And I'll select one of the responses above as the "Answer" to this discussion. |
Beta Was this translation helpful? Give feedback.
-
|
@PerryShafran-NOAA @JohnHalleyGotway @hsoh-u Couple of questions:
Would appreciate any feedback. Thanks. |
Beta Was this translation helpful? Give feedback.
-
|
It is potentially likely that those PBL heights over 5000 m are real, as it is possible to have a PBL value more than 5000 m. It would be good to know what those PBL values those are that were reset to 5000 m. Also, our radiosondes generally do not measure all the way up to 0 hPa, so if that is the pressure, then it's likely a bad observation. Looking at that level, none of the observations in that level are reasonable. Therefore, if it's just one level, you could throw away the level of that observation and calculate the PBL using all the other levels with reasonable data. |
Beta Was this translation helpful? Give feedback.
-
|
Please note that this discussion was moved over to a MET development issue: |
Beta Was this translation helpful? Give feedback.
Thanks @hsoh-u, here's the line of code that does that check with MAX_PBL = 5000 being the upper limit:
https://github.com/dtcenter/MET/blob/fb666bc3289e2bf49ccfc85e716ab812e10a7628/met/src/tools/other/pb2nc/pb2nc.cc#L3184
@gsketefian, as we discussed in an email thread with @jwolff-ncar, I pulled down some sample data and did more digging.
Using an input NDAS file from Hera (prepbufr.ndas.2019053000) and deriving PBL in PB2NC results in 54 of 136 PBL values being computed as 5000. Here's a histogram showing this result:
Running pb2nc at verbosity level 7 (-v 7) produces lots of useful info about the derivation of PBL in this
run_pb2nc.log file.
PBL derivation results in two types of war…