Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update develop-ref after #2758 #2789

Merged
merged 29 commits into from
Jan 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
ac9f454
Store change of sonar.login to sonar.token.
Nov 20, 2023
c431c85
Store change of sonar.login to sonar.token.
Nov 20, 2023
16c058a
Hotfix to develop, updating aclocal.m4 and config.h.in to what is cre…
Nov 27, 2023
ba0f3b2
Adding -lnetcdf to configure_lib_args for NetCDF-CXX compilation
jprestop Nov 28, 2023
c7e3951
Added eckit and atlas loads and paths
jprestop Nov 29, 2023
60bfc0f
Added atlas and eckit loads and paths
jprestop Nov 29, 2023
0481f05
Changing -j to "-j 5" as the recommended value
jprestop Nov 30, 2023
80bf1c5
Update ensemble-stat.rst
CPKalb Nov 30, 2023
f99f626
Updated values for GRIB2CLIB_NAME and BUFRLIB_NAME
jprestop Dec 4, 2023
9ee3b88
Updated for proj, eckit, and atlas
jprestop Dec 8, 2023
4135e27
Feature #2761 develop seneca (#2762)
JohnHalleyGotway Dec 12, 2023
8f99ef0
Per #2761, define runtime python version for testing rather than usi…
Dec 12, 2023
40ae2fb
Per #2761, fix setting ci-skip-all
Dec 13, 2023
7c6df3c
Per #2761, patching test_util.R to use the -C command line option for…
Dec 13, 2023
59d3afa
#2652 Added find_var_by_standard_name and separated common codes to f…
Dec 13, 2023
2d65071
Merge remote-tracking branch 'origin/develop' into bugfix_2652_polar_CF
Dec 13, 2023
50bcea9
#2757 Get the email list from the environment variable MET_CRON_EMAIL…
Dec 13, 2023
7ff8131
#2757 THe SonarQube token and URL are replaced by ujsing the environm…
Dec 13, 2023
dde1255
#2757 The SonarQube token and URL are replaced with the pre-defined s…
Dec 13, 2023
ea9e543
Merge pull request #2764 from dtcenter/bugfix_2652_polar_CF
hsoh-u Dec 14, 2023
c8e4a17
Merge pull request #2765 from dtcenter/feature_2757_SonarQube_token
hsoh-u Dec 14, 2023
79654f0
Bugfix #2670 develop --enable-python (#2768)
JohnHalleyGotway Dec 15, 2023
bca2c83
Update install_met_env.acorn_py3.10
jprestop Dec 19, 2023
9e2825c
Update install_met_env.wcoss2_py3.10
jprestop Dec 19, 2023
7615f67
Feature #2776 cleanup (#2777)
JohnHalleyGotway Jan 10, 2024
07cacb0
Bugfix #2782 develop MASSDEN (#2783)
JohnHalleyGotway Jan 11, 2024
ce97b05
Merge remote-tracking branch 'origin/develop-ref' into develop
metplus-bot Jan 11, 2024
65b2663
Feature #2701 ismn (#2758)
JohnHalleyGotway Jan 16, 2024
96fa912
Merge remote-tracking branch 'origin/develop-ref' into develop
metplus-bot Jan 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/Users_Guide/reformat_point.rst
Original file line number Diff line number Diff line change
Expand Up @@ -456,6 +456,8 @@ While initial versions of the ASCII2NC tool only supported a simple 11 column AS

• `National Data Buoy (NDBC) Standard Meteorlogical Data format <https://www.ndbc.noaa.gov/measdes.shtml>`_. See the :ref:`MET_NDBC_STATIONS` environment variable.

• `International Soil Moisture Network (ISMN) Data format <https://ismn.bafg.de/en/>`_.

• `AErosol RObotic NEtwork (AERONET) versions 2 and 3 format <http://aeronet.gsfc.nasa.gov/>`_

• Python embedding of point observations, as described in :numref:`pyembed-point-obs-data`. See example below in :numref:`ascii2nc-pyembed`.
Expand Down Expand Up @@ -539,7 +541,7 @@ Required Arguments for ascii2nc
Optional Arguments for ascii2nc
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

3. The **-format ASCII_format** option may be set to "met_point", "little_r", "surfrad", "wwsis", "airnowhourlyaqobs", "airnowhourly", "airnowdaily_v2", "ndbc_standard", "aeronet", "aeronetv2", "aeronetv3", or "python". If passing in ISIS data, use the "surfrad" format flag.
3. The **-format ASCII_format** option may be set to "met_point", "little_r", "surfrad", "wwsis", "airnowhourlyaqobs", "airnowhourly", "airnowdaily_v2", "ndbc_standard", "ismn", "aeronet", "aeronetv2", "aeronetv3", or "python". If passing in ISIS data, use the "surfrad" format flag.

4. The **-config file** option is the configuration file for generating time summaries.

Expand Down
13 changes: 11 additions & 2 deletions internal/test_unit/xml/unit_ascii2nc.xml
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,16 @@
</output>
</test>



<test name="ascii2nc_ismn_SNOTEL">
<exec>&MET_BIN;/ascii2nc</exec>
<param> \
-format ismn \
&DATA_DIR_OBS;/ismn/SNOTEL/*/*.stm \
&OUTPUT_DIR;/ascii2nc/ismn_SNOTEL_20220924_20220927.nc
</param>
<output>
<point_nc>&OUTPUT_DIR;/ascii2nc/ismn_SNOTEL_20220924_20220927.nc</point_nc>
</output>
</test>

</met_test>
4 changes: 3 additions & 1 deletion src/tools/other/ascii2nc/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@ ascii2nc_SOURCES = ascii2nc.cc \
ndbc_handler.cc ndbc_handler.h \
ndbc_locations.cc ndbc_locations.h \
airnow_locations.cc airnow_locations.h \
aeronet_handler.cc aeronet_handler.h $(OPT_PYTHON_SOURCES)
aeronet_handler.cc aeronet_handler.h \
ismn_handler.cc ismn_handler.h \
$(OPT_PYTHON_SOURCES)

ascii2nc_CPPFLAGS = ${MET_CPPFLAGS} -I../../../basic/vx_log
ascii2nc_LDFLAGS = ${MET_LDFLAGS}
Expand Down
28 changes: 25 additions & 3 deletions src/tools/other/ascii2nc/Makefile.in
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,8 @@ am__ascii2nc_SOURCES_DIST = ascii2nc.cc ascii2nc_conf_info.cc \
airnow_handler.h ndbc_handler.cc ndbc_handler.h \
ndbc_locations.cc ndbc_locations.h airnow_locations.cc \
airnow_locations.h aeronet_handler.cc aeronet_handler.h \
python_handler.h python_handler.cc
ismn_handler.cc ismn_handler.h python_handler.h \
python_handler.cc
@ENABLE_PYTHON_TRUE@am__objects_1 = ascii2nc-python_handler.$(OBJEXT)
am__objects_2 = $(am__objects_1)
am_ascii2nc_OBJECTS = ascii2nc-ascii2nc.$(OBJEXT) \
Expand All @@ -124,7 +125,8 @@ am_ascii2nc_OBJECTS = ascii2nc-ascii2nc.$(OBJEXT) \
ascii2nc-ndbc_handler.$(OBJEXT) \
ascii2nc-ndbc_locations.$(OBJEXT) \
ascii2nc-airnow_locations.$(OBJEXT) \
ascii2nc-aeronet_handler.$(OBJEXT) $(am__objects_2)
ascii2nc-aeronet_handler.$(OBJEXT) \
ascii2nc-ismn_handler.$(OBJEXT) $(am__objects_2)
ascii2nc_OBJECTS = $(am_ascii2nc_OBJECTS)
am__DEPENDENCIES_1 =
ascii2nc_DEPENDENCIES = $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \
Expand Down Expand Up @@ -153,6 +155,7 @@ am__depfiles_remade = ./$(DEPDIR)/ascii2nc-aeronet_handler.Po \
./$(DEPDIR)/ascii2nc-ascii2nc.Po \
./$(DEPDIR)/ascii2nc-ascii2nc_conf_info.Po \
./$(DEPDIR)/ascii2nc-file_handler.Po \
./$(DEPDIR)/ascii2nc-ismn_handler.Po \
./$(DEPDIR)/ascii2nc-little_r_handler.Po \
./$(DEPDIR)/ascii2nc-met_handler.Po \
./$(DEPDIR)/ascii2nc-ndbc_handler.Po \
Expand Down Expand Up @@ -387,7 +390,9 @@ ascii2nc_SOURCES = ascii2nc.cc \
ndbc_handler.cc ndbc_handler.h \
ndbc_locations.cc ndbc_locations.h \
airnow_locations.cc airnow_locations.h \
aeronet_handler.cc aeronet_handler.h $(OPT_PYTHON_SOURCES)
aeronet_handler.cc aeronet_handler.h \
ismn_handler.cc ismn_handler.h \
$(OPT_PYTHON_SOURCES)

ascii2nc_CPPFLAGS = ${MET_CPPFLAGS} -I../../../basic/vx_log
ascii2nc_LDFLAGS = ${MET_LDFLAGS}
Expand Down Expand Up @@ -514,6 +519,7 @@ distclean-compile:
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-ascii2nc.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-ascii2nc_conf_info.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-file_handler.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-ismn_handler.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-little_r_handler.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-met_handler.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-ndbc_handler.Po@am__quote@ # am--include-marker
Expand Down Expand Up @@ -710,6 +716,20 @@ ascii2nc-aeronet_handler.obj: aeronet_handler.cc
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-aeronet_handler.obj `if test -f 'aeronet_handler.cc'; then $(CYGPATH_W) 'aeronet_handler.cc'; else $(CYGPATH_W) '$(srcdir)/aeronet_handler.cc'; fi`

ascii2nc-ismn_handler.o: ismn_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-ismn_handler.o -MD -MP -MF $(DEPDIR)/ascii2nc-ismn_handler.Tpo -c -o ascii2nc-ismn_handler.o `test -f 'ismn_handler.cc' || echo '$(srcdir)/'`ismn_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-ismn_handler.Tpo $(DEPDIR)/ascii2nc-ismn_handler.Po
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='ismn_handler.cc' object='ascii2nc-ismn_handler.o' libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-ismn_handler.o `test -f 'ismn_handler.cc' || echo '$(srcdir)/'`ismn_handler.cc

ascii2nc-ismn_handler.obj: ismn_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-ismn_handler.obj -MD -MP -MF $(DEPDIR)/ascii2nc-ismn_handler.Tpo -c -o ascii2nc-ismn_handler.obj `if test -f 'ismn_handler.cc'; then $(CYGPATH_W) 'ismn_handler.cc'; else $(CYGPATH_W) '$(srcdir)/ismn_handler.cc'; fi`
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-ismn_handler.Tpo $(DEPDIR)/ascii2nc-ismn_handler.Po
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='ismn_handler.cc' object='ascii2nc-ismn_handler.obj' libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-ismn_handler.obj `if test -f 'ismn_handler.cc'; then $(CYGPATH_W) 'ismn_handler.cc'; else $(CYGPATH_W) '$(srcdir)/ismn_handler.cc'; fi`

ascii2nc-python_handler.o: python_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-python_handler.o -MD -MP -MF $(DEPDIR)/ascii2nc-python_handler.Tpo -c -o ascii2nc-python_handler.o `test -f 'python_handler.cc' || echo '$(srcdir)/'`python_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-python_handler.Tpo $(DEPDIR)/ascii2nc-python_handler.Po
Expand Down Expand Up @@ -857,6 +877,7 @@ distclean: distclean-am
-rm -f ./$(DEPDIR)/ascii2nc-ascii2nc.Po
-rm -f ./$(DEPDIR)/ascii2nc-ascii2nc_conf_info.Po
-rm -f ./$(DEPDIR)/ascii2nc-file_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-ismn_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-little_r_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-met_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-ndbc_handler.Po
Expand Down Expand Up @@ -915,6 +936,7 @@ maintainer-clean: maintainer-clean-am
-rm -f ./$(DEPDIR)/ascii2nc-ascii2nc.Po
-rm -f ./$(DEPDIR)/ascii2nc-ascii2nc_conf_info.Po
-rm -f ./$(DEPDIR)/ascii2nc-file_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-ismn_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-little_r_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-met_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-ndbc_handler.Po
Expand Down
6 changes: 3 additions & 3 deletions src/tools/other/ascii2nc/aeronet_handler.h
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
////////////////////////////////////////////////////////////////////////


#ifndef __AERONETHANDLER_H__
#define __AERONETHANDLER_H__
#ifndef __AERONET_HANDLER_H__
#define __AERONET_HANDLER_H__


////////////////////////////////////////////////////////////////////////
Expand Down Expand Up @@ -125,7 +125,7 @@ class AeronetHandler : public FileHandler
////////////////////////////////////////////////////////////////////////


#endif /* __AERONETHANDLER_H__ */
#endif /* __AERONET_HANDLER_H__ */


////////////////////////////////////////////////////////////////////////
Expand Down
2 changes: 1 addition & 1 deletion src/tools/other/ascii2nc/airnow_handler.cc
Original file line number Diff line number Diff line change
Expand Up @@ -648,7 +648,7 @@ time_t AirnowHandler::_getValidTime(const string &dateStr, const string &timeStr
struct tm time_struct;
memset(&time_struct, 0, sizeof(time_struct));

time_struct.tm_year = atoi(year.c_str()) -1900;
time_struct.tm_year = atoi(year.c_str()) - 1900;
time_struct.tm_mon = atoi(mon.c_str()) - 1;
time_struct.tm_mday = atoi(mday.c_str());
time_struct.tm_hour = atoi(hour.c_str());
Expand Down
6 changes: 3 additions & 3 deletions src/tools/other/ascii2nc/airnow_handler.h
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
////////////////////////////////////////////////////////////////////////


#ifndef __AIRNOWHANDLER_H__
#define __AIRNOWHANDLER_H__
#ifndef __AIRNOW_HANDLER_H__
#define __AIRNOW_HANDLER_H__


////////////////////////////////////////////////////////////////////////
Expand Down Expand Up @@ -198,7 +198,7 @@ class AirnowHandler : public FileHandler
////////////////////////////////////////////////////////////////////////


#endif /* __AERONETHANDLER_H__ */
#endif /* __AERONET_HANDLER_H__ */


////////////////////////////////////////////////////////////////////////
Expand Down
27 changes: 25 additions & 2 deletions src/tools/other/ascii2nc/ascii2nc.cc
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@
// 019 07/06/22 Howard Soh METplus-Internal #19 Rename main to met_main
// 020 08/26/22 Dave Albo MET #2142 Add AirNow observations
// 021 10/03/22 Prestopnik MET #2227 Remove using namespace std from header files
// 022 10/07/22 Dave Albo MET #2276 Add NDBC Buoy data
// 022 10/07/22 Dave Albo MET #2276 Add NDBC buoy data
// 023 11/28/23 Halley Gotway MET #2701 Add ISMN soil moisture data
//
////////////////////////////////////////////////////////////////////////

Expand Down Expand Up @@ -86,6 +87,7 @@ using namespace std;
#include "aeronet_handler.h"
#include "airnow_handler.h"
#include "ndbc_handler.h"
#include "ismn_handler.h"

#ifdef ENABLE_PYTHON
#include "global_python.h"
Expand Down Expand Up @@ -113,6 +115,7 @@ enum ASCIIFormat {
ASCIIFormat_Airnow_hourlyaqobs,
ASCIIFormat_Airnow_hourly,
ASCIIFormat_NDBC_standard,
ASCIIFormat_ISMN,
ASCIIFormat_Aeronet_v2,
ASCIIFormat_Aeronet_v3,
ASCIIFormat_Python,
Expand Down Expand Up @@ -322,6 +325,10 @@ FileHandler *create_file_handler(const ASCIIFormat format, const ConcatString &a
return((FileHandler *) handler);
}

case ASCIIFormat_ISMN: {
return((FileHandler *) new IsmnHandler(program_name));
}

case ASCIIFormat_Aeronet_v2: {
AeronetHandler *handler = new AeronetHandler(program_name);
handler->setFormatVersion(2);
Expand Down Expand Up @@ -459,9 +466,21 @@ FileHandler *determine_ascii_format(const ConcatString &ascii_filename) {
delete ndbc_file;

//
// If we get here, we didn't recognize the file contents.
// See if this is an ISMN file.
//
f_in.rewind();
IsmnHandler *ismn_file = new IsmnHandler(program_name);

if(ismn_file->isFileType(f_in)) {
f_in.close();
return((FileHandler *) ismn_file);
}

delete ismn_file;

//
// If we get here, we didn't recognize the file contents.
//
mlog << Error << "\ndetermine_ascii_format() -> "
<< "could not determine file format based on file contents\n\n";

Expand Down Expand Up @@ -503,6 +522,7 @@ void usage() {
<< AirnowHandler::getFormatStringHourlyAqObs() << "\", \""
<< AirnowHandler::getFormatStringHourly() << "\", \""
<< NdbcHandler::getFormatStringStandard() << "\", \""
<< IsmnHandler::getFormatString() << "\", \""
<< AeronetHandler::getFormatString() << "\", \""
<< AeronetHandler::getFormatString_v2() << "\", \""
<< AeronetHandler::getFormatString_v3() << "\"";
Expand Down Expand Up @@ -585,6 +605,9 @@ void set_format(const StringArray & a) {
else if(NdbcHandler::getFormatStringStandard() == a[0]) {
ascii_format = ASCIIFormat_NDBC_standard;
}
else if(IsmnHandler::getFormatString() == a[0]) {
ascii_format = ASCIIFormat_ISMN;
}
else if(AeronetHandler::getFormatString() == a[0]
|| AeronetHandler::getFormatString_v2() == a[0]) {
ascii_format = ASCIIFormat_Aeronet_v2;
Expand Down
6 changes: 3 additions & 3 deletions src/tools/other/ascii2nc/file_handler.h
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
////////////////////////////////////////////////////////////////////////


#ifndef __FILEHANDLER_H__
#define __FILEHANDLER_H__
#ifndef __FILE_HANDLER_H__
#define __FILE_HANDLER_H__


////////////////////////////////////////////////////////////////////////
Expand Down Expand Up @@ -158,7 +158,7 @@ inline void FileHandler::setMessageTypeMap(map<ConcatString, ConcatString> m) {
////////////////////////////////////////////////////////////////////////


#endif /* __FILEHANDLER_H__ */
#endif /* __FILE_HANDLER_H__ */


////////////////////////////////////////////////////////////////////////
Expand Down
Loading