Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 2142 ascii2nc airnow #2256

Merged
merged 5 commits into from
Sep 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions data/table_files/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ tablefiles_DATA = \
met_header_columns_V3.0.1.txt \
met_header_columns_V3.0.txt \
met_header_columns_V2.0.txt \
airnow_monitoring_site_locations_v2.txt \
stat_column_description.txt \
obs_error_table.txt \
grib1_af_131_57.txt \
Expand Down
1 change: 1 addition & 0 deletions data/table_files/Makefile.in
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,7 @@ tablefiles_DATA = \
met_header_columns_V3.0.1.txt \
met_header_columns_V3.0.txt \
met_header_columns_V2.0.txt \
airnow_monitoring_site_locations_v2.txt \
stat_column_description.txt \
obs_error_table.txt \
grib1_af_131_57.txt \
Expand Down
23,292 changes: 23,292 additions & 0 deletions data/table_files/airnow_monitoring_site_locations_v2.txt

Large diffs are not rendered by default.

10 changes: 10 additions & 0 deletions docs/Users_Guide/config_options.rst
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,16 @@ MET tool stating the location of the parsing error.
Runtime Environment Variables
-----------------------------

MET_AIRNOW_STATIONS
^^^^^^^^

The MET_AIRNOW_STATIONS environment variable can be used to specify a file that
will override the default file. If set it should be a full path to the file.
The default is:
"MET_BASE/table_files/airnow_monitoring_site_locations_v2.dat"
This file contains ascii column data that allows lookups of latitude,longitude,
and elevation for all airnow stations based on stationId and/or AqSid.

MET_BASE
^^^^^^^^

Expand Down
4 changes: 2 additions & 2 deletions docs/Users_Guide/reformat_point.rst
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,7 @@ ASCII2NC tool

This section describes how to run the ASCII2NC tool. The ASCII2NC tool is used to reformat ASCII point observations into the NetCDF format expected by the Point-Stat tool. For those users wishing to verify against point observations that are not available in PrepBUFR format, the ASCII2NC tool provides a way of incorporating those observations into MET. If the ASCII2NC tool is used to perform a reformatting step, no configuration file is needed. However, for more complex processing, such as summarizing time series observations, a configuration file may be specified. For details on the configuration file options, see :numref:`config_options` and example configuration files distributed with the MET code.

Initial versions of the ASCII2NC tool supported only a simple 11 column ASCII point observation format. It currently supports point observation data in the following formats: the default 11 column format, little_r format, `SURFace RADiation (SURFRAD) <http://www.esrl.noaa.gov/gmd/grad/surfrad/>`_ and Integrated Surface Irradiance Study (ISIS) formats, the Western Wind and Solar Integration Study (WWSIS) format, and the `AErosol RObotic NEtwork (AERONET) versions 2 and 3 format. <http://aeronet.gsfc.nasa.gov/>`_ WWSIS data are available by request from National Renewable Energy Laboratory (NREL) in Boulder, CO.
Initial versions of the ASCII2NC tool supported only a simple 11 column ASCII point observation format. It currently supports point observation data in the following formats: the default 11 column format, little_r format, `SURFace RADiation (SURFRAD) <http://www.esrl.noaa.gov/gmd/grad/surfrad/>`_ and Integrated Surface Irradiance Study (ISIS) formats, the Western Wind and Solar Integration Study (WWSIS) format, the `AirNow DailyData_v2, AirNow HourlyData, and AirNow HourlyAQObs formats. <https://www.epa.gov/outdoor-air-quality-data>`_, and the `AErosol RObotic NEtwork (AERONET) versions 2 and 3 format. <http://aeronet.gsfc.nasa.gov/>`_ WWSIS data are available by request from National Renewable Energy Laboratory (NREL) in Boulder, CO.

MET version 9.0 adds support for the passing observations to ascii2nc using a Python script with the "-format python" option. An example of running ASCII2NC with Python embedding is included below.

Expand Down Expand Up @@ -505,7 +505,7 @@ Required arguments for ascii2nc
Optional arguments for ascii2nc
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

3. The **-format ASCII_format** option may be set to "met_point", "little_r", "surfrad", "wwsis", "aeronet", "aeronetv2", "aeronetv3", or "python". If passing in ISIS data, use the "surfrad" format flag.
3. The **-format ASCII_format** option may be set to "met_point", "little_r", "surfrad", "wwsis", "airnowhourlyaqobs", "airnowhourly", "airnowdaily_v2", "aeronet", "aeronetv2", "aeronetv3", or "python". If passing in ISIS data, use the "surfrad" format flag.

4. The **-config file** option is the configuration file for generating time summaries.

Expand Down
33 changes: 33 additions & 0 deletions internal/test_unit/xml/unit_ascii2nc.xml
Original file line number Diff line number Diff line change
Expand Up @@ -117,4 +117,37 @@
</output>
</test>

<test name="ascii2nc_airnow_daily_v2">
<exec>&MET_BIN;/ascii2nc</exec>
<param> \
&DATA_DIR_OBS;/airnow/daily_data_v2.dat \
&OUTPUT_DIR;/ascii2nc/airnow/daily_data_v2_20220312.nc
</param>
<output>
<point_nc>&OUTPUT_DIR;/ascii2nc/airnow/daily_data_v2_20220312.nc</point_nc>
</output>
</test>

<test name="ascii2nc_airnow_hourly_aqobs">
<exec>&MET_BIN;/ascii2nc</exec>
<param> \
&DATA_DIR_OBS;/airnow/HourlyAQObs_2022031200.dat \
&OUTPUT_DIR;/ascii2nc/airnow/HourlyAQObs_20220312.nc
</param>
<output>
<point_nc>&OUTPUT_DIR;/ascii2nc/airnow/HourlyAQObs_20220312.nc</point_nc>
</output>
</test>

<test name="ascii2nc_airnow_hourly">
<exec>&MET_BIN;/ascii2nc</exec>
<param> \
&DATA_DIR_OBS;/airnow/HourlyData_2022031200.dat \
&OUTPUT_DIR;/ascii2nc/airnow/HourlyData_20220312.nc
</param>
<output>
<point_nc>&OUTPUT_DIR;/ascii2nc/airnow/HourlyData_20220312.nc</point_nc>
</output>
</test>

</met_test>
2 changes: 2 additions & 0 deletions src/tools/other/ascii2nc/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ ascii2nc_SOURCES = ascii2nc.cc \
met_handler.cc met_handler.h \
surfrad_handler.cc surfrad_handler.h \
wwsis_handler.cc wwsis_handler.h \
airnow_handler.cc airnow_handler.h \
airnow_locations.cc airnow_locations.h \
aeronet_handler.cc aeronet_handler.h $(OPT_PYTHON_SOURCES)

ascii2nc_CPPFLAGS = ${MET_CPPFLAGS} -I../../../basic/vx_log
Expand Down
46 changes: 44 additions & 2 deletions src/tools/other/ascii2nc/Makefile.in
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,10 @@ am__ascii2nc_SOURCES_DIST = ascii2nc.cc ascii2nc_conf_info.cc \
ascii2nc_conf_info.h file_handler.cc file_handler.h \
little_r_handler.cc little_r_handler.h met_handler.cc \
met_handler.h surfrad_handler.cc surfrad_handler.h \
wwsis_handler.cc wwsis_handler.h aeronet_handler.cc \
aeronet_handler.h python_handler.h python_handler.cc
wwsis_handler.cc wwsis_handler.h airnow_handler.cc \
airnow_handler.h airnow_locations.cc airnow_locations.h \
aeronet_handler.cc aeronet_handler.h python_handler.h \
python_handler.cc
@ENABLE_PYTHON_TRUE@am__objects_1 = ascii2nc-python_handler.$(OBJEXT)
am__objects_2 = $(am__objects_1)
am_ascii2nc_OBJECTS = ascii2nc-ascii2nc.$(OBJEXT) \
Expand All @@ -117,6 +119,8 @@ am_ascii2nc_OBJECTS = ascii2nc-ascii2nc.$(OBJEXT) \
ascii2nc-met_handler.$(OBJEXT) \
ascii2nc-surfrad_handler.$(OBJEXT) \
ascii2nc-wwsis_handler.$(OBJEXT) \
ascii2nc-airnow_handler.$(OBJEXT) \
ascii2nc-airnow_locations.$(OBJEXT) \
ascii2nc-aeronet_handler.$(OBJEXT) $(am__objects_2)
ascii2nc_OBJECTS = $(am_ascii2nc_OBJECTS)
am__DEPENDENCIES_1 =
Expand All @@ -140,6 +144,8 @@ DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir)
depcomp = $(SHELL) $(top_srcdir)/depcomp
am__maybe_remake_depfiles = depfiles
am__depfiles_remade = ./$(DEPDIR)/ascii2nc-aeronet_handler.Po \
./$(DEPDIR)/ascii2nc-airnow_handler.Po \
./$(DEPDIR)/ascii2nc-airnow_locations.Po \
./$(DEPDIR)/ascii2nc-ascii2nc.Po \
./$(DEPDIR)/ascii2nc-ascii2nc_conf_info.Po \
./$(DEPDIR)/ascii2nc-file_handler.Po \
Expand Down Expand Up @@ -359,6 +365,8 @@ ascii2nc_SOURCES = ascii2nc.cc \
met_handler.cc met_handler.h \
surfrad_handler.cc surfrad_handler.h \
wwsis_handler.cc wwsis_handler.h \
airnow_handler.cc airnow_handler.h \
airnow_locations.cc airnow_locations.h \
aeronet_handler.cc aeronet_handler.h $(OPT_PYTHON_SOURCES)

ascii2nc_CPPFLAGS = ${MET_CPPFLAGS} -I../../../basic/vx_log
Expand Down Expand Up @@ -478,6 +486,8 @@ distclean-compile:
-rm -f *.tab.c

@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-aeronet_handler.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-airnow_handler.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-airnow_locations.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-ascii2nc.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-ascii2nc_conf_info.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-file_handler.Po@am__quote@ # am--include-marker
Expand Down Expand Up @@ -605,6 +615,34 @@ ascii2nc-wwsis_handler.obj: wwsis_handler.cc
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-wwsis_handler.obj `if test -f 'wwsis_handler.cc'; then $(CYGPATH_W) 'wwsis_handler.cc'; else $(CYGPATH_W) '$(srcdir)/wwsis_handler.cc'; fi`

ascii2nc-airnow_handler.o: airnow_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-airnow_handler.o -MD -MP -MF $(DEPDIR)/ascii2nc-airnow_handler.Tpo -c -o ascii2nc-airnow_handler.o `test -f 'airnow_handler.cc' || echo '$(srcdir)/'`airnow_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-airnow_handler.Tpo $(DEPDIR)/ascii2nc-airnow_handler.Po
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='airnow_handler.cc' object='ascii2nc-airnow_handler.o' libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-airnow_handler.o `test -f 'airnow_handler.cc' || echo '$(srcdir)/'`airnow_handler.cc

ascii2nc-airnow_handler.obj: airnow_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-airnow_handler.obj -MD -MP -MF $(DEPDIR)/ascii2nc-airnow_handler.Tpo -c -o ascii2nc-airnow_handler.obj `if test -f 'airnow_handler.cc'; then $(CYGPATH_W) 'airnow_handler.cc'; else $(CYGPATH_W) '$(srcdir)/airnow_handler.cc'; fi`
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-airnow_handler.Tpo $(DEPDIR)/ascii2nc-airnow_handler.Po
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='airnow_handler.cc' object='ascii2nc-airnow_handler.obj' libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-airnow_handler.obj `if test -f 'airnow_handler.cc'; then $(CYGPATH_W) 'airnow_handler.cc'; else $(CYGPATH_W) '$(srcdir)/airnow_handler.cc'; fi`

ascii2nc-airnow_locations.o: airnow_locations.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-airnow_locations.o -MD -MP -MF $(DEPDIR)/ascii2nc-airnow_locations.Tpo -c -o ascii2nc-airnow_locations.o `test -f 'airnow_locations.cc' || echo '$(srcdir)/'`airnow_locations.cc
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-airnow_locations.Tpo $(DEPDIR)/ascii2nc-airnow_locations.Po
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='airnow_locations.cc' object='ascii2nc-airnow_locations.o' libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-airnow_locations.o `test -f 'airnow_locations.cc' || echo '$(srcdir)/'`airnow_locations.cc

ascii2nc-airnow_locations.obj: airnow_locations.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-airnow_locations.obj -MD -MP -MF $(DEPDIR)/ascii2nc-airnow_locations.Tpo -c -o ascii2nc-airnow_locations.obj `if test -f 'airnow_locations.cc'; then $(CYGPATH_W) 'airnow_locations.cc'; else $(CYGPATH_W) '$(srcdir)/airnow_locations.cc'; fi`
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-airnow_locations.Tpo $(DEPDIR)/ascii2nc-airnow_locations.Po
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='airnow_locations.cc' object='ascii2nc-airnow_locations.obj' libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-airnow_locations.obj `if test -f 'airnow_locations.cc'; then $(CYGPATH_W) 'airnow_locations.cc'; else $(CYGPATH_W) '$(srcdir)/airnow_locations.cc'; fi`

ascii2nc-aeronet_handler.o: aeronet_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-aeronet_handler.o -MD -MP -MF $(DEPDIR)/ascii2nc-aeronet_handler.Tpo -c -o ascii2nc-aeronet_handler.o `test -f 'aeronet_handler.cc' || echo '$(srcdir)/'`aeronet_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-aeronet_handler.Tpo $(DEPDIR)/ascii2nc-aeronet_handler.Po
Expand Down Expand Up @@ -762,6 +800,8 @@ clean-am: clean-binPROGRAMS clean-generic mostlyclean-am

distclean: distclean-am
-rm -f ./$(DEPDIR)/ascii2nc-aeronet_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-airnow_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-airnow_locations.Po
-rm -f ./$(DEPDIR)/ascii2nc-ascii2nc.Po
-rm -f ./$(DEPDIR)/ascii2nc-ascii2nc_conf_info.Po
-rm -f ./$(DEPDIR)/ascii2nc-file_handler.Po
Expand Down Expand Up @@ -816,6 +856,8 @@ installcheck-am:

maintainer-clean: maintainer-clean-am
-rm -f ./$(DEPDIR)/ascii2nc-aeronet_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-airnow_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-airnow_locations.Po
-rm -f ./$(DEPDIR)/ascii2nc-ascii2nc.Po
-rm -f ./$(DEPDIR)/ascii2nc-ascii2nc_conf_info.Po
-rm -f ./$(DEPDIR)/ascii2nc-file_handler.Po
Expand Down
Loading