Automated framework to render GRIB1/2 data with ECMWF's Magics, using a repository of simple YAML recipes
Supported models:
- COSMO-LAMI
- ICON
- IFS-ECMWF
- ERG5 (https://dati.arpae.it/dataset/erg5-interpolazione-su-griglia-di-dati-meteo)
- PRAGA (https://github.com/ARPA-SIMC/PRAGA)
- ADRIAC (partially) (https://www.arpae.it/it/temi-ambientali/mare/previsioni-mare/previsioni-oceanografiche)
Arkimaps works both as a standalone program and as a postprocessor for https://github.com/ARPA-SIMC/arkimet
- python3 >= 3.7
- python3-magics
- eccodes
optional requirements:
- https://github.com/ARPA-SIMC/libsim (for some specific preprocessing, i.e.: accumulated precipitation, relative humidity, wind speed)
- https://github.com/ARPA-SIMC/arkimet
./arkimaps process -o out.tar --grib < test_data.grib
# t2m example:
arki-query --inline 'Reftime:=today 00:00;product:GRIB1,80,2,11;level:g02' http://arkimet.metarpa:8090/dataset/cosmo_5M_ita > test_data.arkimet
./arkimaps process -o out.tar < test_data.arkimet
To obtain the parameters need for an arki-query for all available recipes:
./arkimaps print-arki-query
The output is a tar
file containing one directory for each recipe
name; inside every directory there's a plot for each forecast step.
To dinamically unpack the tar
file:
./arkimaps process --filter=eccodes < test.arkimet |tar -xf - -C out/
The --flavours
option can be used to specify different output
possibilities.
For example, for plots tailored on Emilia-Romagna:
./arkimaps process --filter=eccodes --flavours=emro < test.arkimet
The full list of implemented areas is available in the directory doc/prodotti.
To add new areas it's possibile to add files to the flavour directory (see below at the section "Creating new flavours for rendering".
arkimaps produces by default every possibile output with the input data. To reduce the number of products there are two ways:
- copying a subset of recipes from the
recipes
directory in a custom path and pointing arkimaps to it with the--recipes
option - filtering the products using a "flavour" (see below at the section "Creating new flavours for rendering")
Example:
./arkimaps process --filter=eccodes --recipes /tmp/temprecipes < test.arkimet
- Informations on implemented products andareas (in Italian only): doc/prodotti/README.md
- Glossary
- Recipe inputs
- Steps for creating new recipes
- Description on how data are processed from a recipe to the desired output: Recipe workflow.
For a description of all available subcommands:
./arkimaps -h
For a description of each subcommand:
./arkimaps process -h
Inputs defined in a recipe are common to all the recipes. The same input name defined in more recipes results in an error. It's possible to create recipes without steps just for input definition.
Static files such as geopoints or shapefiles can be placed in a special path
$recipes/static
and referenced in recipes (recipes/flavours/flavours_inputs.yaml
is an example)
To speed up debugging, there are some arkimaps
subcommands:
./arkimaps --debug dispatch workdir
to organize input data in a defined work directory./arkimaps --debug preview workdir t2m+000
to preview a single output from a pre-organized work directory
The preview opens xdg-open
, that usses the predefined system application to open
the resulting image.
If the recipe is nested in a subdirectory, the subdirectory must be specified:
e.g. ./arkimaps --debug preview workdir standalone/jet+000
(v. #99).
It's possible to specify the reference time using --date
. The syntax is the same
of the date --date=
command.
In the .yaml
files contained in the recipes
directory is possible to add a
flavours
sections that adds rendering variants.
The arkimaps
rendering subcommands have a --flavours
option that allows to
choose which variant to use.
The structure of a "flavour" contains:
name
: variant namerecipes_filter
: list of products (recipes name). If present, only the listed recipes would be plotted. Allowed globs (e.g.:tp*
)steps
: configurations for single steps
The configuration of a "step" may have:
skip: yes
: ignore a given stepparams
: defaultparams
value if not specified in the single recipe
To generate tiled web map
output, it's possible to add a tile
keyword containing:
zoom_min: Int = 3
: minimum zoom levelzoom_max: Int = 5
: maximum zoom levellat_min: Float
: minimum latitudelat_max: Float
: maximum latitudelon_min: Float
: minimum longitudelon_max: Float
: maximum longitude
arkimaps is Copyright (C) 2020-2023 ARPAE-SIMC urpsim@arpae.it
arkimaps is Free Software, licensed under the terms of the GNU General Public License version 2.