Stage 1: Template Tool
The Template Tool can be used to manage any flat files or strings that are designed as Jinja2 Templates as outlined by the Jinja2 Template Designer Docs.
The first use-cases of this tool will be applied to UFS Weather Model configuration files currently managed with the atparse template utility, which includes, but is not limited to:
- Fortran namelists
- UFS-specific configure files:
- model_configure files (YAML-like)
- data_table (CSV)
- NEMS configure (NEMS-specific)
- WW3 (CSV with header/footer templated)
- MOM6 configure (INI-like)
- diag_table (CSV divided into irregular sections with header templated)
There exist additional use cases under the UFS Umbrella of Applications that use Jinja2, including:
- XML Files
- Extending configuration ingest
The User provides a template file or string and a configuration object, then the template is read in, the undefined variables are gathered. The user-provided configuration object is validated against the set of undefined template variables, then the template is rendered and written to an output file.
The actions taken in this tool can be captured in a Template class with a few basic methods that encapsulate Jinja2 processes.
One of the following is required:
template_path - a path to a template file
template_string - a templated string
Optional arguments:
configure_obj – a dictionary containing the variables required by the template. When none is provided, use environment variables.
The __init__ method will determine to load the template depending on whether a path or string was provided. It will proceed to load the provided template, validate it against the provide Config object, and attempt to render the template. During the course of these actions, it should also set all the appropriate instance variables in the UML diagram above.
It is up to the caller to choose when and where to write the resulting template to a file.
undeclared_variables – an attribute method that scrapes all undeclared variables in the template and returns a list of variable names
load_file – a wrapper around the jinja2 file loader
load_string – a wrapper around the jinja2 string loader
dump_file – writes the rendered template to a specified file path
validate_config – compares the configure_obj to the undeclared_variables, and provides useful feedback to user when required variables are not provided (specifically, the list of variables needed)
To meet the needs of bash-based workflows that will use the tool, and to transition to this tool as smoothly as possible, the following UI should be included with the final delivered tool.
python templater.py -i <path_to_template> -o <path_to_rendered_template>
By default, if no configuration information is provided the user's shell environment variables will be used to fill the template. The user should be able to provide configuration information in two additional ways:
- By providing a path to a YAML file that contains the necessary settings. An optional section can be provided in the event the YAML file contains tiered data.
python templater.py -i <path_to_template> -o <path_to_rendered_template> -c <path_to_yaml>
- By providing any number of command line arguments
python templater.py -i <path_to_template> -o <path_to_rendered_template> config.a=12 config.b=34
When providing both the YAML file and the config options via command line, the command line options will overwrite any set in the YAML. YAML settings will overwrite those found in the user's shell environment. The tool will combine all 3 configuration settings to generate the final configuration object needed to render the template.
In addition, the tool will provide some helpful user features for running in a standalone mode.
- Dry-run (--dry-run) will render the template and print the result to the screen, and tell the user where the output file would be placed, but will not write a file.
- Values-needed (--values-needed) will generate a list of values required by the provided template.