Configuration Class Tech Design Doc (TDD)
The Config Class will be a widely used class among many of the Unified Workflow tools. Its main purpose is to provide the parameters in a uniform way to any of a host of tools developed by extending the Python dictionary base class. We can create subclasses of Config that all interface nicely and uniformly with a variety of input file types, extend configuration languages in the same ways, and give UFS Users a more straightforward learning curve when configuring their experiments. More documentation on the file types used in UFS software can be found in the UFS Weather Model Config File Types page.
Its purpose is to add additional functionality to the Python dict base class that allows us to generate dictionaries from a variety of input sources, and to serve as a base class to many of the dictable configuration types used among UFS components. More documentation on those can be found in the UFS Weather Model Config File Types page. The purpose of the TDD for the Config class is to outline the design of the base class, and its potential subclasses to support standard languages.
The Config class will serve as a mostly abstract class defining the interface to a variety of subclasses that parse various known configuration files, including Fortran namelists, YAML, and INI, and of course incorporating standard Python dictionaries, command line arguments, and environment variables. There will be a variety of tools that will interact with Config objects that allow us to manipulate the data through updating dictionary contents, comparing two Config objects, and interrogating a Config object's contents, and reformatting a dictionary's output and getter methods.
The UML diagram shows the inheritance tree for various types of configuration file types that are all dictable, and should be sufficient to handle the UFS dictables.
User provided parameters
Required:
config_path - a path to a configuration file
The __init__ method
The constructor should set the config_path instance variable.
Many in the community have expressed a desire to have "dot notation" to more easily call dictionary keys as attributes. This should likely be instantiated in the Config class, if we decide we need it at all. We have seen that overriding the get_attr provides that feature, however we may want to explore using the types.SimpleNamespace from the standard library as an additional base class for a more robust solution.
config_path as described above.
replace_templates() This method will be used as a method by which any Config object can cycle through its key/value pairs recursively, replacing Jinja2 templates as necessary. This is a placeholder until more details are fleshed out in work scheduled for PI6. load() An abstract method to be implemented by each of the subclasses, depending on the input file type. dump_file(output_path) An abstract method to be implemented by each of the subclasses, depending on the output file type. parse_include() Borrowing from pyyaml, traverse the dict to treat the !INCLUDE tag just the same as it would be treated in YAML. This is left for further exploration and design in PI6, and applies to any non-YAML config file types, so putting it in the base class for now.
The __init__ method
The constructor should invoke the init method of the Config class. It should also load the YAML file and update its own dict contents (i.e., call self.update()).
Nothing additional.
load() Call the YAML load function. Return the resulting dict. dump_file(output_path) Call the YAML dump function, writing to the output_path provided by the caller.
Resource Files have an additional double-colon notation that indicates the section is plain text CSV section, but it can be parsed as YAML. Here, override the dump_file to call format_output first. Override the Constructor to manage parsing those CSV sections. As a first pass, let's see how configuring this file type works as a plain YAML file.
The __init__ method
The constructor should invoke the init method of the Config class. It should also load the YAML file, handle the CSV sections, and update its own dict contents (i.e., call self.update()).
Nothing additional.
dump_file(output_path) Call the YAML dump function, writing to the output_path provided by the caller. format_output() Given the Config dict contents, find any that contain CSV sections and ensure they are reformatted properly to write out in the expected format. csv_handler() Find and replace any of the Config dict entries that contain the CSV sections of the input file. Add some sort of the flag to the Config dict to indicate this should be a CSV section upon output.
Field tables generally contain information that could be stored in YAML format. This class will exist only to write out field_table format given that its configuration has been set by a YAML file. There will be no attempt to write a field_table parser, so the YAML load() method should be sufficient to load the configuration object.
The __init__ method
The constructor should invoke the init method of the Config class. It should also load the YAML file and update its own dict contents (i.e., call self.update()).
Nothing additional.
dump_file(output_path) Call the generic python write function to write the formatted output. format_output() Given the Config dict contents, create the unique output format required by a field_table, documented here.
The __init__ method
The constructor should invoke the init method of the Config class. It should also load the F90 namelist file and update its own dict contents (i.e., call self.update()).
Nothing additional.
load() Call the f90nml load function. Parse all !INCLUDE tags. Return the resulting dict. dump_file(output_path) Call the f90nml dump function, writing to the output_path provided by the caller.
The __init__ method
The constructor should invoke the init method of the Config class. It should also load the INI file and update its own dict contents (i.e., call self.update()).
User provided parameters
Optional:
space_around_delimiters - bool to be used in the dump_file method. True for INI, False for bash. Default True.
Nothing additional.
load() Call the configparse load function. Parse all !INCLUDE tags. Return the resulting dict. dump_file(output_path) Call the configparse dump function, writing to the output_path provided by the caller using the space_around_delimiters argument keyword argument.
The caller will need to know which type of config file it has before starting creating a Config object so that it can create the correct one.