OmegaConf visitor pattern

[Jasha10]

Should OmegaConf include a visitor API for recursively traversing OmegaConf objects?

Motivation

Recursively visiting subnodes of DictConfig/ListConfig objects is a frequent usage pattern when working with OmegaConf.
Here is an incomplete list of examples:

• In hydra/tests/instantiate/__init__.py, there is the partial_equal(obj1, obj2) function.
• If hydra.utils.instantiate is called with _recursive_=True, then subobjects of the given omegaconf object are recursively visited and instantiated.
• OmegaConf.missing_keys visits all leaf nodes of the given object to generate a list of missing keys.
• The implementation of OmegaConf.{to_container,to_object} recursively visits and converts subnodes of the given OmegaConf object.
• There have been OmegaConf feature requests for functions that visit subobjects and print some information, e.g. #996.

Exposing an abstract base class OmegaConfVisitor, which users could subclass with custom behavior, would allow encapsulation of object-traversal logic in the above examples.

Safety

Using @abc.abstractmethod to decorate methods of OmegaConfVisitor, the visitor pattern could be used to enforce some degree of safety, ensuring that visitor implementations are able to handle all possible OmegaConf objects.

This would be particularly helpful when introducing new subclasses of omegaconf.Node OmegaConf's type heirarchy (e.g. UnionNode was recently introduced, and TupleNode may be introduced in the future). If a new OmegaConf Node subclass is introduced (say, TupleNode), we could use the @abc.abstractmethod decorator (or some other form of metaclass magic) to ensure that users' custom visitor classes are able to handle the newly-introduced Node subclass.

Design Considerations

class OmegaConfVisitor(ABC):

    @abstractmethod
    def visit_dict_config(self, cfg: DictConfig) -> ...:
        ...

    @abstractmethod
    def visit_list_config(self, cfg: ListConfig) -> ...:
        ...

There are several standard patterns for traversing DictConfig/ListConfig subobjects. For example, you can do cfg[key] or cfg._get_node(key) or cfg._get_child(key). The choice of access pattern would be made in the methods MyVisitor.visit_dict_config/MyVisitor.visit_list_config defined on the subclass.

A few questions regarding API design:

• Should every subclass of omegaconf.Node have a corresponding OmegaConfVisitor.visit_... method? Should all such methods be decorated with @abstractmethod? What if the user does not care about ValueNode vs IntegerNode vs AnyNode? Perhaps the use-case is simple (e.g. OmegaConf.missing_keys mentioned above), in which case it would be overkill to define custom methods for all possible subtypes of omegaconf.Node...
• Is there a standard way to handle visiting two different OmegaConf objects simultaneously (as with the partial_equal example given above)?