Config when subdag poor error message

[skrawcz]

When you use @config in subdag, and that node doesn't exist but is being requested you'll get a cryptic error.

Current behavior

If you are using subdag, and mess up config, you can get this cryptic error.

Stack Traces

Traceback (most recent call last):
  File "user_help/seth/may_26.py", line 27, in <module>
    dr = driver.Driver({}, temp_module)  # "foo": False, "baz": "qux"
  File "hamilton/hamilton/driver.py", line 142, in __init__
    raise e
  File "hamilton/hamilton/driver.py", line 137, in __init__
    self.graph = graph.FunctionGraph(*modules, config=config, adapter=adapter)
  File "hamilton/hamilton/graph.py", line 213, in __init__
    self.nodes = create_function_graph(*modules, config=self._config, adapter=adapter)
  File "hamilton/hamilton/graph.py", line 88, in create_function_graph
    for n in fm_base.resolve_nodes(f, config):
  File "hamilton/hamilton/function_modifiers/base.py", line 715, in resolve_nodes
    nodes = node_creator.generate_nodes(fn, filter_config(config, node_creator))
  File "hamilton/hamilton/function_modifiers/recursive.py", line 403, in generate_nodes
    nodes += self._create_additional_static_nodes(nodes, namespace)
  File "hamilton/hamilton/function_modifiers/recursive.py", line 282, in _create_additional_static_nodes
    typ=node_types[new_node_name],
KeyError: 'subdag_qux.input_df'

Steps to replicate behavior

Run this to get the error

import pandas as pd
from hamilton.function_modifiers import config, subdag, source, value

@config.when(foo=True)
def data_file__hi() -> pd.DataFrame:
    return pd.DataFrame({"c": [1, 2, 3]})

@config.when(baz="qux")
def a_transform(input_df: pd.DataFrame, data_file: pd.DataFrame=None) -> pd.DataFrame:
    print(data_file)
    return input_df.assign(b=input_df.a + 1)


@subdag(data_file__hi, a_transform,
        inputs={"input_df": source("some_df")},
        )
def subdag_qux(a_transform: pd.DataFrame) -> pd.DataFrame:
    return a_transform


if __name__ == '__main__':
    from hamilton import ad_hoc_utils
    from hamilton import driver
    temp_module = ad_hoc_utils.create_temporary_module(subdag_qux)
    dr = driver.Driver({}, temp_module)  # "foo": False, "baz": "qux"
    dr.display_all_functions("seth.dot", {"format": "png"})
    df = dr.execute(["subdag_qux"],
                    inputs={"some_df": pd.DataFrame({"a": [3, 4, 5]})})
    print(df)

Library & System Information

E.g. python version, hamilton library version, linux, etc.

Expected behavior

There should be a better error here. It's unclear what to do.

[elijahbenizzy]

Fix has been released: https://github.com/DAGWorks-Inc/hamilton/releases/tag/sf-hamilton-1.26.2