KedroContextError in empty 'kedro new' project: expected ConfigLoader is NoneType

[Spooky-0]

Description

My fairly new project runs into kedro.framework.context.context.KedroContextError: Expected an istance of "ConfigLoader", got "NoneType" instead.
I tried to follow the error, and I believe that the Hooks are not loaded yet, however the logger is already trying to use them.

Since I am trying it on my Work station where I don't have access to github ( I know, kill me pls) I am typing this out and can't really copy paste long error messages. Sorry.

I proceded to try it with a completely empty project (kedro new, kedro ipython) which runs into the same issue.

Context

In my new Project I have followed the two tutorials
https://kedro.readthedocs.io/en/latest/03_tutorial/04_create_pipelines.html
https://kedro.readthedocs.io/en/latest/11_tools_integration/01_pyspark.html

only after which I tried 'kedro run', which lead to the Issue.

Steps to Reproduce

1. kedro --version -> 0.17.1
2. kedro new
3. kedro ipython

Expected Result

the session should be able to start.

Actual Result

As I said, no copy paste :(

with KedroSession.create(....
...
conf_logging = self.get_logging_config()
...
return self._get_config_loader().

the Exception raised is in this function where I added one line of debug:

    def _get_config_loader(self) -> ConfigLoader:
        """A hook for changing the creation of a ConfigLoader instance.
        Returns:
            Instance of `ConfigLoader` created by `register_config_loader` hook.
        Raises:
            KedroContextError: Incorrect ``ConfigLoader`` registered for the project.
        """
        conf_root = settings.CONF_ROOT
        conf_paths = [
            str(self.project_path / conf_root / "base"),
            str(self.project_path / conf_root / self.env),
        ]
        hook_manager = get_hook_manager()
        config_loader = hook_manager.hook.register_config_loader(  # pylint: disable=no-member
            conf_paths=conf_paths, env=self.env, extra_params=self._extra_params,
        )

        # I ADDED THIS:
        print(hook_manager.hook.register_config_loader.get_hookimpls()) --- > []
    
        if not isinstance(config_loader, ConfigLoader):
            raise KedroContextError(
                f"Expected an instance of `ConfigLoader`, "
                f"got `{type(config_loader).__name__}` instead."
            )
        return config_loader

This is why I believe that the hooks are not registered yet.

Your Environment

Include as many relevant details about the environment in which you experienced the bug:

• Kedro version used (pip show kedro or kedro -V):
• Python version used (python -V): 3.6
• Operating system and version: Linux 3.10.0 on Cloudera Workbench 5.xxx

[Spooky-0]

I guess its the same Issue as
#707
but for 0.17.1.
Since 0.17.1 is released (as opposed to 18 being dev) i opened this issue again.

[merelcht]

Hi @Spooky-0 , this is a known issue which we will very soon release a fix for. In the meantime we recommend to just roll back to Kedro 0.17.0.

[Spooky-0]

Ok great. Thanks for the heads up. I went back to 0.17.0 and all works fine :)

[Vinay1233]

Hi, I understand that you have closed this, but do you know if this will be fixed in 0.17.3?

I still face this issue with 0.17.2

[merelcht]

Hi @Vinay1233, I'm sorry to hear you're still facing this issue. Can you provide steps to reproduce?

[Vinay1233]

Hi @MerelTheisenQB,

Im using the register_config_loader hook as i would eventually like to update the globals.yml with the extra_params (I see that can now be done in 0.17.1 from the release notes, though I don't see that updated in the hook docs).

This was working with 0.17.0.

The below is a test without passing extra_params:

 @hook_impl
    def register_config_loader(self, conf_paths: Iterable[str]) -> ConfigLoader:
        loader = TemplatedConfigLoader(
            conf_paths,
            globals_pattern="*globals.yml",
            globals_dict=None,
        )

        return loader

error:

---------------------------------------------------------------------------
KedroContextError                         Traceback (most recent call last)
<command-3439166310301806> in <module>
     16                     }
     17   update_hive_col_comments = { 'db_name' : 'aca_dev', 'update_flag' : 0}
---> 18   aca.run.main(tags=[*nodes],pipeline_name= 'de',extra_params=config_override, update_hive_col_comments=update_hive_col_comments)

/tmp/aca111111111111/ACA/src/aca/run.py in main(tags, runner, node_names, from_nodes, to_nodes, from_inputs, load_versions, pipeline_name, extra_params, update_hive_col_comments)
     75         save_on_close=True,
     76         env=None,
---> 77         extra_params=extra_params) as session: 
     78 
     79         update_comments = 0

/local_disk0/pythonVirtualEnvDirs/virtualEnv-408403f5-d1e9-4a76-bcb0-33b8ef51a6ac/lib/python3.7/site-packages/kedro/framework/session/session.py in create(cls, package_name, project_path, save_on_close, env, extra_params)
    202 
    203         # we need a ConfigLoader registered in order to be able to set up logging
--> 204         session._setup_logging()
    205         return session
    206 

/local_disk0/pythonVirtualEnvDirs/virtualEnv-408403f5-d1e9-4a76-bcb0-33b8ef51a6ac/lib/python3.7/site-packages/kedro/framework/session/session.py in _setup_logging(self)
    220     def _setup_logging(self) -> None:
    221         """Register logging specified in logging directory."""
--> 222         conf_logging = self._get_logging_config()
    223         logging.config.dictConfig(conf_logging)
    224 

/local_disk0/pythonVirtualEnvDirs/virtualEnv-408403f5-d1e9-4a76-bcb0-33b8ef51a6ac/lib/python3.7/site-packages/kedro/framework/session/session.py in _get_logging_config(self)
    208         context = self.load_context()
    209 
--> 210         conf_logging = context.config_loader.get(
    211             "logging*", "logging*/**", "**/logging*"
    212         )

/local_disk0/pythonVirtualEnvDirs/virtualEnv-408403f5-d1e9-4a76-bcb0-33b8ef51a6ac/lib/python3.7/site-packages/kedro/framework/context/context.py in config_loader(self)
    454 
    455         """
--> 456         return self._get_config_loader()
    457 
    458     def _get_feed_dict(self) -> Dict[str, Any]:

/local_disk0/pythonVirtualEnvDirs/virtualEnv-408403f5-d1e9-4a76-bcb0-33b8ef51a6ac/lib/python3.7/site-packages/kedro/framework/context/context.py in _get_config_loader(self)
    438         if not isinstance(config_loader, ConfigLoader):
    439             raise KedroContextError(
--> 440                 f"Expected an instance of `ConfigLoader`, "
    441                 f"got `{type(config_loader).__name__}` instead."
    442             )

KedroContextError: Expected an instance of `ConfigLoader`, got `NoneType` instead.

Thanks in advance!

[merelcht]

Just to confirm @Vinay1233, you're seeing this error when doing kedro run? I've tried this on a new project and it seems to run fine. Can you double check that the version of Kedro you're running is the same one as in which the project was created? You can find the project version in pyproject.toml

[Vinay1233]

So im not using the CLI to run kedro, I run a kedro session by doing the below inside therun.mainfunction.

I did update the pyproject.toml with the correct project version too.

    with KedroSession.create(package_name,
        project_path=None,
        save_on_close=True,
        env=None,
        extra_params=extra_params) as session: 
                
        session.run(
            pipeline_name=None,
            tags=tags,
            runner=runner,
            node_names=node_names,
            from_nodes=from_nodes,
            to_nodes=to_nodes,
            from_inputs=from_inputs,
            load_versions=load_versions,
            )

[merelcht]

@Vinay1233 did you actually create the project itself with kedro 0.17.2 or did you only update the version? We made some changes to the structure, so simply updating the version won't take in all of those changes.

[Vinay1233]

ahh I see, sorry I just changed the version hoping that was all I needed to do.

Ill create a new project and test try it out

Thanks!

[Vinay1233]

I added configure_project(Path(__file__).parent.name) to my main function and works now.

Thanks for your help @MerelTheisenQB!

[merelcht]

I'm glad it's working now @Vinay1233 😄

[adasiitm]

Hello @Vinay1233, I am also getting same error using py3.7 & kedro 0.17.3 version. May I know where you added this? configure_project(Path(file).parent.name)
Thanks!