You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
the KedroPipelineModel has a initial_catalog property which causes some problems. This initial_catalog can contain some Kedro Datasets but it's not necessary to log them when you train your model. because of this property I can't load my model anymore. I have to train it again.
I explain : when I trained my model I used a kedro home-made plugin to load a specific dataset (which has no impact for my model). After that, I updated this plugin independently of my ML project. Today, I want to load my model but I can't because the load function uses the old Kedro Catalog with my old plugin version which is not in my environnement anymore.
Context
It would be great if we can update the kedro-catalog (only dataset and not the artifacts for the model of course !) without having to retrain our models.
Possible Implementation
Log in Mlflow what is only necessary.
I hope my issue is clear.
thank you
The text was updated successfully, but these errors were encountered:
Hi, I can reproduce the issue, thank you very much for the feedback. To clarify, what happens here is the following:
the input of your inference pipeline is persisted in Kedro because you load it from the disk (e.g., pandas.ExcelDataSet)
after you log it in mlflow, it will be converted to a MemoryDataSet, and you directly pass a pandas Dataframe when you want to reuse it. Mlflow complains that you need to have openpyxl installed, while you never use it in your pipeline, and you don't need it to predict.
This extra dependency is not useful as you mention. I will remove it in a patch release soon.
Galileo-Galilei
changed the title
problem with the initial_catalog property of KedroPipelineModel
KedroPipelineModel requires unnecessary pipeline input dependencies to be executed
Feb 9, 2022
Hi @Galileo-Galilei
Description
the KedroPipelineModel has a
initial_catalog
property which causes some problems. Thisinitial_catalog
can contain some Kedro Datasets but it's not necessary to log them when you train your model. because of this property I can't load my model anymore. I have to train it again.I explain : when I trained my model I used a kedro home-made plugin to load a specific dataset (which has no impact for my model). After that, I updated this plugin independently of my ML project. Today, I want to load my model but I can't because the load function uses the old Kedro Catalog with my old plugin version which is not in my environnement anymore.
Context
It would be great if we can update the kedro-catalog (only dataset and not the artifacts for the model of course !) without having to retrain our models.
Possible Implementation
Log in Mlflow what is only necessary.
I hope my issue is clear.
thank you
The text was updated successfully, but these errors were encountered: