Avoid systematic deepcopy of inference datasets

[takikadiri]

Description

Actually kedro_mlflow_model create a new catalog called loaded_catalog where it declare all the pipeline_ml artifacts with the new filepath. see here
Our current problem is that each of these datasets are deep-copied between the kedro nodes, and some artifacts/datasets take a long time to be deep-copied (keras model for example), and this is not suitable in an API serving pattern.

We need to be able to avoid deepcopy of some (or all) datasets in the inference pipeline

Possible Implementation

Defining inference datasets type to MemoryDataset(name, copy_mode="assign") solve the problem, but can break some inference pipeline that mutate the dataset state between the nodes. We can configure the copy_mode in one of these two levels :

• Propose an option to redefine inference datasets type at PipelineML level
• Propose an options to redefine inference datasets type at kedro_pipeline_model level