Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MlflowArtifactDataSet does not work with PartitionedDataSet #258

Closed
Galileo-Galilei opened this issue Nov 4, 2021 · 0 comments · Fixed by #287
Closed

MlflowArtifactDataSet does not work with PartitionedDataSet #258

Galileo-Galilei opened this issue Nov 4, 2021 · 0 comments · Fixed by #287
Assignees
Labels
bug Something isn't working

Comments

@Galileo-Galilei
Copy link
Owner

Description

It is not possible to store a PartitionedDataSet as an mlflow artifact with the MlflowArtifactDataSet.

Context

I had a use case where I need to save a dict with many small result tables to mlflow, and I tried to use PartitionedDataSet for this.

Steps to Reproduce

# catalog.yml

my_dataset:
    type: kedro_mlflow.io.artifacts.MlflowArtifactDataSet
    data_set:
        type: PartitionedDataSet  # or any valid kedro DataSet
        path: /path/to/a/local/folder # the attribute is "path", and not "filepath"!
        dataset: "pandas.CSVDataSet"

then save a dict using this dataset:

catalog.save("my_dataset", dict("a": pd.DataFrame(data=[1,2,3], columns=["a"], "b": pd.DataFrame(data=[1,2,3], columns=["b"])

Expected Result

The 2 Dataframes should be logged as artifacts in the current mlflow run.

Actual Result

An error dataset has not attribute "_filepath" is raised.

Does the bug also happen with the last version on master?

Yes

Potential solution

The error comes from this line:

maybe we can add a better condition here to default to "path" if there is no "filepath" attribute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant