Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin/mlflow custom flavor #1069

Merged
merged 3 commits into from
Jul 29, 2024
Merged

Plugin/mlflow custom flavor #1069

merged 3 commits into from
Jul 29, 2024

Conversation

skrawcz
Copy link
Collaborator

@skrawcz skrawcz commented Jul 27, 2024

Adds the ability to save/load custom flavors for MLFlow.

Current features

Currently, the MLFlow plugin would allow for 2 main patterns.

to.mlflow(...)  # infer the flavor

to.mlflow(..., flavor="sklearn")  # mlflow supported flavor 

^Note that this should support the pyfunc flavor, which is a convenient way to create custom estimators (MLFlow reference)

Solution

The most straight forward solution is to allow flavor to receive the Python module defining the custom flavor.

import custom_flavor
to.mlflow(..., flavor=custom_flavor)

consequently, this would also work

import mlflow
to.mlflow(..., flavor=mlflow.sklearn)

Changes

  • mlflow data saver and data loaders

How I tested this

  • via tests

Notes

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future TODOs are captured in comments
  • Project documentation has been updated if adding/changing functionality.

zilto and others added 3 commits July 26, 2024 20:48
Thanks for adding the custom flavor option to mlflow. I tested it and only the flavor specific kwargs somehow got mixed up and tried to be called somewhere else which lead to a fatal error.

Changes
renamed class attribute to avoid conflicting calls

How I tested this
added unit test that doubles as minimal example for reproducing the error

Notes
only tested the DataSaver (due to internal legacy structure, DataLoader is not utilised/hard to implement)

* working mlflow plugin need to add kwargs for custom flavor

* flavor kwargs used outside of mlflow saver/loader class

* updated class description

* flavor kwargs used outside of mlflow saver/loader class

* removed incidental files

---------

Co-authored-by: Jernej Frank <jernej.frank@oxehealth.com>
To complement saver.
@skrawcz skrawcz requested a review from elijahbenizzy July 27, 2024 04:53
@skrawcz skrawcz merged commit 1a19dd2 into main Jul 29, 2024
24 checks passed
@skrawcz skrawcz deleted the plugin/mlflow-custom-flavor branch July 29, 2024 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants