Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add warning when artifact store cannot be loaded #2011

Merged

Conversation

strickvl
Copy link
Contributor

@strickvl strickvl commented Nov 2, 2023

Added:

  • a more verbose warning when trying to import an artifact store flavor without all dependencies installed
  • a boolean flag to specify that you want artifacts loading using only the current stack's artifact store

Pre-requisites

Please ensure you have done the following:

  • I have read the CONTRIBUTING.md document.
  • If my change requires a change to docs, I have updated the documentation accordingly.
  • If I have added an integration, I have updated the integrations table and the corresponding website section.
  • I have added tests to cover my changes.
  • I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Other (add details above)

@strickvl strickvl added enhancement New feature or request internal To filter out internal PRs and issues labels Nov 2, 2023
Copy link
Contributor

@schustmi schustmi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I misremembered this function when talking about it on discord. I thought this was some kind of utility function which takes as input a run_name, step_name and output_name and then fetches/loads the artifact.

If this is really just loading a specific artifact then it seems like a user error to try loading it when the integrations aren't available. Maybe in that case my suggestion of having this boolean flag does not make much sense, and we should rather just go with the improved error message that you already implemented as part of this PR as well?

src/zenml/utils/artifact_utils.py Outdated Show resolved Hide resolved
@strickvl
Copy link
Contributor Author

strickvl commented Nov 2, 2023

@schustmi the improved error message doesn't help with the fact that I might not want it to assume to use an artifact store other than the one in my current stack. (Agreed though that not having the right dependencies installed is just user error which we can make a bit nicer with a nicer error message).

@schustmi
Copy link
Contributor

schustmi commented Nov 2, 2023

But isn't that your fault then, if you want artifacts only from a specific store, then you should handle that when fetching the ArtifactResponseModel that you pass into this function. E.g. Client.list_artifacts(artifact_store_id=...)?

@strickvl
Copy link
Contributor Author

strickvl commented Nov 2, 2023

@schustmi ok you're probably right. The confusion (when using it) came when getting an artifact via the last_successful_run (where potentially this might have been from a different artifact store and which (when getting the artifact) was calling load_artifact under the hood. This flag wouldn't have helped in that case. I'll remove it.

@schustmi
Copy link
Contributor

schustmi commented Nov 2, 2023

Yes, the flag should maybe be on the last_succesful_run method instead, where people can use it to fetch just the runs on the active stack/artifact store?
But then it's probably too far removed from the artifact loading for anyone to realize its use..

src/zenml/utils/artifact_utils.py Outdated Show resolved Hide resolved
strickvl and others added 2 commits November 2, 2023 17:50
Co-authored-by: Michael Schuster <schustmi@users.noreply.github.com>
@strickvl strickvl changed the title Add flag to specify artifact loading only using current stack's artifact store Add warning when artifact store cannot be loaded Nov 5, 2023
@stefannica stefannica merged commit b09fa8b into develop Nov 10, 2023
24 of 26 checks passed
@stefannica stefannica deleted the feature/OSS-2584-artifact-loading-current-stack-only branch November 10, 2023 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request internal To filter out internal PRs and issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants