Flyte Callback

[peridotml]

Feature request

Hi! I am an OSS contributor to Flyte, a workflow orchestration tool. I am working on a HuggingFace plugin to integrate better with Hugging Face.

I am working on a FlyteCallback that will automatically integrate with Flyte's checkpointing, visualizations, and eventually logging!

I was hoping that we could add it to the integrations.py similar to other ML Tools, but I wanted to check with you all before making a PR.

Motivation

A good way to help Flyte users working with Hugging Face and Hugging Face users who end up using Flyte for orchestration.

Your contribution

I would clean up and extend the callback in this gist and create a PR.

[sgugger]

Hi there! You can definitely open a PR. integrations.py is open to the community and callbacks defined there are maintained by their authors and not by us. So as long as you're fine getting pinged by users who get a problem with this callback in the future, please share your new callback :-)