Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YOLOV8 port to keras-hub #1899

Open
wants to merge 21 commits into
base: master
Choose a base branch
from
Open

YOLOV8 port to keras-hub #1899

wants to merge 21 commits into from

Conversation

oarriaga
Copy link

@oarriaga oarriaga commented Oct 1, 2024

This PR ports YOLOV8 from keras-cv to keras-hub (#176). All necessary YOLOV8 functions are now found inside keras-hub:

  • Add CIOU loss.
  • Add missing masking functionality in the bounding_boxes module.
  • Add multibackend non maximum supression layer.
  • Add label encoder.
  • Build basic abstract object detector task class.
  • Add YOLOV8 backbone and detector.

Missing steps include:

  • Upload previous presets to Kaggle.
  • Remove skipping tests with presets.
  • Add colab with basic functionality.
  • Add weight transfer script from keras-cv to keras-hub.
  • Add training script.

Copy link
Collaborator

@divyashreepathihalli divyashreepathihalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @oarriaga left some initial comments

keras_hub/src/layers/modeling/non_max_suppression.py Outdated Show resolved Hide resolved
keras_hub/src/layers/modeling/non_max_suppression.py Outdated Show resolved Hide resolved
keras_hub/src/layers/modeling/non_max_suppression.py Outdated Show resolved Hide resolved
keras_hub/src/layers/modeling/non_max_suppression.py Outdated Show resolved Hide resolved
keras_hub/src/layers/modeling/non_max_suppression.py Outdated Show resolved Hide resolved
label_encoder=None,
prediction_decoder=None,
**kwargs,
):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

restructure the code to define all teh layers first, functional model next and config last with these comments
=== Layers ===
.
.
=== Functional model ===
.
.
=== Config ===
.
.
example : https://github.com/keras-team/keras-hub/blob/master/keras_hub/src/models/bert/bert_backbone.py#L92

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Divya, the current model applies multiple blocks of layers. That will imply that we would need to initialize many layers in the constructor. Moreover, the connections between those layers are not so straightforward as in Bert. What would you suggest? Shall we still initialize all layers in the layer block and connect them in the functional block?

def predict_step(self, *args):
outputs = super().predict_step(*args)
if isinstance(outputs, tuple):
return self.decode_predictions(outputs[0], args[-1]), outputs[1]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will model.fit work ?

@@ -0,0 +1,318 @@
import os

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you will need to add preprocessor flow - example follow resnet - https://github.com/keras-team/keras-hub/tree/master/keras_hub/src/models/resnet


@pytest.mark.large # Saving is slow, so mark these large.
def test_saved_model(self):
model = keras_hub.models.YOLOV8Detector(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


# TODO(tirthasheshpatel): Support updating prediction decoder in Keras Core.
@pytest.mark.skip(reason="Missing presets")
@pytest.mark.tf_keras_only
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no tf_keras_only in KerasHub

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants