Exposing OpenCLIP embeddings

[jacobmarks]

What changes are proposed in this pull request?

Exposing prompt embedding capabilities for OpenCLIP model recently added to zoo

• Set can_embed_prompts to True
• Added an embed_prompts() method

Test the embedding directly:

import fiftyone as fo
import fiftyone.zoo as foz

model = foz.load_zoo_model("open-clip-torch")
embeddings = model.embed_prompts(["dog", "cat"])

Now you can also do natural language image searches with OpenCLIP:

import fiftyone as fo
import fiftyone.brain as fob
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

fob.compute_similarity(dataset, model="open-clip-torch", brain_key="oc_sim")
dataset.sort_by_similarity("cat")

Additional Notes

• We may want to go through the docs and wherever CLIP is used, offer this as an alternative
• Should add this to the model zoo listing
• At present, this only allows for using the DEFAULT OpenCLIP weights/architecture, not EVA-CLIP, MetaCLIP etc, because the natural language image search only works when you pass the name of the model into compute_similarity, not the model object itself!

What areas of FiftyOne does this PR affect?

• App: FiftyOne application changes
• Build: Build and test infrastructure changes
• Core: Core fiftyone Python library changes
• Documentation: FiftyOne documentation changes
• Other

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7eaeee9) 15.86% compared to head (12c3d32) 15.85%.
Report is 45 commits behind head on develop.

❗ Current head 12c3d32 differs from pull request most recent head 78a4463. Consider uploading reports for the commit 78a4463 to get more accurate results

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3960      +/-   ##
===========================================
- Coverage    15.86%   15.85%   -0.02%     
===========================================
  Files          731      731              
  Lines        81853    81878      +25     
  Branches      1093     1093              
===========================================
- Hits         12987    12979       -8     
- Misses       68866    68899      +33     

Flag	Coverage Δ
app	15.85% <ø> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[brimoor]

LGTM 🥇