Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup ORT for Text and Images #139

Merged
merged 16 commits into from
Nov 26, 2024
Merged

Setup ORT for Text and Images #139

merged 16 commits into from
Nov 26, 2024

Conversation

HAKSOAT
Copy link
Collaborator

@HAKSOAT HAKSOAT commented Oct 30, 2024

Main Changes

This pull requests made the following changes:

  • Removed dependence on FastEmbed to use ORT solely
  • Added support for ClipViT Text model
  • Implements Preprocessors and Postprocessors for Image inputs
  • Implements Preprocessors and Postprocessors for Text inputs
  • Brushed up the Python demo file

On removing dependence on FastEmbed

This is because ORT was used in the implementation for ClipViT, meaning other models could as well just use ORT. The reason for using ORT is because FastEmbed takes away control on how preprocessing works from us, taking raw texts and file names as inputs. Fastembed uses ORT under the hood and a lot of their utility methods are private, so can't be called directly.

Copy link

github-actions bot commented Oct 30, 2024

Test Results

153 tests   153 ✅  2m 39s ⏱️
  8 suites    0 💤
  2 files      0 ❌

Results for commit 4f7d123.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Oct 30, 2024

Benchmark Results

group                                                        main                                   pr
-----                                                        ----                                   --
store_batch_insertion_without_predicates/size_100            1.00    823.5±5.88µs        ? ?/sec    1.00    825.4±4.85µs        ? ?/sec
store_batch_insertion_without_predicates/size_1000           1.02      9.4±0.13ms        ? ?/sec    1.00      9.2±0.08ms        ? ?/sec
store_batch_insertion_without_predicates/size_10000          1.10     94.7±0.66ms        ? ?/sec    1.00     85.9±3.67ms        ? ?/sec
store_batch_insertion_without_predicates/size_100000         1.02   855.9±72.67ms        ? ?/sec    1.00   838.0±76.24ms        ? ?/sec
store_retrieval_no_condition/size_100                        1.00      2.6±0.05ms        ? ?/sec    1.04      2.7±0.06ms        ? ?/sec
store_retrieval_no_condition/size_1000                       1.00     18.7±0.13ms        ? ?/sec    1.04     19.4±0.32ms        ? ?/sec
store_retrieval_no_condition/size_10000                      1.00    186.0±1.60ms        ? ?/sec    1.03    192.5±3.27ms        ? ?/sec
store_retrieval_no_condition/size_100000                     1.00  1893.7±30.97ms        ? ?/sec    1.01  1918.0±20.76ms        ? ?/sec
store_retrieval_non_linear_kdtree/size_100                   1.00      2.8±0.02ms        ? ?/sec    1.03      2.8±0.03ms        ? ?/sec
store_retrieval_non_linear_kdtree/size_1000                  1.00     20.7±0.25ms        ? ?/sec    1.02     21.1±0.19ms        ? ?/sec
store_retrieval_non_linear_kdtree/size_10000                 1.00    206.3±3.98ms        ? ?/sec    1.01    208.9±2.44ms        ? ?/sec
store_retrieval_non_linear_kdtree/size_100000                1.00       2.1±0.03s        ? ?/sec    1.00       2.1±0.03s        ? ?/sec
store_sequential_insertion_without_predicates/size_100       1.01  1715.6±32.70µs        ? ?/sec    1.00  1698.1±35.04µs        ? ?/sec
store_sequential_insertion_without_predicates/size_1000      1.00     17.5±0.35ms        ? ?/sec    1.04     18.2±0.42ms        ? ?/sec
store_sequential_insertion_without_predicates/size_10000     1.00    179.4±2.77ms        ? ?/sec    1.01    181.8±2.92ms        ? ?/sec
store_sequential_insertion_without_predicates/size_100000    1.00  1693.0±37.25ms        ? ?/sec    1.10  1858.7±28.70ms        ? ?/sec

@HAKSOAT HAKSOAT marked this pull request as draft October 30, 2024 03:49
@@ -16,6 +17,7 @@ pub enum StringAction {
pub enum ImageAction {
ResizeImage,
ErrorIfDimensionsMismatch,
ModelPreprocessing
Copy link
Owner

@deven96 deven96 Nov 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching to using ModelProcessing and SkipProcessing

@HAKSOAT HAKSOAT marked this pull request as ready for review November 26, 2024 10:17
@HAKSOAT HAKSOAT changed the title Rough setup for ClipVit Text Setup ORT for Text and Images Nov 26, 2024
@deven96 deven96 merged commit b6a1769 into main Nov 26, 2024
5 checks passed
@deven96 deven96 deleted the feat/clipvittext branch November 27, 2024 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants