optimize rerank with backend ref #579

a32543254 · 2024-08-29T08:19:12Z

Description

Optimize the rerank with backend ref module

model type	input shape	Mosec (s)	NS (s)	ratio
bge-reranker-large	[2, 14]	0.0537	0.0359	1.5x
bge-reranker-large	[24, 512]	2.610966	1.184117	2.2x

Issues

List the issue or RFC link this PR is working on. If there is no such link, please mark it as n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds new functionality)
Breaking change (fix or feature that would break existing design and interface)
Others (enhancement, documentation, validation, etc.)

Dependencies

List the newly introduced 3rd party dependency if exists.

Tests

Describe the tests that you ran to verify your changes.

lvliang-intel · 2024-08-29T13:14:07Z

@a32543254,

Please fix pre-commit issue and also add a test case.

lvliang-intel · 2024-09-02T06:50:31Z

@a32543254，
Please fix the Docker file issue.

a32543254 · 2024-09-03T04:14:51Z

@a32543254， Please fix the Docker file issue.

Thanks, I'm co-working with Suyue on this ut test now, will be fixed soon.

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

for more information, see https://pre-commit.ci Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

…t#568) * fix mismatched response format w/wo streaming guardrails * fix & debug * fix & rm debug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

…project#571) * fix mismatched response format w/wo streaming guardrails * fix & debug * fix & rm debug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * debug * debug * debug * fix pre-space and linebreak * fix pre-space and linebreak * fix single/double quote * fix single/double quote * remove debug * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add url validator Signed-off-by: BaoHuiling <huiling.bao@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add validation for video_url Signed-off-by: BaoHuiling <huiling.bao@intel.com> --------- Signed-off-by: BaoHuiling <huiling.bao@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* Reduced output. Signed-off-by: zepan <ze.pan@intel.com> * Output the location where the modified Dockerfile file is referenced. Signed-off-by: zepan <ze.pan@intel.com> * for test Signed-off-by: zepan <ze.pan@intel.com> * Restore test file. Signed-off-by: zepan <ze.pan@intel.com> --------- Signed-off-by: zepan <ze.pan@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add ut and refine logging code. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update microservice port. --------- Co-authored-by: root <root@idc708073.jf.intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* adding lancedb to langchain vectorstores Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: lvliang-intel <liang1.lv@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

for more information, see https://pre-commit.ci Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

for more information, see https://pre-commit.ci Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* multimodal embedding for MM RAG for videos Signed-off-by: Tiep Le <tiep.le@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * develop data prep first commit Signed-off-by: Tiep Le <tiep.le@intel.com> * develop dataprep microservice for multimodal data Signed-off-by: Tiep Le <tiep.le@intel.com> * multimodal langchain for dataprep Signed-off-by: Tiep Le <tiep.le@intel.com> * update README Signed-off-by: Tiep Le <tiep.le@intel.com> * update README Signed-off-by: Tiep Le <tiep.le@intel.com> * update README Signed-off-by: Tiep Le <tiep.le@intel.com> * update README Signed-off-by: Tiep Le <tiep.le@intel.com> * cosmetic Signed-off-by: Tiep Le <tiep.le@intel.com> * test for multimodal dataprep Signed-off-by: Tiep Le <tiep.le@intel.com> * update test Signed-off-by: Tiep Le <tiep.le@intel.com> * update test Signed-off-by: Tiep Le <tiep.le@intel.com> * update test Signed-off-by: Tiep Le <tiep.le@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cosmetic update Signed-off-by: Tiep Le <tiep.le@intel.com> * remove langsmith Signed-off-by: Tiep Le <tiep.le@intel.com> * update API to remove /dataprep from API names and remove langsmith Signed-off-by: Tiep Le <tiep.le@intel.com> * update test Signed-off-by: Tiep Le <tiep.le@intel.com> * update the error message per PR reviewer Signed-off-by: Tiep Le <tiep.le@intel.com> --------- Signed-off-by: Tiep Le <tiep.le@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

…project#342) * nb Signed-off-by: Berke <berkecanrizai1@gmail.com> * init changes Signed-off-by: Berke <berkecanrizai1@gmail.com> * docker Signed-off-by: Berke <berkecanrizai1@gmail.com> * example data Signed-off-by: Berke <berkecanrizai1@gmail.com> * docs(readme): update, add commands Signed-off-by: Berke <berkecanrizai1@gmail.com> * fix: formatting, data sources Signed-off-by: Berke <berkecanrizai1@gmail.com> * docs(readme): update instructions, add comments Signed-off-by: Berke <berkecanrizai1@gmail.com> * fix: rm unused parts Signed-off-by: Berke <berkecanrizai1@gmail.com> * fix: image name, compose env vars Signed-off-by: Berke <berkecanrizai1@gmail.com> * fix: rm unused part Signed-off-by: Berke <berkecanrizai1@gmail.com> * fix: logging name Signed-off-by: Berke <berkecanrizai1@gmail.com> * fix: env var Signed-off-by: Berke <berkecanrizai1@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Berke <berkecanrizai1@gmail.com> * fix: rename pw docker Signed-off-by: Berke <berkecanrizai1@gmail.com> * docs(readme): update input sources Signed-off-by: Berke <berkecanrizai1@gmail.com> * nb Signed-off-by: Berke <berkecanrizai1@gmail.com> * init changes Signed-off-by: Berke <berkecanrizai1@gmail.com> * fix: formatting, data sources Signed-off-by: Berke <berkecanrizai1@gmail.com> * docs(readme): update instructions, add comments Signed-off-by: Berke <berkecanrizai1@gmail.com> * fix: rm unused part Signed-off-by: Berke <berkecanrizai1@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Berke <berkecanrizai1@gmail.com> * fix: rename pw docker Signed-off-by: Berke <berkecanrizai1@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Berke <berkecanrizai1@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * feat: mv vector store, naming, clarify instructions, improve ingestion components Signed-off-by: Berke <berkecanrizai1@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests: add pw retriever test fix: update docker to include libmagic Signed-off-by: Berke <berkecanrizai1@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * implement suggestions from review, entrypoint, reqs, comments, https_proxy. Signed-off-by: Berke <berkecanrizai1@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: update docker tags in test and readme Signed-off-by: Berke <berkecanrizai1@gmail.com> * tests: add separate pathway vectorstore test Signed-off-by: Berke <berkecanrizai1@gmail.com> --------- Signed-off-by: Berke <berkecanrizai1@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* initial commit Signed-off-by: BaoHuiling <huiling.bao@intel.com> * save Signed-off-by: BaoHuiling <huiling.bao@intel.com> * add readme, test script, fix bug Signed-off-by: BaoHuiling <huiling.bao@intel.com> * update video URL Signed-off-by: BaoHuiling <huiling.bao@intel.com> * use default Signed-off-by: BaoHuiling <huiling.bao@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update core dependency Signed-off-by: BaoHuiling <huiling.bao@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * use p 5000 Signed-off-by: BaoHuiling <huiling.bao@intel.com> * use 5037 Signed-off-by: BaoHuiling <huiling.bao@intel.com> * update ctnr name Signed-off-by: BaoHuiling <huiling.bao@intel.com> * remove langsmith Signed-off-by: BaoHuiling <huiling.bao@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add rerank algo desc in readme Signed-off-by: BaoHuiling <huiling.bao@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: BaoHuiling <huiling.bao@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: chen, suyue <suyue.chen@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: zepan <ze.pan@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: BaoHuiling <huiling.bao@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: zepan <ze.pan@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add microservice for intent detection Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update license copyright Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * add ut Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * refine Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update folder Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix test Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> --------- Signed-off-by: Liangyx2 <yuxiang.liang@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: zepan <ze.pan@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* Check if the document exists. Signed-off-by: zepan <ze.pan@intel.com> * Add flag output. Signed-off-by: zepan <ze.pan@intel.com> * Modify nginx readme. Signed-off-by: zepan <ze.pan@intel.com> * Modify document detection logic Signed-off-by: zepan <ze.pan@intel.com> --------- Signed-off-by: zepan <ze.pan@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

comps/reranks/neural-speed/reranking_neuralspeed_svc.py

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

for more information, see https://pre-commit.ci

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

for more information, see https://pre-commit.ci

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Signed-off-by: chensuyue <suyue.chen@intel.com>

a32543254 requested a review from XuhuiRen as a code owner August 29, 2024 08:19

a32543254 and others added 26 commits September 2, 2024 23:33

add rerank with neural speed

0837cc4

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

add the code

7a2a90b

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

add the code

4bf3b59

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

aa28c54

for more information, see https://pre-commit.ci Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Refine Dataprep Milvus MS (opea-project#570)

6f92766

Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

final version

cff5686

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

1e00804

for more information, see https://pre-commit.ci Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

update the readme

45c3940

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

add the sign

d20d16f

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

90dae42

for more information, see https://pre-commit.ci Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

fix error for pre ci

eed64ed

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

add the ut

fd72d52

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

update docker file

7f5228d

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

update CI test log achieve (opea-project#577)

1845be6

Signed-off-by: chensuyue <suyue.chen@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Add Scan Container. (opea-project#560)

0687fd3

Signed-off-by: zepan <ze.pan@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

fix SearchedMultimodalDoc in docarray (opea-project#583)

6860e0c

Signed-off-by: BaoHuiling <huiling.bao@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

update image build yaml (opea-project#529)

2ea22d0

Signed-off-by: chensuyue <suyue.chen@intel.com> Signed-off-by: zepan <ze.pan@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Liangyx2 and others added 6 commits September 2, 2024 23:33

Make the scanning method optional. (opea-project#580)

d868291

Signed-off-by: zepan <ze.pan@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

add code owners (opea-project#586)

c8c5cf7

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

remove revision for tei (opea-project#584)

f7a469c

Signed-off-by: letonghan <letong.han@intel.com> Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

fix ut issue

98fd44c

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

a32543254 force-pushed the main branch from eb3ef45 to 98fd44c Compare September 3, 2024 06:33

a32543254 requested review from XinyuYe-Intel, letonghan, Spycsh and lvliang-intel as code owners September 3, 2024 06:33

a32543254 and others added 2 commits September 3, 2024 14:35

Merge branch 'main' into main

6976116

merge the main

138edc3

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

lvliang-intel reviewed Sep 3, 2024

View reviewed changes

comps/reranks/neural-speed/reranking_neuralspeed_svc.py Outdated Show resolved Hide resolved

a32543254 and others added 10 commits September 5, 2024 02:01

align with new pipeline

2cc419c

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

0e8d887

for more information, see https://pre-commit.ci

align with newest pipeline

59be42b

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

f98c654

for more information, see https://pre-commit.ci

Merge branch 'main' into main

dcfcade

upload code

6662397

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

update the ut

b6c0d9f

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

Merge branch 'main' into main

70f4b8a

add docker path

5729e0f

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

add the docker path

3fc4d7a

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

lvliang-intel approved these changes Sep 10, 2024

View reviewed changes

Spycsh approved these changes Sep 10, 2024

View reviewed changes

lvliang-intel merged commit d76751a into opea-project:main Sep 10, 2024
9 of 10 checks passed

lkk12014402 pushed a commit that referenced this pull request Sep 19, 2024

add secrets for test (#579)

3c9e2aa

Signed-off-by: chensuyue <suyue.chen@intel.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

optimize rerank with backend ref #579

optimize rerank with backend ref #579

a32543254 commented Aug 29, 2024 •

edited

Loading

lvliang-intel commented Aug 29, 2024

lvliang-intel commented Sep 2, 2024

a32543254 commented Sep 3, 2024

optimize rerank with backend ref #579

optimize rerank with backend ref #579

Conversation

a32543254 commented Aug 29, 2024 • edited Loading

Description

Issues

Type of change

Dependencies

Tests

lvliang-intel commented Aug 29, 2024

lvliang-intel commented Sep 2, 2024

a32543254 commented Sep 3, 2024

a32543254 commented Aug 29, 2024 •

edited

Loading