[Feature Request]: Add threshold to retrieval config

[szwang-seedtech]

Is your feature request related to a problem? Please describe.

Hello,

The current rag seems to list the top few numbers of chunks into the retrieval content, and then filter them by determing whether they are related to the question, and this will still waste a certain number of tokens at the first place.

Is it possible to add a threshold setting to the retrieval config, so that only the similarity over the threshold value will be listed in the retrieval content?

Describe the solution you'd like

No response

Additional context

No response

[thinkall]

Hi @szwang-seedtech , we don't support a threshold, but you can set context_max_tokens to control the maximum number of tokens in the context, the default is max_tokens * 0.8 and max_token means the token limit of the model.

[szwang-seedtech]

@thinkall Hello, I am just wondering, will the threshold feature be added in the future. According to my test, the unrelated information in the retrieval_content will actually make an impact on the replies from agents.

[thinkall]

@thinkall Hello, I am just wondering, will the threshold feature be added in the future. According to my test, the unrelated information in the retrieval_content will actually make an impact on the replies from agents.

Yes, this is a good-to-have feature. We're currently working on a refactor of RAG, this is on the to-do list #1657

[szwang-seedtech]

I see. Thank you. I will close this request then : )