New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add end-to-end RAG example #966

Open

assaftibm opened this issue Jun 30, 2024 · 1 comment · May be fixed by #953

Assignees

Labels

Member

assaftibm commented Jun 30, 2024

I created a branch: and PR in draft state - #953

The task is there, but it fails to run because rouge is missing. I believe the metric is in Unitxt, just need to fix the pointer in the catalog.
Then we need to add an example that read ClapNQ from HF, samples k examples from it, creates temporary datasets of benchmark and documents, and runs RAG. There is the example that @yoavkatz shared in Slack - we can build on top of that. Or use ChromaDB as an in-memory vector-store.

assaftibm added the enhancement label

assaftibm assigned benjaminsznajder

Collaborator

benjaminsznajder commented Jul 23, 2024

The metrics and tasks are added, for clarity in another issue and PR

benjaminsznajder linked a pull request

that will close this issue

adding the end2end rag task #953

Draft

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment