docs: Add code for experiments (#93)

* docs: Add code for experiments These use RAGAS to measure various retrieval configurations on the history of alexnet dataset. * ruff
DewyKB · Feb 28, 2024 · e641bd8 · e641bd8
1 parent bdbc989
commit e641bd8
Show file tree

Hide file tree

Showing 7 changed files with 2,933 additions and 1 deletion.
diff --git a/.gitignore b/.gitignore
@@ -2,4 +2,4 @@
 __pycache__
 .env
 .vscode/
-/dewy-client/dist
+/dewy-client/dist.venv
diff --git a/experiments/.gitignore b/experiments/.gitignore
@@ -0,0 +1,2 @@
+.env
+*.sqlite
diff --git a/experiments/README.md b/experiments/README.md
@@ -0,0 +1,16 @@
+# Experiments
+
+## RAG Configuration
+
+This experiment plays with some of the standard configuration options available in LangChain.
+It runs a variety of extraction, splitting, and retrieval configurations on the AlexNet PDF and Q/A set.
+
+To execute:
+
+```sh
+python rag_configuration.py run       # --help to see options for filtering configurations
+
+python rag_configuration.py clear     # to clear Tru DB
+python rag_configuration.py dashboard # to display dashboard
+python rag_configuration.py sreve     # to serve the browser dashboard
+```
diff --git a/experiments/__init__.py b/experiments/__init__.py
diff --git a/experiments/datasets/history_of_alexnet/rag_dataset.json b/experiments/datasets/history_of_alexnet/rag_dataset.json
diff --git a/experiments/datasets/history_of_alexnet/source_documents/HistoryOfAlexnet.pdf b/experiments/datasets/history_of_alexnet/source_documents/HistoryOfAlexnet.pdf