Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update qdrant notebook for new qdrant vectordb #3140

Merged
merged 5 commits into from
Jul 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion autogen/agentchat/contrib/vectordb/base.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,16 @@
from typing import Any, List, Mapping, Optional, Protocol, Sequence, Tuple, TypedDict, Union, runtime_checkable
from typing import (
Any,
Callable,
List,
Mapping,
Optional,
Protocol,
Sequence,
Tuple,
TypedDict,
Union,
runtime_checkable,
)

Metadata = Union[Mapping[str, Any], None]
Vector = Union[Sequence[float], Sequence[int]]
Expand Down Expand Up @@ -49,6 +61,9 @@ class VectorDB(Protocol):

active_collection: Any = None
type: str = ""
embedding_function: Optional[Callable[[List[str]], List[List[float]]]] = (
None # embeddings = embedding_function(sentences)
)

def create_collection(self, collection_name: str, overwrite: bool = False, get_or_create: bool = True) -> Any:
"""
Expand Down
8 changes: 7 additions & 1 deletion autogen/agentchat/contrib/vectordb/qdrant.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ def __init__(
collection_options: dict | The options for creating the collection.
kwargs: dict | Additional keyword arguments.
"""
self.client: QdrantClient = client if client is not None else QdrantClient(location=":memory:")
self.client: QdrantClient = client or QdrantClient(location=":memory:")
self.embedding_function = FastEmbedEmbeddingFunction() or embedding_function
self.collection_options = collection_options
self.content_payload_key = content_payload_key
Expand All @@ -102,6 +102,10 @@ def __init__(
def create_collection(self, collection_name: str, overwrite: bool = False, get_or_create: bool = True) -> None:
"""
Create a collection in the vector database.
Case 1. if the collection does not exist, create the collection.
Case 2. the collection exists, if overwrite is True, it will overwrite the collection.
Case 3. the collection exists and overwrite is False, if get_or_create is True, it will get the collection,
otherwise it raise a ValueError.

Args:
collection_name: str | The name of the collection.
Expand All @@ -122,6 +126,8 @@ def create_collection(self, collection_name: str, overwrite: bool = False, get_o
vectors_config=models.VectorParams(size=embeddings_size, distance=models.Distance.COSINE),
**self.collection_options,
)
elif not get_or_create:
raise ValueError(f"Collection {collection_name} already exists.")

def get_collection(self, collection_name: str = None):
"""
Expand Down

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion website/blog/2023-10-18-RetrieveChat/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -483,4 +483,4 @@ The online app and the source code are hosted in [HuggingFace](https://huggingfa
You can check out more example notebooks for RAG use cases:
- [Automated Code Generation and Question Answering with Retrieval Augmented Agents](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_RetrieveChat.ipynb)
- [Group Chat with Retrieval Augmented Generation (with 5 group member agents and 1 manager agent)](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat_RAG.ipynb)
- [Automated Code Generation and Question Answering with Qdrant based Retrieval Augmented Agents](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_qdrant_RetrieveChat.ipynb)
- [Automated Code Generation and Question Answering with Qdrant based Retrieval Augmented Agents](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_RetrieveChat_qdrant.ipynb)
2 changes: 1 addition & 1 deletion website/docs/Examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Links to notebook examples:

- Automated Task Solving with Code Generation, Execution & Debugging - [View Notebook](/docs/notebooks/agentchat_auto_feedback_from_code_execution)
- Automated Code Generation and Question Answering with Retrieval Augmented Agents - [View Notebook](/docs/notebooks/agentchat_RetrieveChat)
- Automated Code Generation and Question Answering with [Qdrant](https://qdrant.tech/) based Retrieval Augmented Agents - [View Notebook](/docs/notebooks/agentchat_qdrant_RetrieveChat)
- Automated Code Generation and Question Answering with [Qdrant](https://qdrant.tech/) based Retrieval Augmented Agents - [View Notebook](/docs/notebooks/agentchat_RetrieveChat_qdrant)

### Multi-Agent Collaboration (>3 Agents)

Expand Down
2 changes: 1 addition & 1 deletion website/docs/ecosystem/pgvector.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

[PGVector](https://github.com/pgvector/pgvector) is an open-source vector similarity search for Postgres.

- [PGVector + AutoGen Code Examples](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_pgvector_RetrieveChat.ipynb)
- [PGVector + AutoGen Code Examples](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_RetrieveChat_pgvector.ipynb)
2 changes: 1 addition & 1 deletion website/docs/installation/Optional-Dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ Example notebooks:

[Group Chat with Retrieval Augmented Generation (with 5 group member agents and 1 manager agent)](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat_RAG.ipynb)

[Automated Code Generation and Question Answering with Qdrant based Retrieval Augmented Agents](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_qdrant_RetrieveChat.ipynb)
[Automated Code Generation and Question Answering with Qdrant based Retrieval Augmented Agents](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_RetrieveChat_qdrant.ipynb)

## Teachability

Expand Down
4 changes: 2 additions & 2 deletions website/docs/topics/retrieval_augmentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,8 @@ ragproxyagent.initiate_chat(
## More Examples and Notebooks
For more detailed examples and notebooks showcasing the usage of retrieval augmented agents in AutoGen, refer to the following:
- Automated Code Generation and Question Answering with Retrieval Augmented Agents - [View Notebook](/docs/notebooks/agentchat_RetrieveChat)
- Automated Code Generation and Question Answering with [PGVector](https://github.com/pgvector/pgvector) based Retrieval Augmented Agents - [View Notebook](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_pgvector_RetrieveChat.ipynb)
- Automated Code Generation and Question Answering with [Qdrant](https://qdrant.tech/) based Retrieval Augmented Agents - [View Notebook](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_qdrant_RetrieveChat.ipynb)
- Automated Code Generation and Question Answering with [PGVector](https://github.com/pgvector/pgvector) based Retrieval Augmented Agents - [View Notebook](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_RetrieveChat_pgvector.ipynb)
- Automated Code Generation and Question Answering with [Qdrant](https://qdrant.tech/) based Retrieval Augmented Agents - [View Notebook](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_RetrieveChat_qdrant.ipynb)
- Chat with OpenAI Assistant with Retrieval Augmentation - [View Notebook](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_oai_assistant_retrieval.ipynb)
- **RAG**: Group Chat with Retrieval Augmented Generation (with 5 group member agents and 1 manager agent) - [View Notebook](/docs/notebooks/agentchat_groupchat_RAG)

Expand Down
Loading