OpenSearchVectorSearch fails to detect AOSS is enabled when using RequestAWS4SignerAuth from opensearch-py

[slangenbach]

System Info

langchain 0.0.335
macOS 14.1.1 / Windows 10.0.19045
Python 3.11.6 / 3.10.9

Who can help?

@naveentatikonda

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

Install dependencies

pip install langchain opensearch-py boto3

Import packages

import boto3
from langchain.embeddings import FakeEmbeddings
from langchain.vectorstores.opensearch_vector_search import OpenSearchVectorSearch
from opensearchpy import RequestsAWSV4SignerAuth, RequestsHttpConnection

Create AOSS auth

boto_session = boto3.Session()
aoss_auth = RequestsAWSV4SignerAuth(credentials=boto_session.get_credentials(), region=boto_session.region_name, service="aoss")

Create AOSS vector search class

embeddings = FakeEmbeddings(size=42)
database = OpenSearchVectorSearch(
    opensearch_url="https://some_aoss_endpoint:443",
    http_auth=aoss_auth,
    connection_class=RequestsHttpConnection,
    index_name="test",
    embedding_function=embeddings,

Check if AOSS vector search identifies itself as AOSS

database.is_aoss

Returns False

Expected behavior

AOSS should correctly identify itself as AOSS given the configuration above, which means that database.is_aoss should return True in this case.

The issue is related to how the internal _is_aoss_enabled method works:

def _is_aoss_enabled(http_auth: Any) -> bool:
    """Check if the service is http_auth is set as `aoss`."""
    if (
        http_auth is not None
        and hasattr(http_auth, "service")
        and http_auth.service == "aoss"
    ):
        return True
    return False

The RequestsAWSV4SignerAuth does not expose the service property directly, but via the signer attribute. In order to respect that, is_aoss_enabled should be adapted as following:

def _is_aoss_enabled(http_auth: Any) -> bool:
    """Check if the service set via http_auth equals `aoss`."""
    if http_auth is not None:
        if hasattr(http_auth, "service") and http_auth.service == "aoss":
            return True
        elif hasattr(http_auth, "signer") and http_auth.signer.service == "aoss":
            return True
        else:
            return False
    return False

[dosubot]

🤖

Hello @slangenbach! I'm Dosu, a bot here to assist you with your issues, answer your questions, and guide you on your journey to becoming a contributor. While we're waiting for a human maintainer to join us, feel free to ask me anything about the LangChain repo. It's great to meet you!

Thank you for the detailed report and the proposed solution. It seems like the OpenSearchVectorSearch class in LangChain is not correctly identifying the AOSS service when using RequestAWS4SignerAuth from opensearch-py. This is due to the fact that the _is_aoss_enabled method is checking for a service attribute in the http_auth object, which RequestAWS4SignerAuth does not expose directly.

Your proposed solution to check for the signer attribute in the http_auth object and then check if its service attribute is set to "aoss" seems to be a valid workaround for this issue. Here's the adapted _is_aoss_enabled method:

def _is_aoss_enabled(http_auth: Any) -> bool:
    """Check if the service set via http_auth equals `aoss`."""
    if http_auth is not None:
        if hasattr(http_auth, "service") and http_auth.service == "aoss":
            return True
        elif hasattr(http_auth, "signer") and http_auth.signer.service == "aoss":
            return True
        else:
            return False
    return False

This change should allow the OpenSearchVectorSearch class to correctly identify when AOSS is enabled using RequestAWS4SignerAuth.

However, as a maintainer, I don't have the ability to create pull requests or issues. I would recommend you to create a pull request with this change in the LangChain repository. This way, the maintainers can review your changes and merge them if they are appropriate.

Please let me know if you need further assistance.

Sources

• libs/langchain/langchain/vectorstores/opensearch_vector_search.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[timvw]

Can not reproduce this on
langchain 0.0.348
opensearch-py 2.4.2

With RequestsAWSV4SignerAuth and AWSV4SignerAuth(credentials, region, 'aoss') as auth it correctly detects aoss..

[slangenbach]

Apologies, you are correct @timvw.
I was using an older version open opensearchpy which indeed had this issue, but it was fixed with version v2.4.2