Performance issues while creating embeddings with ai-sdk-js

[prashantrakheja]

Describe the Bug

While creating embeddings with ai-sdk-js, the performance seems slow.

Given a set of doc chunks, while the python SDK for gen AI hub takes about 20-30 seconds, the ai-sdk-js takes about 3-4 minutes.

Steps to Reproduce

1. Create doc chunks
2. Embed with Python SDK
3. Embed with ai-sdk-js

The time taken using ai-sdk-js is much higher

Expected Behavior

Notes on improving performance -

In the python implementation of GenAI Hub SDK - I notice that the chunk_size is being overriden to 16 -

https://github.wdf.sap.corp/AI/generative-ai-hub-sdk/blob/main/gen_ai_hub/proxy/langchain/openai.py#L228

The default value of this chunk_size is 1000 -
https://github.com/langchain-ai/langchain/blob/master/libs/partners/openai/langchain_openai/embeddings/base.py#L218

In the ai-sdk-js - there doesn't seem to be any implementation wrt chunk_size - we are making a call for each chunk which can overwhelm AI Core Service -
https://github.com/SAP/ai-sdk-js/blob/main/packages/langchain/src/openai/embedding.ts#L29

Screenshots

No response

Used Versions

"@sap-ai-sdk/langchain": "^1.1.0",
"@types/node": "^22.7.3",
"typescript": "^5.6.2"

Code Examples

    const embeddings = new AzureOpenAiEmbeddingClient({
      modelName: MODEL_TEXT_EMBEDDING_ADA,
      maxRetries: MAX_RETRIES,
    });

    const args: HanaDBArgs = {
      connection: client,
      tableName: TERRAFORM_TABLE_JS,
    };

    const vectorStore = new HanaDB(embeddings, args);
    await vectorStore.initialize();

    const docs = await fetchDocs();

    const chunks = await splitDocumentsIntoChunksRecursively(docs, 1000, 100);

    const documents = chunks.map(chunk => ({
      pageContent: chunk, 
      metadata: { source: 'some-source' }  
    }));

    await vectorStore.addDocuments(documents);

Log File

No response

Affected Development Phase

Development

Impact

Inconvenience

Timeline

No response

Additional Context

No response

[marikaner]

Hey @prashantrakheja, we are looking into this. I will let you know once I have some insights.

[marikaner]

Hey @prashantrakheja, we believe we have fixed the issue. We sent all given chunks as individual requests, but now we changed this to happen in a batch. This issue will close once the PR is merged and we will release a new version soon.

[jjtang1985]

A new minor version has been released.