Integrating langchainJS with fine tuned gemini model

[drorIvry]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import { ChatVertexAI } from "@langchain/google-vertexai";

    const model = new ChatVertexAI({
      temperature: 0.0,
      model: "projects/${PROJECT}/locations/us-central1/endpoints/${ID}",
    })

Description

I'm trying to use ChatVertexAI to work with a fine tuned version of gemini deployed via VertexAI.
using the above code I got the following error: Error: Unable to verify model params

I couldn't find anything in the documentation regarding this, anyone has an idea?

System Info

pnpm info langchain

langchain@0.1.34 | MIT | deps: 17 | versions: 265
Typescript bindings for langchain

platform mac/linux

node --version
v20.3.1

[dosubot[bot]]

The error "Unable to verify model params" you're encountering with ChatVertexAI when using a fine-tuned Gemini model on VertexAI suggests there's a mismatch or incorrect specification of the required model parameters. To address this, ensure you're including all necessary parameters, particularly if you're connecting to a custom or fine-tuned model. Here's how you might correctly initialize your ChatGoogleVertexAI instance, including a hypothetical customModelURL for fine-tuned models:

const chat = new ChatGoogleVertexAI({
  customModelURL: "your-custom-model-url", // Replace with your model's URL
  temperature: 0.7, // Adjust as needed
  maxOutputTokens: 256, // Adjust as needed
  topP: 0.9, // Adjust as needed
  topK: 40 // Adjust as needed
});

Replace "your-custom-model-url" with the actual URL of your model. Also, adjust the temperature, maxOutputTokens, topP, and topK parameters based on your model's requirements and your specific use case.

Ensure the following to troubleshoot further:

• The custom model URL is correctly specified and the model is accessible.
• Your VertexAI project and authentication credentials are correctly set up.
• The version of langchainjs you're using supports fine-tuned models on VertexAI.

For detailed examples and more information, consider looking into the langchainjs library's documentation and source code, especially around tests and type definitions related to Google VertexAI models. This should help you resolve the parameter verification issue and successfully integrate your fine-tuned Gemini model with ChatVertexAI.

Sources

• libs/langchain-community/src/chat_models/tests/chatgooglevertexai_web.int.test.ts
• langchain/src/types/googlevertexai-types.ts
• libs/langchain-community/src/types/googlevertexai-types.ts
• libs/langchain-community/src/chat_models/googlevertexai/index.ts
• examples/src/models/chat/integration_googlevertexai.ts
• examples/src/models/chat/chat_vertexai_agents.ts
• libs/langchain-google-vertexai-web/src/chat_models.ts
• libs/langchain-google-vertexai/src/chat_models.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Storkki]

@drorIvry Hi, did u found solution?

[drorIvry]

Sadly no