Google Vertex API support

[madppiper]

This commit adds support for the google cloud vertex API. It relies on https://github.com/googleapis/nodejs-vertexai to connect.

To enable:

1. Select or create a Google Cloud project.
2. Enable billing for your project.
3. Enable the Vertex AI API.
4. Set up authentication with a service account
   so you can access the API from your local workstation.

The service account credentials file can be imported as an environmental variable:

    GOOGLE_APPLICATION_CREDENTIALS = clientid.json

Make sure docker has access to the file. Afterwards Google Vertex endpoints can be configured as following:

MODELS=`[
//...
    {
       "name": "gemini-1.0-pro", //model-name
       "displayName": "Vertex code-bison",
       "location": "europe-west3",
       "apiEndpoint": "", //alternative api endpoint url
       "endpoints" : [{
         "type": "vertex"
       }]
     },
]`

Tested locally, but I had to merge it over to a new branch, to make the reviewing process easier - so please evaluate. Setting up the google permissions can be a bit tricky and model availability can differ for location & your own access level. So beware of this if a connection fails.

[nsarrazin]

Hi! Thanks for the contribution, I'll take a look soon 😄

[madppiper]

Hi! Thanks for the contribution, I'll take a look soon 😄

No problem - you guys are doing a fantastic job. Let me know if this requires any improvements.

[nsarrazin]

First of all thanks for the PR! It looks great 🤗

Two small things I noticed:

• Could you run npm run format so the linter checks pass? 😁
• Can you install @google-cloud/vertexai as an optional dependency like for the other endpoints?

And I was wondering, would it be bad practice to put the content of the JSON credentials into an env variable secret rather than specifying the file path and reading the file? Trying to read the file could cause quite some headaches with different infra & docker setups I imagine, but not sure if it's safe to just pass the JSON content directly.

[madppiper]

Two small things I noticed:

• Could you run npm run format so the linter checks pass? 😁
• Can you install @google-cloud/vertexai as an optional dependency like for the other endpoints?

Yea, of course. I just pushed both.

And I was wondering, would it be bad practice to put the content of the JSON credentials into an env variable secret rather than specifying the file path and reading the file? Trying to read the file could cause quite some headaches with different infra & docker setups I imagine, but not sure if it's safe to just pass the JSON content directly.

I agree and I am not a fan either. But I checked against the google docs and it was my impression that the json file was the standard way of doing it with google. Personally I would have preferred an apiToken (which funnily enough you can generate, but isn't supported by the node api of theirs):

googleapis/nodejs-vertexai#197

This is the doc in question:
https://cloud.google.com/docs/authentication/application-default-credentials#GAC

GOOGLE_APPLICATION_CREDENTIALS environment variable
You can use the GOOGLE_APPLICATION_CREDENTIALS environment variable to provide the location of a credential JSON file. This JSON file can be one of the following types of files: ...

I should mention: there are still a few things untested from my part - though connecting to Gemini-Pro works, I couldn't test it with any other model, nor could I set up a custom endpoint, as I was lacking an account with the proper privileges and or available resources. So i did the apiUrl implementation blindly.

It could also use a more thorough test of the parameters. In principle it should work, but I only tested it as far as it was required from our end.

[analyticsrepo01]

Awesome, +1 to check if merge is approved

[nsarrazin]

Thanks for the explanation! 😄 I think we can merge this now, looks good to me

[nsarrazin]

lol you got double approved apparently 😆 thanks for the contribution!

[madppiper]

lol you got double approved apparently 😆 thanks for the contribution!

Haha, I'll start using that in my resume from now on

My contributions were "double approved"