How to host Gemini AI on a local environment

This is a super fast tutorial on how to host Gemini AI on a local environment. This guide is mostly for users from EU who don't have (yet) access to the Gemini API.

Although the tutorials from Google are far more comprehensive, for a newbie they can be quite overwhelming so this contains only the bare minimum to experiment with the Gemini models.

Even if the links provided will perhaps be useful, for more elaborate projects I advise you to follow another guide.

Warning

These steps were tested on Ubuntu 22.04 LTS and Linux Mint 21.3 "Virginia". If you have a different OS (not Debian-based) or a different version you can still follow the tutorial, however you will have to check the links/ search for alternatives. The steps remain the same.

I - Vertex AI

Using Google Cloud Computing Service connect to the Free Trial, then Try Gemini 1.5 Pro. You will be redirected to Vertex AI.
Create a new project (remember the name) and experiment with the models and settings.

II - Virtualenv

Clone this repo and open the terminal.

pip install virtualenv
virtualenv <your-env>
source <your-env>/bin/activate
python3 -m pip install -r requirements.txt

Note

If you encounter Command 'virtualenv' not found, use sudo apt install python3-virtualenv instead.

III - Set up Application Default Credentials

Install the gcloud CLI.

For a faster setup use the script.

Note

If you do not have the same OS or you have a different version of it, please follow the guide from the link provided adjusted to your case!

chmod +x script.sh
./script.sh

You will be redirected to a login screen, then asked to choose the project (see Step I). Follow the steps accordingly.

If succesful, you will be redirected to this page.

Now create your credential file using gcloud auth application-default login.

If succesful, you should see the same page again.

IV - Google Cloud console

For enabling PDF prompts and analysis, the object must be uploaded in a Bucket. Follow the steps accordingly.

V - Have fun!

Modify main.py to your liking and enjoy the newest Gemini model!

Don't forget python3 main.py to run it.

Tip

If you have any suggestions or you want to add explanations/ commands or you faced some errors I did not, you are free to contribute!

This is a quick guide I made in the middle of the night after some hours spent figuring out how I can use Gemini locally.