Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: update ollama models #105

Merged
merged 1 commit into from
Dec 16, 2024
Merged

refactor: update ollama models #105

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
[![dev.to blog post walkthrough](https://img.shields.io/badge/Blog%20post-black?style=flat-square&logo=dev.to)](https://dev.to/azure/build-a-serverless-chatgpt-with-rag-using-langchainjs-3487)
[![Build Status](https://img.shields.io/github/actions/workflow/status/Azure-Samples/serverless-chat-langchainjs/build-test.yaml?style=flat-square&label=Build)](https://github.com/Azure-Samples/serverless-chat-langchainjs/actions)
![Node version](https://img.shields.io/badge/Node.js->=20-3c873a?style=flat-square)
[![Ollama + Mistral](https://img.shields.io/badge/Ollama-Mistral-ff7000?style=flat-square)](https://ollama.com/library/mistral)
[![Ollama + Llama3.1](https://img.shields.io/badge/Ollama-Llama3.1-ff7000?style=flat-square)](https://ollama.com/library/llama3.1)
[![TypeScript](https://img.shields.io/badge/TypeScript-blue?style=flat-square&logo=typescript&logoColor=white)](https://www.typescriptlang.org)
[![License](https://img.shields.io/badge/License-MIT-yellow?style=flat-square)](LICENSE)

Expand Down Expand Up @@ -154,12 +154,12 @@ The resource group and all the resources will be deleted.
If you have a machine with enough resources, you can run this sample entirely locally without using any cloud resources. To do that, you first have to install [Ollama](https://ollama.com) and then run the following commands to download the models on your machine:

```bash
ollama pull mistral:v0.2
ollama pull all-minilm:l6-v2
ollama pull llama3.1:latest
ollama pull nomic-embed-text:latest
```

> [!NOTE]
> The `mistral` model with download a few gigabytes of data, so it can take some time depending on your internet connection.
> The `llama3.1` model with download a few gigabytes of data, so it can take some time depending on your internet connection.

After that you have to install the NPM dependencies:

Expand Down
4 changes: 2 additions & 2 deletions docs/faq.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,8 @@ You may also need to adjust the capacity in `infra/main.bicep` file, depending o
To change the local models used by Ollama, you can edit the file `packages/api/src/constants.ts`:

```typescript
export const ollamaEmbeddingsModel = 'all-minilm:l6-v2';
export const ollamaChatModel = 'mistral:v0.2';
export const ollamaEmbeddingsModel = 'nomic-embed-text:latest';
export const ollamaChatModel = 'llama3.1:latest';
```

You can see the complete list of available models at https://ollama.ai/models.
Expand Down
26 changes: 13 additions & 13 deletions docs/tutorial/04-preparing-understanding-language-models.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Preparing and Understanding Language Models: Configuring Azure OpenAI Service and Installing Ollama with Mistral 7B
# Preparing and Understanding Language Models: Configuring Azure OpenAI Service and Installing Ollama with Llama3.1 8B

This section we will cover the language models used in the project. Throughout the tutorial, we will also learn how to generate the environment variables needed to use the Azure Services, including the **[Azure OpenAI Service](https://learn.microsoft.com/azure/ai-services/openai/overview)**.

We will also teach you how to use **[Ollama](https://ollama.com/)** with **[Mistral 7B](https://mistral.ai/)**, an Open Source Language Model, if you want to use it locally.
We will also teach you how to use **[Ollama](https://ollama.com/)** with **[Llama3.1 8B](https://www.llama.com/)**, an Open Source Language Model, if you want to use it locally.

## Models to be used in the project

We will teach you how to use two different language models: GPT-3.5 Turbo integrated with _Azure OpenAI Service_ (on Azure) and _Ollama with Mistral 7B_ (if you decide to use a model locally). Let's take a look at each of them.
We will teach you how to use two different language models: GPT-3.5 Turbo integrated with _Azure OpenAI Service_ (on Azure) and _Ollama with Llama3.1 8B_ (if you decide to use a model locally). Let's take a look at each of them.

### GPT-3.5 Turbo Integrated with Azure OpenAI Service

Expand All @@ -18,17 +18,17 @@ You have the choice to use either **[OpenAI Service](https://openai.com/)** or *

Azure OpenAI Service provides REST API access in many programming languages, including Python, Node.js, and C#. Additionally, it offers advanced language models like GPT-4, GPT-4-Turbo with Vision, which are versatile and adaptable to various tasks such as content generation, summarization, image recognition, semantic search, and text-to-code translation.

### Ollama with Mistral 7B
### Ollama with Llama3.1 8B

![Ollama Page](./images/ollama-page.png)

**[Ollama](https://ollama.com/)** presents itself as an open-source solution, offering a transparent and modifiable platform. The Mistral 7B model has 7 billion parameters and is designed to be effective, efficient in terms of cost and scability.
**[Ollama](https://ollama.com/)** presents itself as an open-source solution, offering a transparent and modifiable platform. The Llama3.1 8B has 8 billion parameters and is designed to be effective, efficient in terms of cost and scability.

Ollama's openness encourages innovation and collaboration within the developer community. Users can adapt the model to their specific needs, experiment with innovative ideas, or integrate the model in ways that proprietary services might not allow.

Additionally, using an open-source language model can decrease expenses, which is a crucial factor for projects with restricted budgets or for those who only wish to experiment with language models.

![Mistral 7B Page](./images/mistral-7b-page.png)
![Llama3.1 8B Page](./images/mistral-7b-page.png)

## Creating Azure resources

Expand Down Expand Up @@ -83,31 +83,31 @@ Before installing Ollama, please ensure you meet the prerequisites, which includ
To begin, download the necessary models for this project by running the following commands in your terminal:

```bash
ollama pull mistral:v0.2
ollama pull all-minilm:l6-v2
ollama pull llama3.1:latest
ollama pull nomic-embed-text:latest
```

We will use the Mistral 7B model, a powerful language model, and the All-MiniLM model, a small embedding model, to generate vectors from the text for the chatbot.
We will use the Llama3.1 8B model, a powerful language model, and the All-MiniLM model, a small embedding model, to generate vectors from the text for the chatbot.

> **Note:** The Mistral model will download several gigabytes of data, so the process may take some time depending on your internet connection.
> **Note:** The Llama3.1 model will download several gigabytes of data, so the process may take some time depending on your internet connection.

After downloading the models, you can verify the proper functioning of the Ollama server by executing the following command:

```bash
ollama run mistral:v0.2
ollama run llama3.1:latest
```

An invite will be displayed in your terminal, allowing you to directly communicate with the AI model in a chat-like format.

![Ollama Mistral](./images/ollama-mistra-cli.png)
![Ollama Llama3.1 8B](./images/ollama-mistra-cli.png)

Ask the model some questions and watch the answers. This will provide insight into the model's capabilities and how to interact with it.

After to finish to test the Ollama server, you can stop it by pressing **Ctrl+D** in your terminal.

## Next Steps

This tutorial covers language models that will be used in the project. Choose the best model to suit your needs. To use the Azure OpenAI Service, follow the instructions to set up the service in Azure. To use Ollama with Mistral 7B, follow the instructions to install Ollama and the local models.
This tutorial covers language models that will be used in the project. Choose the best model to suit your needs. To use the Azure OpenAI Service, follow the instructions to set up the service in Azure. To use Ollama with Llama3.1 8B, follow the instructions to install Ollama and the local models.

To begin developing the application, we first need to create some configuration files for the project. We'll cover this in the next section!

Expand Down
4 changes: 2 additions & 2 deletions packages/api/src/constants.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Ollama models configuration
// You can see the complete list of available models at https://ollama.ai/models
export const ollamaEmbeddingsModel = 'all-minilm:l6-v2';
export const ollamaChatModel = 'mistral:v0.2';
export const ollamaEmbeddingsModel = 'nomic-embed-text:latest';
export const ollamaChatModel = 'llama3.1:latest';

// Faiss local store folder
export const faissStoreFolder = '.faiss';
Loading