Ollama Cloud

Ollama Cloud is a Highly Scalable Cloud-native Stack for Ollama. Help us shine by turning a ★ into a ⭐!

This is not a Code repository, please read Components

Development usually kicks off on your local machine, comfy and controlled.

But moving to production? That’s a huge leap — hello, delay, inconsistency, and dependence.

Ollama Cloud steps in with some solid perks:

• Consistency: If it works locally, it'll work at scale with no change to your code.
• Simplicity: Can be deployed at scale with no DevOps or SRE skills.
• A Cloud Platform: Empowers developers with self-service capabilities.

What you see locally is what you get in production. Basically, "Local Is Production". Maybe we should call it LIP 🫦, lol.

Notice the use of host calls over REST APIs, which significantly speeds up communication, enhances reliability, and ensures privacy.

Now you can focus on building amazing AI applications without worrying about the usual steps involved in taking them to production, such as DevOps or service dependencies. You can relax, as now it's a mere git push away!

Components

Ollama Cloud is based on the following components:

• tau: The implementation of taubyte, a solution to build autonomous cloud computing platforms.
• ollama plugin: A plugin that allows calling ollama from WebAssembly.
• dreamland: A tool that allows running a taubyte based cloud on your computer for local development and E2E automated testing.

Set Up Your Local Environment

First, prepare your local environment with dreamland, tau-cli, docker, and the necessary tools to build the ollama plugin.

Dreamland & Tau-CLI

If you have Node.js, just run:

npm i @taubyte/dream @taubyte/cli

If not, check dreamland and tau-cli for other installation methods.

Docker

To install Docker simply use:

curl -fsSL https://get.docker.com | sh

Build Tools

Building the plugin requires go, but ollama itself needs:

• cmake version 3.24 or higher
• go version 1.22 or higher
• gcc version 11.4.0 or higher

On Ubuntu Linux, install gcc & cmake with:

sudo apt install build-essential

For Go, follow the steps at Go's official installation guide.

On macOS, you can install these using:

brew install go cmake gcc

Hardware Acceleration

If you're unsure whether you have the necessary development libraries for your acceleration hardware, please refer to https://github.com/ollama/ollama/blob/main/docs/development.md.

Build the Plugin

To build the plugin, first clone the repository:

git clone https://github.com/samyfodil/ollama.git

Then, compile the C++ backend:

cd ollama
go generate ./...

Finally, build the plugin itself:

cd tau
go build . -o plugin

That's it! You should now have a plugin executable.

Start a Local Cloud

With the plugin built, it's time to start a local cloud. Run:

dream new multiverse

Wait until it indicates that the universe is ready.

Attach Plugin

The Cloud you've just started lacks LLM capabilities. To add these, load the plugin by running:

dream inject attach-plugin -p /path/to/plugin

If you haven't changed directories, it should be:

dream inject attach-plugin -p $(pwd)/plugin

Now, you're all set to kick ass!

What's next

• Create your first application

Acknowledgement

This project stands on the shoulders of the projects mentioned in the introduction, as well as ollama itself. Please show your support by leaving a ⭐ if you find them useful.