Merge remote-tracking branch 'refs/remotes/origin/main'

gist-rs · Jul 24, 2024 · 129f1ae · 129f1ae
2 parents b2712bd + 3cfb429
commit 129f1ae
Show file tree

Hide file tree

Showing 4 changed files with 22 additions and 2 deletions.
diff --git a/src/SUMMARY.md b/src/SUMMARY.md
@@ -67,6 +67,7 @@
 - [ML](ml/mod.md)
   - [Large Language Model](ml/large-language-model.md)
   - [Inferences](ml/infer/mod.md)
+    - [ollama](ml/infer/ollama.md)
     - [TabbyML](ml/infer/tabbyml.md)
     - [WasmEdge](ml/infer/wasmedge.md)
     - [Hugging Face - Text Generation Inference](ml/infer/huggingface-text-generation-inference.md)

diff --git a/src/ml/infer/llama.cpp.md b/src/ml/infer/llama.cpp.md
diff --git a/src/ml/infer/mod.md b/src/ml/infer/mod.md
@@ -5,4 +5,4 @@
 - [x] [WasmEdge](./wasmedge.md) // MacOS/Windows/WSL2
 - [x] [TabbyML](./tabbyml.md) // MacOS/Windows/WSL2
 - [x] [llama.cpp](https://github.com/ggerganov/llama.cpp) // MacOS/Windows // 13GB // 34.01 tokens/sec
-- [x] [ollama](https://ollama.ai/) // MacOS
+- [x] [ollama](./ollama.md) // MacOS/Windows/WSL2
diff --git a/src/ml/infer/ollama.md b/src/ml/infer/ollama.md
@@ -0,0 +1,20 @@
+# ollama
+
+## Setup
+
+### Windows
+
+Follow the step at [ollama](https://ollama.ai/)
+
+```
+ollama run llama3.1
+```
+
+## Serve (optional via 13434 PORT)
+
+### Windows
+
+```
+SET OLLAMA_HOST=0.0.0.0:13434
+ollama serve
+```