feat: add llm-flow

gist-rs · Oct 25, 2023 · 6a5af30 · 6a5af30
1 parent b00d267
commit 6a5af30
Show file tree

Hide file tree

Showing 3 changed files with 30 additions and 18 deletions.
diff --git a/src/ml/assets/llm-flow.jpeg b/src/ml/assets/llm-flow.jpeg
diff --git a/src/ml/large-language-model.md b/src/ml/large-language-model.md
@@ -2,20 +2,24 @@
 
 FYI: We will focus this topic in `Rust` landscape which usually follow Python and C++ developments.
 
+![](./assets/llm-flow.jpeg)
+
+> ref: https://twitter.com/_aigeek/status/1717046220714308026/photo/1
+
 ## TL;DR
 
-- ⚠️ Below info is outdates, The clear `2023/10` winner is [Mistral-7B](https://huggingface.co/mistralai)
+- `2023/10` model winner is [Mistral-7B](https://huggingface.co/mistralai) and [Zephyr 7B Alpha](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha).
 - For `Rust` consider use [Candle](https://github.com/huggingface/candle)
 - For coding related see: [Replit Code V-1.5 3B](https://huggingface.co/replit) and newcomer [Tabby](https://tabby.tabbyml.com/)
-- The format is `GGUF`.
-
-| Feature                    | GGUF                                              | GGML                                 |
-| -------------------------- | ------------------------------------------------- | ------------------------------------ |
-| Extensibility              | Yes                                               | No                                   |
-| Future-proofing            | Yes                                               | No                                   |
-| Performance                | Better, especially for models with special tokens | Good                                 |
-| Support for various models | Yes                                               | Yes, but limited to llama.cpp models |
-| Status                     | Current standard format                           | Deprecated                           |
+- The winner format is `GGUF`.
+  | Feature | GGUF | GGML |
+  | -------------------------- | ------------------------------------------------- | ------------------------------------ |
+  | Extensibility | Yes | No |
+  | Future-proofing | Yes | No |
+  | Performance | Better, especially for models with special tokens | Good |
+  | Support for various models | Yes | Yes, but limited to llama.cpp models |
+  | Status | Current standard format | Deprecated |
+- ⚠️ Below info is outdates.
 
 ## llm (aka LLaMA-rs)
 

diff --git a/src/ml/llm-train-finetune/mod.md b/src/ml/llm-train-finetune/mod.md
@@ -10,7 +10,7 @@
 
 ### A Beginner’s Guide to Fine-Tuning Mistral 7B Instruct Model
 
-> Fixed notebook is [Mistral_7B_qLora_Finetuning.ipynb](./Mistral_7B_qLora_Finetuning.ipynb). But prompt formatting is still questioned.
+> Fixed notebook is [Mistral_7B_qLora_Finetuning.ipynb](./Mistral_7B_qLora_Finetuning.ipynb). But prompt formatting is still in doubt.
 
 - Colab: [https://adithyask.medium.com/a-beginners-guide-to-fine-tuning-mistral-7b-instruct-model-0f39647b20fe]()
 - Source: [https://github.com/adithya-s-k/CompanionLLM]()
@@ -48,20 +48,28 @@
 
 ## TODO
 
-- [ ] Understand `pgvector`: https://bugbytes.io/posts/vector-databases-pgvector-and-langchain/
-- [ ] Try RAG with `pgvector` for faster query: https://bugbytes.io/posts/retrieval-augmented-generation-with-langchain-and-pgvector/
-- [ ] Use `MistralLite` for longer context retrieval: https://huggingface.co/amazon/MistralLite
-- [ ] Create POC PDF pipeline service.
+- [x] Understand `pgvector`: https://bugbytes.io/posts/vector-databases-pgvector-and-langchain/
+- [ ] Try `RAG` with `pgvector` for faster query: https://bugbytes.io/posts/retrieval-augmented-generation-with-langchain-and-pgvector/
+
+## TOTRY - `TimeGPT`
+
+- [ ] Read `TimeGPT`: https://docs.nixtla.io/docs
+- [ ] Try `TimeGPT` with [LlamaIndex's Supabase Vector Store](https://colab.research.google.com/github/supabase/supabase/blob/master/examples/ai/llamaindex/llamaindex.ipynb).
+
+## TOTRY - `ChatWithPDF`
+
+- [ ] `ChatWithPDF`: Use `MistralLite` for longer context retrieval: https://huggingface.co/amazon/MistralLite
+  - [ ] try [Using MistralLite with HuggingFace Transformers and FlashAttention-2](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/blob/main/MistralLite/huggingface-transformers/example_usage.ipynb)
+- [ ] `ChatWithPDF`: Try: https://github.com/aigeek0x0/zephyr-7b-alpha-langchain-chatbot
+- [ ] `ChatWithPDF`: Take a look: https://github.com/chatchat-space/Langchain-Chatchat
 
 ## TOREAD
 
-- [ ] TimeGPT: https://docs.nixtla.io/docs
 - [ ] Read, try and summary: https://github.com/replit/replitLM#alpaca-style-instruct-tuning-with-hugging-face-transformers
 - [ ] Try fine-tune-mistral: https://github.com/abacaj/fine-tune-mistral
 - [ ] Try fine-tune-mistral: https://github.com/brevdev/notebooks/blob/main/mistral-finetune-own-data.ipynb
-- [ ] Try LocalGPT: https://github.com/PromtEngineer/localGPT
+- [ ] Try `LocalGPT`: https://github.com/PromtEngineer/localGPT
 - [ ] Read and summary: https://twitter.com/manelferreira_/status/1711788177458090388
 - [ ] Read and summary: https://docs.llamaindex.ai/en/stable/core_modules/data_modules/index/metadata_extraction.html
 - [ ] LLaVA: Visual and language AI with GPT-4, instruction tuning to visual data.
 - [ ] MUFFIN: follow Scaling Tasks per Input paradigm to collect multiple task instructions for a single input. (Keywords: Instruction Brainstorm, Instruction Rematching)
-- [ ] Take a look: https://github.com/chatchat-space/Langchain-Chatchat