Skip to content
This repository has been archived by the owner on Oct 25, 2024. It is now read-only.

[NeuralChat] Support Mixtral-8x7B-v0.1 model #972

Merged
merged 7 commits into from
Dec 25, 2023
Merged

[NeuralChat] Support Mixtral-8x7B-v0.1 model #972

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
bf20c91
Support Mixstral-8x7b model
lvliang-intel Dec 20, 2023
b8b9799
Merge branch 'main' of https://github.com/intel/intel-extension-for-t…
lvliang-intel Dec 20, 2023
0c4996a
update readme
lvliang-intel Dec 21, 2023
03d5618
Merge branch 'main' of https://github.com/intel/intel-extension-for-t…
lvliang-intel Dec 21, 2023
99baec6
Merge branch 'main' of https://github.com/intel/intel-extension-for-t…
lvliang-intel Dec 22, 2023
3691930
fix pylint
lvliang-intel Dec 22, 2023
85ee426
Merge branch 'main' of https://github.com/intel/intel-extension-for-t…
lvliang-intel Dec 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions intel_extension_for_transformers/neural_chat/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@ The table below displays the validated model list in NeuralChat for both inferen
|LLaMA2 series| ✅| ✅|✅| ✅ |
|MPT series| ✅| ✅|✅| ✅ |
|Mistral| ✅| ✅|✅| ✅ |
|Mixtral-8x7b-v0.1| ✅| ✅|✅| ✅ |
|ChatGLM series| ✅| ✅|✅| ✅ |
|Qwen series| ✅| ✅|✅| ✅ |
|StarCoder series| | | | ✅ |
Expand Down
1 change: 1 addition & 0 deletions intel_extension_for_transformers/neural_chat/chatbot.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ def build_chatbot(config: PipelineConfig=None):
"flan-t5" in config.model_name_or_path.lower() or \
"bloom" in config.model_name_or_path.lower() or \
"starcoder" in config.model_name_or_path.lower() or \
"mixtral" in config.model_name_or_path.lower() or \
"codegen" in config.model_name_or_path.lower():
from .models.base_model import BaseModel
adapter = BaseModel()
Expand Down
3 changes: 2 additions & 1 deletion intel_extension_for_transformers/neural_chat/models/model_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -489,6 +489,7 @@ def load_model(
or re.search("starcoder", model_name, re.IGNORECASE)
or re.search("codellama", model_name, re.IGNORECASE)
or re.search("mistral", model_name, re.IGNORECASE)
or re.search("mixtral", model_name, re.IGNORECASE)
or re.search("codegen", model_name, re.IGNORECASE)
) and not ipex_int8) or re.search("opt", model_name, re.IGNORECASE):
with smart_context_manager(use_deepspeed=use_deepspeed):
Expand Down Expand Up @@ -545,7 +546,7 @@ def load_model(
)
else:
raise ValueError(f"unsupported model name or path {model_name}, \
only supports FLAN-T5/LLAMA/MPT/GPT/BLOOM/OPT/QWEN/NEURAL-CHAT/MISTRAL/CODELLAMA/STARCODER/CODEGEN now.")
only supports t5/llama/mpt/gptj/bloom/opt/qwen/mistral/mixtral/gpt_bigcode model type now.")
except EnvironmentError as e:
if "not a local folder and is not a valid model identifier" in str(e):
raise ValueError("load_model: model name or path is not found")
Expand Down
Loading