-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add gcu llama readme #8950
Add gcu llama readme #8950
Conversation
Thanks for your contribution! |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #8950 +/- ##
===========================================
+ Coverage 54.06% 54.14% +0.08%
===========================================
Files 650 650
Lines 103883 103883
===========================================
+ Hits 56164 56250 +86
+ Misses 47719 47633 -86 ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
llm/gcu/llama/README.md
Outdated
@@ -0,0 +1,162 @@ | |||
## 🚣♂️ 使用PaddleNLP在GCU下运行llama2-13b模型 🚣 | |||
|
|||
PaddleNLP在燧原GCU([了解燧原](https://www.enflame-tech.com/))上对llama2-13B模型进行了深度适配和优化,实现了燧原GCU推理入口和GPU的基本统一,仅需修改device即可完成推理任务的迁移。 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
可以加上一些GCU硬件的描述
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已添加S60加速卡基本情况介绍,并附官方链接
llm/gcu/llama/README.md
Outdated
### 1. 环境准备:(这将花费您10~20min时间) | ||
|
||
1. 初始化环境,安装驱动<br/> | ||
**注:您可以联系燧原工程师以获取软件驱动包和其他帮助** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
联系方式提供下?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已添加联系方式
cd PaddleNLP | ||
python -m pip install -r requirements.txt | ||
# 源码编译安装 paddlenlp | ||
python setup.py bdist_wheel && python -m pip uninstall paddlenlp -y && python -m pip install dist/paddlenlp* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
推荐一个固定版本的PaddleNLP版本会不会更好,然后再提供 git clone & python setup.py install 的方式
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
推荐使用PaddleNLP release版本v3.0.0-beta0,已更新步骤描述
llm/gcu/llama/README.md
Outdated
tar -zxf wikitext-103.tar.gz | ||
``` | ||
### 3. 推理:(这将花费您15~30min时间) | ||
下载权重文件后需要更改推理配置文件,以获取更大的性能提升。将```/root/.paddlenlp/models/__internal_testing__/sci-benchmark-llama-13b-5k/config.json```更改为下面的内容: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
默认是不会提供 model config.json的?这里提供下怎么下载 internal_testing/sci-benchmark-llama-13b-5k 的模型方式?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
首次推理将自动下载权重和配置,已更新描述
f3a7669
to
16f869c
Compare
16f869c
to
6116026
Compare
2. 拉取镜像 | ||
```bash | ||
# 注意此镜像仅为paddle开发环境,镜像中不包含预编译的飞桨安装包、TopsPlatform安装包等 | ||
docker pull registry.baidubce.com/paddlepaddle/paddle:latest-dev |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里建议限定可运行版本,latest-dev不是好的tag
``` | ||
3. 参考如下命令启动容器 | ||
```bash | ||
docker run --name paddle-gcu-test -v /home:/home --network=host --ipc=host -it --privileged registry.baidubce.com/paddlepaddle/paddle:latest-dev /bin/bash |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
此处docker启动命令也需按照对应镜像名称修改。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR types
New features
PR changes
Docs
Description
Add gcu llama2-13b readme