New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

how to train for 30b or 65b on multiple GPU（4x80G） #3

Open

lishangjin opened this issue Mar 27, 2023 · 1 comment

lishangjin commented Mar 27, 2023

No description provided.

Author

lishangjin commented Mar 27, 2023 •

edited

Loading

https://zhuanlan.zhihu.com/p/616853024 提及：依赖的lora中引入的transformers支持多GPU存在问题。可以使用https://github.com/kooshi/alpaca-lora/tree/llama-parallelism llama-parallelism分支替换

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment