请教一下，我该如何使用2D Tensor parallelism 来训练resnet模型？ #4376
Unanswered
alphabewitch
asked this question in
Community | Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
我想用框架的2d张量并行跑resnet，原本pytorch写的resnet模型里的nn库的卷积这种要替换成colossal库的吗，文档里说只会对colossalai.nn的层应用张量并行，所以我不知道要替换哪些。
源码里只找到Linear/LayerNorm/Embedding/Dropout/CrossEntropyLoss这些，conv2d卷积这种现在框架能跑张量并行吗？
Beta Was this translation helpful? Give feedback.
All reactions