CUDA_VISIBLE_DEVICES=4,5,6,7 python Transformer_AutoShard_Test.py
50%+ speed up relative to FSDP
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python Transformer_AutoShard_Test.py
50%+ speed up relative to FSDP
-
Notifications
You must be signed in to change notification settings - Fork 0
Youhe-Jiang/AutoSharded_Transformer_Based_on_PyTorch
About
No description, website, or topics provided.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published