Hello, when I trained an aesthetic model using the default configuration on 8 A800 cards, I found that the training process got stuck after completing one epoch, but it worked fine when using a single A800 card. May I ask what could be the cause of this situation?

[cjt222]

Hello, when I trained an aesthetic model using the default configuration on 8 A800 cards, I found that the training process got stuck after completing one epoch, but it worked fine when using a single A800 card. May I ask what could be the cause of this situation?

[mihirp1998]

same issue after 2 epochs, the model seems to get stuck at waiting for rewards..

Please let me know if you have a soln @cjt222 or @kvablack ?

[cjt222]

Currently, there is no solution, and I am currently using a single A800 card for training.

[cjt222]

Not only the aesthetic model, but also other tasks are facing the same issue.

[desaixie]

This is caused by accelerator.save_state() at this line. This bug appeared after a recent commit . It was meant to solve a bug in diffuers as discussed in #10 . A temporary solution is to simply commenting out this line. A complete solution is to follow this or this training script, use their code that deals with LoRA to replace those in the current DDPO training script.

[kvablack]

Thanks so much @desaixie for investigating this! Strangely enough, it works on my machine.

If you have a working version of the code, would you mind opening a pull request? To be honest, I never really wrapped my head around the accelerate save/load API.

[cjt222]

After testing, it is confirmed that multi-card training is feasible when the acceleration is reduced to 0.17 @kvablack @mihirp1998 @desaixie

[cjt222]

pip install accelerate==0.17

[kvablack]

Thanks everyone, I added accelerate==0.17 to setup.py.