Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]Unable to continue last training session #504

Closed
tao1261060556 opened this issue Aug 29, 2024 · 6 comments
Closed

[BUG]Unable to continue last training session #504

tao1261060556 opened this issue Aug 29, 2024 · 6 comments
Assignees
@AnyaCoder
Labels
bug Something isn't working

Comments

@tao1261060556
Copy link

使用1.2版本，同一个训练集，用webui训练一段时间后，停止训练.重启电脑后，重新打开WebUI,选择好 LLAMA 检查点后，再点训练，就出错了。
不知别人有没有遇到相同的情况。1.1.2版本则不会出错，可以继续接下去训练。
@tao1261060556 tao1261060556 added the bug Something isn't working label Aug 29, 2024
@AnyaCoder
Copy link
Collaborator

1.2默认使用lora，未支持继续训练

@tao1261060556
Copy link
Author

1.2默认使用lora，未支持继续训练

原来如此，我试着把lora的勾去掉，但去不掉。没办法不用lora

@AnyaCoder
Copy link
Collaborator

更新一下代码就可以取消勾选了

@tao1261060556
Copy link
Author

更新一下代码就可以取消勾选了

更新后可以不勾选了，但练训出来的模型，用webui没有加载到，无法推理

@AnyaCoder
Copy link
Collaborator

参见 tools\extract_model.py 提取出 state_dict ，

python tools\extract_model.py results\..\step_0000{xxx}.ckpt model.pth

然后再复制一份 checkpoints/fish-speech-1.2-sft ， 粘贴该文件夹并重命名为 checkpoints/fs-1.2-new (随意取名) , 将提取出的model.pth覆盖进去。最后在webui里选中即可。

@Stardust-minus Stardust-minus changed the title [BUG]无法继续上一次的训练 [BUG]Unable to continue last training session Sep 8, 2024
@Stardust-minus
Copy link
Member

#515 Change title to English.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AnyaCoder AnyaCoder

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tao1261060556 @Stardust-minus @AnyaCoder