continue the interrupted training

[quzopl]

is it possible to resume the interrupted training ? before a few updates ago(Simpletuner) , it was possible to continue the training (flux) , since a while , when trying to resume , evrytime a PEFT error pops up.

[bghira]

yes, it is possible

[quzopl]

@bghira but how ? before it worked without any problem, just change the steps in config and start train again, now every time there is the same PEFT error after loading checkpoint

[bghira]

i will bust out the crystal ball and conjure up an image of the error :) but you have to do some work and tell us what is happening. no one can just solve it for you

[quzopl]

@bghira here is my deploy script

#!/bin/bash

sudo apt -y install python3.11 python3.11-venv mc
git clone https://github.com/bghira/SimpleTuner.git
cd SimpleTuner
python3.11 -m venv .venv
source .venv/bin/activate
pip install -U poetry pip
poetry install

here is config:

here is error

evry time is new fresh linux system , before last updates work without any problems

[bghira]

alright, i think it's fixed on the main branch. it works again here