v0.9.8 - flux 24 gig training has entered the chat

[bghira]

Flux

It's here! Runs on 24G cards using Quanto's 8bit quantisation, or 25.7G on a Macbook system (slowly)!

If you're after accuracy, a 40G card will do Just Fine, with 80G cards being somewhat of a sweet spot for larger training efforts.

What you get:

• LoRA, full tuning (but probably just don't do that)
• Documentation to get you started fast
• Probably better for just square crop training for now - might artifact for weird resolutions
• Quantised base model unlocking the ability to safely use Adafactor, Prodigy, and other neat optimisers as a consolation prize for losing access to full bf16 training (AdamWBF16 just won't work with Quanto)

What's Changed

• trainer: simplify check by @bghira in trainer: simplify check #592
• documentation updates, apple pytorch 2.4 by @bghira in documentation updates, apple pytorch 2.4 #595
• staged storage for image embed support by @bghira in staged storage for image embed support #596
• fix: loading default image embed backend by @bghira in fix: loading default image embed backend #597
• fix: loading default image embed backend by @bghira in fix: loading default image embed backend #598
• multi-gpu console output improvements by @bghira in multi-gpu console output improvements #599
• vae cache: hash_filenames option for image sets by @bghira in vae cache: hash_filenames option for image sets #601
• multi-gpu console output reduction by @bghira in multi-gpu console output reduction #602
• fix for relative cache directories with NoneType being unsubscriptable by @bghira in fix for relative cache directories with NoneType being unsubscriptable #603
• multigpu / relative path fixes for caching by @bghira in multigpu / relative path fixes for caching #604
• backend for csv based datasets by @bghira in backend for csv based datasets #600
• CSV data backend by @bghira in CSV data backend #605
• config file versioning to allow updating defaults without breaking backwards compat by @bghira in config file versioning to allow updating defaults without breaking backwards compat #607
• config file versioning for backwards compat by @bghira in config file versioning for backwards compat #608
• experiment: small DiT model by @bghira in experiment: small DiT model #609
• merge by @bghira in merge #610
• Fix crash when using jsonl files by @swkang-rp in Fix crash when using jsonl files #611
• merge by @bghira in merge #612
• flux training by @bghira in flux training #614
• update base_dir to output_dir by @bghira in update base_dir to output_dir #615
• merge by @bghira in merge #616
• flux: validations should ignore any custom schedulers by @bghira in flux: validations should ignore any custom schedulers #618
• release: flux by @bghira in release: flux #617
• bugfix: correctly set hash_filenames to true or false for an initial dataset creation by @bghira in bugfix: correctly set hash_filenames to true or false for an initial dataset creation #620
• release: minor follow-up fixes by @bghira in release: minor follow-up fixes #628
• Flux: Fix random validation errors due to some tensors being on the cpu by @mhirki in Flux: Fix random validation errors due to some tensors being on the cpu #629
• Improve config support for transformers with accelerate by @touchwolf in Improve config support for transformers with accelerate #630
• quanto: exploring low-precision training. by @bghira in quanto: exploring low-precision training. #622
• remove all text encoders from memory correctly by @bghira in remove all text encoders from memory correctly #637

New Contributors

• @swkang-rp made their first contribution in Fix crash when using jsonl files #611
• @touchwolf made their first contribution in Improve config support for transformers with accelerate #630

Full Changelog: v0.9.7.8...v0.9.8

---

This discussion was created from the release v0.9.8 - flux 24 gig training has entered the chat.

[billnye2]

Nice, is it possible to cache the quantized model on disk for future runs when testing

[bghira]

yes but it was buggy

[nom]

Awesome work! Curious how well LORA on the "Dev" model works, any results already worth sharing?

(Btw the documentation link doesn't work)

[pzc163]

its really a good news！！ So the question is, can you support single-computer, multi-GPU training?？

[riffmaster-2001]

running the training now on a 6000 ada, whats the best way to run the lora after training completes with Flux?

[sdbds]

Any discord for this tools?
I had implemented 15G with batchsize1 rank4 flux dev fp8 lora training , but had met other problem.
(mainly for FP8)