Flux LoRAs not learning objects

[PCanavelli]

Hi everyone,

I am finding myself stuck on what should be a fairly simple use case: I need to train a Flux LoRA on specific objects (e.g. a shirt, a bottle - one object per LoRA). I was expecting this to be straightforward, as I've got it to work dozens of times and with minimal effort using SDXL. However, I can't get a single Flux LoRA to work after about 30 tries.

Instead, it looks like the LoRA just hasn't learned anything at all. The best I can get is vague traces of the object (e.g. the pattern of the shirt vaguely shows up on one character once every 50 inferences). This completely puzzles me, as I've got Flux to learn what intuitively feels like much harder targets (e.g. artistic style, full likeness of a person) with the same method.

Is this a know issue with Flux, and has someone ever managed to to this?

What I've tried:

• Images: 20 to 60, cropped at 1024 x 1024
• Captions: just trigger word; detailed captions without trigger; detailed captions + trigger
• Ranks: from 4 to 64
• Steps: 500 to 10,000
• Learning rate: 1e-5 to 1e-3
• At inference time: loading the LoRA with strength from 0.2 to 1.0, and using guidance from 2.0 to 9.0

[bghira]

https://huggingface.co/salmonrusty/simpletuner-lora

see this example

[PCanavelli]

Amazing, thank you so much