Which training configuration did you use?
Congratulations on the model! I'd like to know what training configuration you used. I'm not familiar with training editing models for Krea 2. The times I've tried, there were many artifacts in the validation results and it seemed to take too long. I'd like to know if there are any important details. Thank you in advance.
Thanks! The short version of why standard attempts give artifacts is that this is not a normal text-to-image LoRA, it uses dual conditioning: the source image goes in both as VAE reference tokens placed in-context and as part of an image-grounded instruction encoding through the vision-language text encoder. Train it like a plain t2i LoRA without that and the model never learns to look at the source, which is where the artifacts come from. Settings are rank 256, 1024 resolution, constant LR 1e-4, bf16, flow matching, but the setup matters more than the numbers. I have a training patch for ostris ai-toolkit that implements exactly this recipe and I plan to release it publicly soon, I will ping this thread when it is up.
Thanks! The short version of why standard attempts give artifacts is that this is not a normal text-to-image LoRA, it uses dual conditioning: the source image goes in both as VAE reference tokens placed in-context and as part of an image-grounded instruction encoding through the vision-language text encoder. Train it like a plain t2i LoRA without that and the model never learns to look at the source, which is where the artifacts come from. Settings are rank 256, 1024 resolution, constant LR 1e-4, bf16, flow matching, but the setup matters more than the numbers. I have a training patch for ostris ai-toolkit that implements exactly this recipe and I plan to release it publicly soon, I will ping this thread when it is up.
Okay, so what you're telling me is that I have to train in simple T2I mode, but using control and target on the dataset? Without using any adapter, or do I need to use an adapter? I've trained many editing models before, but the big issue is that this specific one didn't work the same way as the others; too many artifacts and it never converged.
Okay, so what you're telling me is that I have to train in simple T2I mode, but using control and target on the dataset? Without using any adapter, or do I need to use an adapter? I've trained many editing models before, but the big issue is that this specific one didn't work the same way as the others; too many artifacts and it never converged.
His answer is not anything that wasn't already stated in his model card. What matters is the custom code he used and if its something Ostris can implement - or Kohya in Musubi-Tuner - that would be amazing, because clearly his code just works pretty well. He didn't say to train it in T2I - read again - its the opposite. Its training in 'Edit mode' but in a different way than AI-Toolkit currently does.
In my humble opinion, AI-Toolkit Krea2 Edit training code is simply flawed. I also trained on many Edit models (Kontext, Qwen-Image-Edit, Klein) also Anima with Cosmos Ref, but training Krea2 Edit on AI-Toolkit gave me the worse results so far. EDIT: I tried both KV cache ON and OFF - with it ON the inference speed is increased by at least 50% but regardless I was not able to train properly in both cases
Okay, so what you're telling me is that I have to train in simple T2I mode, but using control and target on the dataset? Without using any adapter, or do I need to use an adapter? I've trained many editing models before, but the big issue is that this specific one didn't work the same way as the others; too many artifacts and it never converged.
His answer is not anything that wasn't already stated in his model card. What matters is the custom code he used and if its something Ostris can implement - or Kohya in Musubi-Tuner - that would be amazing, because clearly his code just works pretty well. He didn't say to train it in T2I - read again - its the opposite. Its training in 'Edit mode' but in a different way than AI-Toolkit currently does.
In my humble opinion, AI-Toolkit Krea2 Edit training code is simply flawed. I also trained on many Edit models (Kontext, Qwen-Image-Edit, Klein) also Anima with Cosmos Ref, but training Krea2 Edit on AI-Toolkit gave me the worse results so far. EDIT: I tried both KV cache ON and OFF - with it ON the inference speed is increased by at least 50% but regardless I was not able to train properly in both cases
Yes, thanks, I understand, I will create my own custom code. I just haven't done it yet because I'm training models for LTX based on custom training code, applying new training modalities, and I haven't had time to do that for Flux Krea 2. I have many head-swap datasets and I intend to train a new version of my BFS with Flux Krea 2.
Good news — the full training code is now public: https://github.com/lbouaraba/krea2edit-trainer
It's the exact ai-toolkit extension these LoRAs were trained with, geometry-matched to the comfyui-krea2edit nodes. The artifacts you hit are most likely the reference-geometry details — docs/GEOMETRY.md pins the full contract (fit protocol, fractional RoPE offsets, grounding template, timestep weighting). :)