Regression: Image generation is broken in 1.113.2

[Leo7k]

In version 1.113.2, image generation is broken (at least, with FLUX.1 models), and instead of the requested image, it produces random noise, stripes, and the like. Sometimes the application crashes with an access violation error, but I haven't been able to reproduce this reliably yet. This was tested both with Vulkan and on CPU without using the graphics card for generation — the result is the same.

In version 1.112.2, there are no such issues; image generation works perfectly.

Hardware: AMD Ryzen 9 3900X, RTX 2080 Ti
OS: Windows 7, KoboldCpp runs with the latest version of VxKex NEXT.

[LostRuins]

@Leo7k I just tried your example on Flux 1, and while the result isn't great (due to your settings), it works

Guidance=21.5 is far too high, Lowering it to 5, it works fine. 256x256 is also rather small

No crashes or incoherence for me.

1. Does this happen on all attempts or just once in a while? What about other models?
2. Could you share your model file url, and your launcher settings?

@wbruna do you have any recent issues on your AMD?

---

@wbruna also on an unrelated note, i noticed we had a regression where CFG scale returned value was wrong. It seems that the logic for determining guidance.img_cfg or guidance.txt_cfg is non-trivial. So for now I swapped it to jsoninfo["cfg_scale"] = sd_params->cfg_scale; but i'm not sure if that's the best option, feel free to modify if you have a better approach cae0375

[wbruna]

Guidance doesn't matter for Flux.1, because we (still) reset it to 1 if it is higher.

Working fine on my end; this is Schnell with 4 steps:

But the random crashes suggest something hardware related: e.g. if we are taking more memory now, it could be getting closer to overflowing VRAM with the same settings.

@wbruna also on an unrelated note, i noticed we had a regression where CFG scale returned value was wrong. It seems that the logic for determining guidance.img_cfg or guidance.txt_cfg is non-trivial. So for now I swapped it to jsoninfo["cfg_scale"] = sd_params->cfg_scale; but i'm not sure if that's the best option, feel free to modify if you have a better approach cae0375

That'd be fine if we didn't mess with CFG, but for certain models (like Flux.1) we do: see the "Flux: clamping CFG Scale to 1" part. The returned info is supposed to get all values from the final params struct because those were sent to the engine: it is not an echo of the received parameters (that'd be the parameter field, which we currently do not set).

[wbruna]

BTW, Schnell also makes bad results with too many steps. This is 256x256, CFG 1, 4 versus 21 steps:

.

So it depends a lot on the specific Flux model type.

[LostRuins]

I see. Then I guess I should reverse my change? At first I thought it was broken cause I was reloading old image gens by clicking txt2img into other models and the loaded cfg was 1 which broke the image

[wbruna]

I see. Then I guess I should reverse my change?

Yeah, the params.sample_params.guidance.txt_cfg field is exactly where it is supposed to be sent, and read back.

At first I thought it was broken cause I was reloading old image gens by clicking txt2img into other models and the loaded cfg was 1 which broke the image

If it was on Schnell, my guess would be that the step count broke it instead. Maybe the model is able to deal with higher step counts in some cases, and that's masking the issue?

[Leo7k]

It seems that guidance and steps count doesn't matter.
Guidance=5, Steps=6:

But on 1.112.2 it works fine.

[Leo7k]

256x256 is also rather small

It's intentional. I suspected it was caused by VRAM overflow. Because on 1.112.2 with resolution 384x384 or more, I got the following:

[LostRuins]

Could you try enable CPU offload --sdoffloadcpu

[Leo7k]

It seems that it doesn't change anything; the result is the same.

[wbruna]

It seems that it doesn't change anything; the result is the same.

That's very odd: it should at least greatly reduce VRAM usage. Could you send a full --debug log, with model loading + a single generation?

[Leo7k]

Where can I find KoboldCPP's logs?

[LostRuins]

It's in the terminal window. You sent a screenshot of it earlier.

[Leo7k]

[Leo7k]

I also tried generating an image with the same parameters (using Vulkan and CPU Offload) on stable-diffusion-cpp, even at a higher resolution (1024 x 1024) - the image was generated correctly.

[wbruna]

Unfortunately, the model initialization (before "Please connect...") is missing from the screenshots. It would help us to know if all expected options are being correctly applied, the size of the model, etc. One way to get it without the need for multiple screenshots could be redirecting the output; something like koboldcpp.exe > logfile.txt.