Blurred & noisy images when used with stable-diffusion-2 and stable-diffusion-2-1

[jan-thoma]

``When i try to build the container from the Dockerfile. Passing either stabilityai/stable-diffusion-2-1 or `stabilityai/stable-diffusion-2-1. Im getting very noisy and blurry images with the parameters adapted from the example:

{
  "modelInputs": {
    "prompt": "Super dog",
    "num_inference_steps": 10,
    "guidance_scale": 7.5,
    "width": 1024,
    "height": 1024,
    "seed": 3239022079
  },
  "callInputs": {
    "MODEL_ID": "stabilityai/stable-diffusion-2-1",
    "PIPELINE": "StableDiffusionPipeline",
    "SCHEDULER": "LMSDiscreteScheduler",
    "safety_checker": true
  }
}

The output looks like this:

Am i missing something important here or this a bug?

[gadicc]

Hey, I think that looks about right for 10 steps for LMS sampler. You usually need about 50 steps for a high quality image. With the DPMSolverMultistepScheduler you can get good results in just 20 steps, and even with the 10 you specified it should look ok.

…

On Wed, 18 Jan 2023, 21:09 Jan Thoma, ***@***.***> wrote: ``When i try to build the container from the Dockerfile. Passing either stabilityai/stable-diffusion-2-1 or `stabilityai/stable-diffusion-2-1. Im getting very noisy and blurry images with the parameters adapted from the example: { "modelInputs": { "prompt": "Super dog", "num_inference_steps": 10, "guidance_scale": 7.5, "width": 1024, "height": 1024, "seed": 3239022079 }, "callInputs": { "MODEL_ID": "stabilityai/stable-diffusion-2-1", "PIPELINE": "StableDiffusionPipeline", "SCHEDULER": "LMSDiscreteScheduler", "safety_checker": true } } The output looks like this: [image: cbimage] <https://user-images.githubusercontent.com/24450307/213272089-6b940d50-ad94-441f-a3e3-575f0a90a86d.png> Am i missing something important here or this a bug? — Reply to this email directly, view it on GitHub <#28>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC5IGWOBL2R4FWX64YKEKDWTA5WDANCNFSM6AAAAAAT7OM6YQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[gadicc]

You may also get better results at the native model resolution of 768x768, but definitely the issue here is the small number of inference steps.

Also, FWIW, I personally preferred the outputs from "super dog" from the original 1.4 model, but it may just be a matter of taste 😅

[jan-thoma]

even with 50 steps, i get about the same output

{
  "modelInputs": {
    "prompt": "super dog",
    "num_inference_steps": 50,
    "guidance_scale": 7.5,
    "width": 768,
    "height": 768,
    "seed": 3239022079
  },
  "callInputs": {
    "MODEL_ID": "stabilityai/stable-diffusion-2-1",
    "PIPELINE": "StableDiffusionPipeline",
    "SCHEDULER": "LMSDiscreteScheduler",
    "safety_checker": true
  }
}

[jan-thoma]

Here are results from the other schedulers, with the same parameters as above

DDIMScheduler

DPMSolverMultistepScheduler

EulerAncestralDiscreteScheduler

IPNDMScheduler

PNDMScheduler

[gadicc]

Oh, you know, did you build from main branch? It's a little behind. I think going back far enough, not all the diffusers schedulers were working with SDv2. You could try with 'dev' branch or see even https://forums.kiri.art/t/wip-upgrading-from-v0-to-v1/116?u=gadicc which will be merged to dev soon (and is available as a prebuilt docker image). Anyway, if that was the issue, sorry for your wasted time, otherwise, I'll take a look into this properly tomorrow. Big changes coming soon to the repo architecture (as per the forum post) which why the old main release is so behind. Will add a note to the readme about it.

…

On Wed, 18 Jan 2023, 22:17 Jan Thoma, ***@***.***> wrote: Here are results from the other schedulers, with the same parameters as above DDIMScheduler [image: DDIM] <https://user-images.githubusercontent.com/24450307/213285037-8266a510-c5d1-424c-981c-582e34f84e47.png> DPMSolverMultistepScheduler [image: DPMSolverMultistepScheduler] <https://user-images.githubusercontent.com/24450307/213285044-642049d7-a1d0-43a7-926f-28175c399f61.png> EulerAncestralDiscreteScheduler [image: EulerAncestralDiscreteScheduler] <https://user-images.githubusercontent.com/24450307/213285053-fe2dc7b8-6659-45db-be40-536baea0caf5.png> IPNDMScheduler [image: IPNDMScheduler] <https://user-images.githubusercontent.com/24450307/213285059-6d68c06c-6ecf-42d8-9383-aa433eb05bfe.png> PNDMScheduler [image: PNDMScheduler] <https://user-images.githubusercontent.com/24450307/213285065-b89f686d-d8cf-48cd-81aa-40b729d9dbdd.png> — Reply to this email directly, view it on GitHub <#28 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC5IGRGOWZ3K6NJEA2FNWDWTBFTXANCNFSM6AAAAAAT7OM6YQ> . You are receiving this because you commented.Message ID: ***@***.***>

[gadicc]

And thanks for sharing all those pics which made the issue much clearer! 🙏🙏

[jan-thoma]

Thanks for the help and the great work with this repo. Switching to dev did the trick. Wired up this container with GPT3 and Twitter to generate fictional metal albums. Works great so far (https://twitter.com/bare_metal_head)

[gadicc]

Awesome, thanks for reporting back and for your kind words 🙏

Very cool app! Thanks for sharing on the forums too, where I've linked back to it. Be in touch with any more issues, otherwise, Happy Diffusing! 😁