[Feature Request]: new attention optimization

[Nidal890]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Wondering if it is possible to incorporate metal-flash-attention into this project. Webui currently uses the mps backend, meanwhile metal flash attention is an open source alternative to mps based off of dao AI labs flashattention v2. It's mainly built for apple silicon gpus and is much faster and less resource hungry than mps. Not sure about amd radeon gpus on macos though. I used it on the draw things app and it's really good. I know that it can't do fp64 calculations and some other stuff I don't understand but I though I should share. The linked repo has more information and benchmarks.
https://github.com/philipturner/metal-flash-attention
This article article talks more in detail about mfa: https://engineering.drawthings.ai/integrating-metal-flashattention-accelerating-the-heart-of-image-generation-in-the-apple-ecosystem-16a86142eb18

Proposed workflow

1. Go to ....
2. Press ....
3. ...

Additional information

No response

[tiancool]

Interrogate CLIP now runs on MPS instead of CPU, how did you do it? This error occurred in my original version, how should I fix it? Thank you for your contribution.

[Nidal890]

@tiancool Unfortunately, only have the most basic of python skills and can't add it myself otherwise I'd this would be a PR. I've been paying attention to the work in this repo and thought I'd at least share the MFA repo I found along with info.

[tiancool]

@tiancool Unfortunately, only have the most basic of python skills and can't add it myself otherwise I'd this would be a PR. I've been paying attention to the work in this repo and thought I'd at least share the MFA repo I found along with info.

Tested it for a few days, love your modified and optimized version, very stable, thanks a lot!

[Nidal890]

@tiancool Unfortunately, only have the most basic of python skills and can't add it myself otherwise I'd this would be a PR. I've been paying attention to the work in this repo and thought I'd at least share the MFA repo I found along with info.

Tested it for a few days, love your modified and optimized version, very stable, thanks a lot!

Nah, you have to thank philipturner and liuliu for putting in all the work to make mfa as good as it is. Hopefully, we can see it implemented here in this repo