rocm / cuda / metal - bfloat8/float8 support

[Geramy]

Hi guys, I been working on the rocm support and I was wondering if you object to adding float8 support and types for native optimizations in backends?

[zcbenz]

I think it is a nice thing to have but as you might have already noticed that we are bottlenecked at the number of reviewers.

So for adding new dtypes I think the only reasonable way to make progress is to refactor existing code to use the dispatch_types utils instead of if/switch first, which can be done in batches of small PRs that we would be happy to review and merge. And then we can gradually add the new dtype for each backend without worrying about missing a function.

If you just start a PR touch hundreds of files I think it would not go anywhere.

[Geramy]

I think it is a nice thing to have but as you might have already noticed that we are bottlenecked at the number of reviewers.

So for adding new dtypes I think the only reasonable way to make progress is to refactor existing code to use the dispatch_types utils instead of if/switch first, which can be done in batches of small PRs that we would be happy to review and merge. And then we can gradually add the new dtype for each backend without worrying about missing a function.

If you just start a PR touch hundreds of files I think it would not go anywhere.

Perfect that sounds great, and the fun begins haha! This will go great with the rocm implementation too! Can metal take advantage on the hardware level too? Example rocm / RDNA 3.5 has two 8-bit capable FPU per CU which should greatly increase speed. I would assume.

[zcbenz]

I have no idea about float8 support in metal actually, @angeloskath @jagrit06 should know about it.

[Geramy]

Pushed a PR regarding this, not implementing functions but just the base needed to start.
The absolute minimum to not create any compile errors.
#3374

[zcbenz]

I just realized there was already an issue on this, I'm merging this to #1670.