Thanks for the FOSS! Suggestion for future possible backends runtimes: Vulkan, OpenCL, SYCL/OpenVino/intel GPU, AMD gpu/ROCm/HIP.

[ghchris2021]

Thanks for the FOSS!

Suggestion for future possible backends runtimes: Vulkan, OpenCL, SYCL/OpenVino/intel GPU, AMD gpu/ROCm/HIP.

Vulkan and OpenCL both have the possibility of being very portable to GPUs and also to some extent CPUs that have supporting SW for it.

SYCL can run on various CPU / GPU platforms; it / openvino etc. is the primary ideal target to support intel gpus.

[James4Ever0]

This software is based on Candle. It is like PyTorch in Rust. So if you want more acceleration you should look for the development there.

[evilsocket]

i like people suggesting a bulk of new inferences backend like it's nothing :D

[malikwirin]

So GPU acceleration through ROCm has to be implemented in Candle first?

[malikwirin]

So GPU acceleration through ROCm has to be implemented in Candle first?

Their discussion about amd support can be found here

There is also a WIP CUDA implementation for non NVIDIA GPUs: https://github.com/vosen/ZLUDA