eLLM can infer LLM on CPUs faster than on GPUs
-
Updated
Apr 30, 2026 - Rust
#
llm-infernece
Here are 4 public repositories matching this topic...
Efficient LLM inference on Slurm clusters.
inference speech-to-text vlm text-embedding multimodal audio-transcription llm vllm reward-model llm-infernece sglang llm-infrastructure
-
Updated
Apr 27, 2026 - Python
Layered prefill changes the scheduling axis from tokens to layers and removes redundant MoE weight reloads while keeping decode stall free. The result is lower TTFT, lower end-to-end latency, and lower energy per token without hurting TBT stability.
-
Updated
Mar 9, 2026 - Python
Improve this page
Add a description, image, and links to the llm-infernece topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-infernece topic, visit your repo's landing page and select "manage topics."