🌱 A tiny, readable LLM serving engine with vLLM/SGLang-style features.
transformers inference pytorch teaching lora quantization fastapi kv-cache ai-infra llm llm-serving vllm llm-inference speculative-decoding paged-attention sglang continuous-batching openai-compatible llm-infra llm-engine
-
Updated
May 28, 2026 - Python