vMLX - JANGTQ Uber Compressed MLX Models - L2 Disk Cache (survives restart) + L1 Paged (super fast ttft) + Hybrid SSM Scheduler + Cont Batching + etc!

天枢 (Tianshu) 是一个全功能、高性能的终端编程智能体运行时（TUI）。它跳出了传统 AI 编程助手把大模型仅当成“工具”的局限，基于认知虚拟机 (CVM)、自感知层和信息素（Stigmergy）自衰减记忆构建，让 AI 成为有独立判断与认知防护的“开发伙伴”。同时针对 DeepSeek V4 做了前缀缓存工程优化（长会话实测稳态命中率 95–99%）

Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.

Freeze Claude Code's prompt prefix so DeepSeek's automatic cache always hits — alignment proxy + coalescing + keepalive, installable as a CC plugin. Measured 64% cheaper on real Claude Code traffic.

DevWhale —— AI 驱动桌面开发工作台。深度契合Deepseek V4,做了针对性缓存优化。Electron + React + TypeScript，流式 Agent对话、Monaco 编辑器、xterm.js 终端、60+ 文件格式多模态输入、多模型切换。DevWhale — AI desktop dev workbench. Electron + React + TS. Streaming Agent chat,Monaco editor, xterm terminal, 60+ file formats, multi-model support.

KVTide is a Kubernetes-native LLM serving system exploring cache-aware scheduling and proactive peer-to-peer KV mobility.

DeepSeek缓存优化器 v1.1 — Reasonix四支柱 + 语义压缩 (命中率+30%)

RACS (Remote Agent Context Store): prefix-cache management for production agents. Stability-aware prompt planning, provider-faithful cache directives for Anthropic, OpenAI, Gemini, Bedrock and more, TTL keep-warm scheduling, prefix-drift detection, and hit-ratio and savings analytics. Zero dependencies, TypeScript, edge-ready.

Cost governance for Massive Intelligence (IM) agent orchestration: hard per-request, per-task, and per-day USD budgets with depletion events, pre-flight cost forecasting, prefix-cache break-even planning across sixteen provider profiles, and cheapest-first cascade routing. Zero dependencies, node-free core, deterministic, TypeScript-first.

Event-driven simulator for prefix KV-cache eviction policies in LLM serving systems

cachepin

End-to-end LLM serving simulator integrating scheduling, prefix caching, tensor allocation, and KV-cache management. 168-run sweep (72 baseline + 96 pressure). Key finding: ChunkedPrefill + LFU cache achieves 41% lower TTFT p95 and 94% prefix hit rate, but hits OOM first under memory pressure.

CacheGuard（缓存卫士）— a drop-in proxy that keeps DeepSeek's server-side prefix-cache stable in front of any coding agent, so cache-hit pricing never silently breaks.

dscache

Correctness-fixed Rust/PyO3 flat-array DFA prefix cache — rewrite of BCR-memory v1 with regression tests for four bugs and an SGLang/vLLM head-to-head harness.

Hermes plugin: DeepSeek prefix-cache wire shaping (Reasonix-inspired)

Measures real prefix cache costs on GPU across 3 experimental versions. Key findings: 2.41x speedup with prefix=512; multi-turn speedup grows to 2.06x over 10 turns; batch sharing breakeven at n=2 (prefix=512) vs n=12 (prefix=128); LFU cache with Zipf alpha=2.0 achieves 82% hit rate with only 4 cache slots.

Production LLM gateway: OpenAI-compatible API in front of OpenAI, Anthropic, and Bedrock. Ordered-fallback routing with per-provider circuit breakers, Redis prefix cache, Prometheus + Grafana, Kubernetes.

The cache of the LLM's memory

Block-aligned resident prefix cache for llama.cpp on Jetson, with CUDA validation, multi-model long-prefix benchmarks, and cache stress testing.