Inference Deceleration

Inference runtimes like llama.cpp and vLLM are experiencing sharp velocity drops, with llama.cpp at 26.0 per day but -21.4 acceleration and vLLM at 14.3 with -7.3, signaling a post-release stabilization after updates like commit f84270e for tile buffers and v0.20.0 CUDA defaults.

Evidence includes Ollama's parallel -5.9 deceleration post v0.21.0 Hermes Agent, where star growth of 11.7 trails llama.cpp's 26.0, reflecting saturated demand for local inference amid hardware constraints.

This implies a pivot opportunity for efficiency-focused forks, as developers shift from broad adoption to specialized tweaks, potentially eroding dominance if acceleration doesn't rebound within 14 days.

Projects in this theme: ggml-org/llama.cpp · vllm-project/vllm · ollama/ollama

Trajectory: appeared in 1 briefing between 2026-04-27 and 2026-04-28.

Briefings that covered this theme

• 2026-04-28 · AI OSS Momentum Decelerates Across Key Projects
  Inference runtimes like llama.cpp and vLLM are experiencing sharp velocity drops, with llama.cpp at 26.0 per day but -21.4 acceleration and vLLM at 14.3 with -7.3, signaling a post-release stabilization after updates like commit f84270e for