Qeravio
Canonical AI event

Llama.cpp server now routes all model loads through a queue

The fix ensures all model loads go through the queue, preventing eviction during transitions. Requests to stopping models are now queued and handled by the next instance.

23 Sept 20261 verified claims1 sources1 observations
What happened

The official source reports this update: b11114: server: fix router eviction races with the existing queue. server: fix router eviction races with the existing queue ( #29217 ) server: route every model load through the queue A model loaded by the fast path has no queue entry, so tick() evicts it at its LOADED transition before its own request is proxied. Every load now joins the queue, whose entry protects the model until its waiters leave. server: do not admit requests into a stopping model A request for a model that is being stopped still sees it LOADED and is proxied into the dying child. Such a request now joins the queue and is served by the next instance.

Why it matters

This official update documents a development concerning b11114: server: fix router eviction races with the existing queue. Its practical significance depends on the scope and evidence stated by the source.

What to watch next

Read the official source update and verify its stated scope, evidence and timing before acting on it.

Connected knowledge

Entities affected by this event

Continue this topic
Evidence trail

Sources behind the event