Qeravio
Canonical AI event

Llama.cpp updates GDN normalization to match flash-linear-attention's l2norm.

The llama.cpp update aligns GDN normalization with flash-linear-attention's l2norm, fixing an epsilon handling discrepancy.

6 Sept 20261 verified claims1 sources1 observations
What happened

The official source reports this update: b10829: models : fix GDN normalization from max to rsqrt. models : fix GDN normalization from max to rsqrt ( #28068 ) models: use flash-linear-attention's l2norm for gated delta net q/k The GDN q/k normalization is defined by flash-linear-attention as l2norm(x) = x * rsqrt(sum(x*x) + eps) with eps inside the root. Every GDN call site in the tree uses ggml_l2_norm instead, which is x / max(sqrt(sum(x*x)), eps), i.e. torch.nn.functional.normalize - its CUDA kernel cites that page. The clamp never engages at these magnitudes, so in practice llama.cpp normalizes with no epsilon at all where the reference has one inside the root.

Why it matters

This official update documents a development concerning b10829: models : fix GDN normalization from max to rsqrt. Its practical significance depends on the scope and evidence stated by the source.

What to watch next

Read the official source update and verify its stated scope, evidence and timing before acting on it.

Connected knowledge

Entities affected by this event

Evidence trail

Sources behind the event