The official source reports this update: b10829: models : fix GDN normalization from max to rsqrt. models : fix GDN normalization from max to rsqrt ( #28068 ) models: use flash-linear-attention's l2norm for gated delta net q/k The GDN q/k normalization is defined by flash-linear-attention as l2norm(x) = x * rsqrt(sum(x*x) + eps) with eps inside the root. Every GDN call site in the tree uses ggml_l2_norm instead, which is x / max(sqrt(sum(x*x)), eps), i.e. torch.nn.functional.normalize - its CUDA kernel cites that page. The clamp never engages at these magnitudes, so in practice llama.cpp normalizes with no epsilon at all where the reference has one inside the root.
Canonical AI event
Llama.cpp updates GDN normalization to match flash-linear-attention's l2norm.
The llama.cpp update aligns GDN normalization with flash-linear-attention's l2norm, fixing an epsilon handling discrepancy.
6 Sept 20261 verified claims1 sources1 observations
This official update documents a development concerning b10829: models : fix GDN normalization from max to rsqrt. Its practical significance depends on the scope and evidence stated by the source.
Read the official source update and verify its stated scope, evidence and timing before acting on it.
Connected knowledge
Entities affected by this event
Evidence trail
Sources behind the event
Editorial presentation
Open story