The official source reports this update: b11177: CUDA: fuse RMS_NORM + SCALE into one kernel. CUDA: fuse RMS_NORM + SCALE into one kernel ( #29393 ) #28068 builds the GDN q/k l2norm as ggml_scale(ggml_rms_norm(x, eps/n), 1/sqrt(n)). This adds 2 SCALE nodes per GDN layer, 96 extra kernel launches per ubatch on Qwen3.8-27B (48 GDN layers). The extra kernels take no measurable GPU time, but each launch has a host/driver cost. It is small with plain batch processing and about 10x larger with draft-mtp speculative decoding. rms_norm_f32 gets a do_scale flag, the same pattern as do_multiply/do_add, so the fused path shares the kernel, the reduction and the launcher.
Canonical AI event
CUDA kernel fusion reduces kernel launches by 96 per batch on Qwen3.8-27B
Fusion of RMS_NORM and SCALE reduces kernel launches by 96 per microbatch on Qwen3.8-27B. Performance improves by up to 4.8% in draft-mtp speculative decoding.
26 Sept 20261 verified claims1 sources1 observations
This official update documents a development concerning b11177: CUDA: fuse RMS_NORM + SCALE into one kernel. Its practical significance depends on the scope and evidence stated by the source.
Read the official source update and verify its stated scope, evidence and timing before acting on it.
Connected knowledge
Entities affected by this event
Continue this topic
Related verified updates
Evidence trail
Sources behind the event
Editorial presentation
Open story