Qeravio
Canonical AI event

CUDA now allows concurrent streams per split for multi-GPU.

CUDA optimization now runs per split in multi-GPU setups, fixing previous graph optimization issues.

4 Sept 20261 verified claims1 sources1 observations
What happened

The official source reports this update: b10782: CUDA: Allow concurrent streams per split for multi-GPU. CUDA: Allow concurrent streams per split for multi-GPU ( #28198 ) CUDA: Allow CUDA optimization per split for multi-GPU. Previous guard caused multi-GPU to skip the graph optimization. The graph is already split per device and the optimization doesnt run over the whole model but once per split, and thus should be allowed. However, the CUDA event ggml_cuda_concurrent_event belongs to whichever GPU was "current" when created. If the pass ran while GPU 0 was current, it would stick and during event creation for the second GPU it would land on GPU 0.

Why it matters

This official update documents a development concerning b10782: CUDA: Allow concurrent streams per split for multi-GPU. Its practical significance depends on the scope and evidence stated by the source.

What to watch next

Read the official source update and verify its stated scope, evidence and timing before acting on it.

Connected knowledge

Entities affected by this event

Evidence trail

Sources behind the event