The official source reports this update: b10782: CUDA: Allow concurrent streams per split for multi-GPU. CUDA: Allow concurrent streams per split for multi-GPU ( #28198 ) CUDA: Allow CUDA optimization per split for multi-GPU. Previous guard caused multi-GPU to skip the graph optimization. The graph is already split per device and the optimization doesnt run over the whole model but once per split, and thus should be allowed. However, the CUDA event ggml_cuda_concurrent_event belongs to whichever GPU was "current" when created. If the pass ran while GPU 0 was current, it would stick and during event creation for the second GPU it would land on GPU 0.
Canonical AI event
CUDA now allows concurrent streams per split for multi-GPU.
CUDA optimization now runs per split in multi-GPU setups, fixing previous graph optimization issues.
4 Sept 20261 verified claims1 sources1 observations
This official update documents a development concerning b10782: CUDA: Allow concurrent streams per split for multi-GPU. Its practical significance depends on the scope and evidence stated by the source.
Read the official source update and verify its stated scope, evidence and timing before acting on it.
Connected knowledge
Entities affected by this event
Evidence trail
Sources behind the event
Editorial presentation
Open story