Random Number Streams on a GPU

By default, the random number generation functions rand, randi, and randn use different generator settings for calculations on a GPU compared to those on a CPU. You can change the behavior of random number generators to generate reproducible sequences of random numbers on the GPU and CPU.

The table below summarizes the default settings for the GPU and CPU on client and worker MATLAB^® sessions:

	Generator	Seed	Normal Transform
Client CPU	`'Twister'` or `'mt19937ar'`	`0`	`'Ziggurat'`
Worker CPU	`'Threefry'` or `'Threefry4x64_20'`	`0`	`'Inversion'`
GPU (on client or worker)	`'Threefry'` or `'Threefry4x64_20'`	`0`	`'BoxMuller'`

In most cases, it does not matter that the default random number generator on the GPU is not the same as the default generators on the client or worker CPU. However, if you need to reproduce the same results on both the GPU and CPU, you can set the generators accordingly.

Client CPU and GPU

In a fresh MATLAB session, MATLAB generates different sequences of random numbers on the CPU and GPU.

Rc = rand(1,4)

Rc =
    0.8147    0.9058    0.1270    0.9134

Rg = rand(1,4,'gpuArray')

Rg =
    0.3640    0.5421    0.6543    0.7436

If you need to generate the same sequence of random numbers on both the GPU and CPU, you can set the generator settings to match.

There are three random number generator algorithms available on the GPU: 'Threefry', 'Philox', and 'CombRecursive'. All are supported on the CPU. The following table lists the algorithms for these generators and their properties.

Keyword	Generator	Multiple Stream and Substream Support	Approximate Period in Full Precision
`"Threefry"` or `"Threefry4x64_20"`	Threefry 4x64 generator with 20 rounds	Yes	2⁵¹⁴ (2²⁵⁶ streams of length 2²⁵⁸)
`"Philox"` or `"Philox4x32_10"`	Philox 4x32 generator with 10 rounds	Yes	2¹⁹³ (2⁶⁴ streams of length 2¹²⁹)
`"CombRecursive"` or `"mrg32k3a"`	Combined multiple recursive generator	Yes	2¹⁹¹ (2⁶³ streams of length 2¹²⁷)

You can use rng and gpurng to set the generator algorithm and seed on the CPU and GPU, respectively. For more information about the GPU random number generators and their performance, see Generating Random Numbers on a GPU.

sc = rng(1,'Threefry');
Rc = rand(1,4)

Rc =
   0.1404    0.8197    0.1073    0.4131

sg = gpurng(1,'Threefry');
Rg = rand(1,4,'gpuArray')

Rg =
    0.1404    0.8197    0.1073    0.4131

rand and randi now generate the same sequences of random numbers on the client CPU and GPU.

You can also use rng and gpurng to reset the generator algorithm and seed to the default values on the CPU and GPU respectively.

rsc = rng('default')
rsg = gpurng('default')

Worker CPU and GPU

A parallel worker CPU uses the same default random number generator type and seed as the client GPU and the worker GPU, if it has one. The GPU and CPU do not share the same stream. By default, rand and randi generate the same sequence of numbers on a GPU and a worker CPU.

The settings are different from those on the client CPU. For more information, see Control Random Number Streams on Workers

If you need to generate different random numbers on each worker, you can change the generator settings. In this example, each worker creates the same sequence on its GPU and CPU, but different sequences are generated on each worker.

p = parpool(2);
spmd
    rng(spmdIndex,'Threefry');
    Rc = rand(1,4)

    gpurng(spmdIndex,'Threefry');
    Rg = rand(1,4,'gpuArray')
end
delete(p)

Normally Distributed Random Numbers

For normally distributed random numbers created using the randn function, MATLAB produces different results on a client CPU, a worker CPU and a GPU. The transformation of uniform random numbers into normally distributed random numbers is controlled by the NormalTransform setting. You can control this on the GPU using parallel.gpu.RandStream.

On a client CPU, the default 'NormalTransform' setting is 'Ziggurat'. On a worker CPU, the default setting is 'Inversion'.

Unless otherwise specified, GPU code uses the 'BoxMuller' transform for the 'Threefry' and 'Philox' generators and the 'Inversion' transform for the 'CombRecursive' generator.

You can set the same generators and transforms on the CPU and the GPU to get the same randn sequences. The only transform supported on both the CPU and GPU is the 'Inversion' transform.

sc = RandStream('Threefry','NormalTransform','Inversion','Seed',1);
RandStream.setGlobalStream(sc)

sg = parallel.gpu.RandStream('Threefry','NormalTransform','Inversion','Seed',1);
parallel.gpu.RandStream.setGlobalStream(sg);

Rc = randn(1,4)

Rc =
   -1.0783    0.9144   -1.2412   -0.2196

Rg = randn(1,4,'gpuArray')

Rg =
   -1.0783    0.9144   -1.2412   -0.2196