Check Accelerated Deep Learning Function Outputs

This example shows how to check that the outputs of accelerated functions match the outputs of the underlying function.

In some cases, the outputs of accelerated functions differ to the outputs of the underlying function. For example, you must take care when accelerating functions that use random number generation, such as a function that generates random noise to add to the network input. When caching the trace of a function that generates random numbers that are not dlarray objects, the accelerated function caches resulting random numbers in the trace. When reusing the trace, the accelerated function uses the cached random values. The accelerated function does not generate new random values.

To check that the outputs of the accelerated function match the outputs of the underlying function, use the CheckMode property of the accelerated function. When the CheckMode property of the accelerated function is 'tolerance' and the outputs differ by more than a specified tolerance, the accelerated function throws a warning.

Accelerate the function myUnsupportedFun, listed at the end of the example using the dlaccelerate function. The function myUnsupportedFun generates random noise and adds it to the input. This function does not support acceleration because the function generates random numbers that are not dlarray objects.

accfun = dlaccelerate(@myUnsupportedFun)

accfun = 
  AcceleratedFunction with properties:

          Function: @myUnsupportedFun
           Enabled: 1
         CacheSize: 50
           HitRate: 0
         Occupancy: 0
         CheckMode: 'none'
    CheckTolerance: 1.0000e-04

Clear any previously cached traces using the clearCache function.

clearCache(accfun)

To check that the outputs of reused cached traces match the outputs of the underlying function, set the CheckMode property to 'tolerance'.

accfun.CheckMode = 'tolerance'

accfun = 
  AcceleratedFunction with properties:

          Function: @myUnsupportedFun
           Enabled: 1
         CacheSize: 50
           HitRate: 0
         Occupancy: 0
         CheckMode: 'tolerance'
    CheckTolerance: 1.0000e-04

Evaluate the accelerated function with an array of ones as input, specified as a dlarray input.

dlX = dlarray(ones(3,3));
dlY = accfun(dlX)

dlY = 
  3×3 dlarray

    1.8147    1.9134    1.2785
    1.9058    1.6324    1.5469
    1.1270    1.0975    1.9575

Evaluate the accelerated function again with the same input. Because the accelerated function reuses the cached random noise values instead of generating new random values, the outputs of the reused trace differs from the outputs of the underlying function. When the CheckMode property of the accelerated function is 'tolerance' and the outputs differ, the accelerated function throws a warning.

dlY = accfun(dlX)

Warning: Accelerated outputs differ from underlying function outputs.

dlY = 
  3×3 dlarray

    1.8147    1.9134    1.2785
    1.9058    1.6324    1.5469
    1.1270    1.0975    1.9575

Random number generation using the 'like' option of the rand function with a dlarray object supports acceleration. To use random number generation in an accelerated function, ensure that the function uses the rand function with the 'like' option set to a traced dlarray object (a dlarray object that depends on an input dlarray object).

Accelerate the function mySupportedFun, listed at the end of the example. The function mySupportedFun adds noise to the input by generating noise using the 'like' option with a traced dlarray object.

accfun2 = dlaccelerate(@mySupportedFun);

Clear any previously cached traces using the clearCache function.

clearCache(accfun2)

To check that the outputs of reused cached traces match the outputs of the underlying function, set the CheckMode property to 'tolerance'.

accfun2.CheckMode = 'tolerance';

Evaluate the accelerated function twice with the same input as before. Because the outputs of the reused cache match the outputs of the underlying function, the accelerated function does not throw a warning.

dlY = accfun2(dlX)

dlY = 
  3×3 dlarray

    1.7922    1.0357    1.6787
    1.9595    1.8491    1.7577
    1.6557    1.9340    1.7431

dlY = accfun2(dlX)

dlY = 
  3×3 dlarray

    1.3922    1.7060    1.0462
    1.6555    1.0318    1.0971
    1.1712    1.2769    1.8235

Checking the outputs match requires extra processing and increases the time required for function evaluation. After checking the outputs, set the CheckMode property to 'none'.

accfun1.CheckMode = 'none';
accfun2.CheckMode = 'none';

Example Functions

The function myUnsupportedFun generates random noise and adds it to the input. This function does not support acceleration because the function generates random numbers that are not dlarray objects.

function out = myUnsupportedFun(dlX)

sz = size(dlX);
noise = rand(sz);
out = dlX + noise;

end

The function mySupportedFun adds noise to the input by generating noise using the 'like' option with a traced dlarray object.

function out = mySupportedFun(dlX)

sz = size(dlX);
noise = rand(sz,'like',dlX);
out = dlX + noise;

end