estimateNetworkMetrics

Estimate network metrics for specific layers of a neural network

Since R2022a

Syntax

dataTable = estimateNetworkMetrics(net)

[dataTable1,dataTable2,…,dataTableN] = estimateNetworkMetrics(net1,net2,…,netN)

Description

Add-On Required: This feature requires the Deep Learning Toolbox Model Compression Library add-on.

dataTable = estimateNetworkMetrics(net) returns a table containing estimated layer-wise metrics for a deep neural network.

This function estimates metrics for learnable layers, which have weights and bias, in the network. Estimated metrics are provided for the following supported layers.

• convolution1dLayer

• convolution2dLayer

• groupedConvolution2dLayer

• fullyConnectedLayer

• lstmLayer

• lstmProjectedLayer

• bilstmLayer

[dataTable1,dataTable2,…,dataTableN] = estimateNetworkMetrics(net1,net2,…,netN) returns metrics for multiple networks.

Examples

Estimate Metrics for Neural Network Layers

This example shows how to estimate layer-wise metrics for a neural network.

Load the pretrained network. net is a SqueezeNet convolutional neural network that has been retrained using transfer learning to classify images in the MerchData data set.

load squeezedlnetmerch
net

net = 
  dlnetwork with properties:

         Layers: [67×1 nnet.cnn.layer.Layer]
    Connections: [74×2 table]
     Learnables: [52×3 table]
          State: [0×3 table]
     InputNames: {'data'}
    OutputNames: {'prob'}
    Initialized: 1

  View summary with summary.

Use the estimateNetworkMetrics function to estimate metrics for the supported layers in your network.

estNet = estimateNetworkMetrics(net)

estNet=26×8 table
        LayerName             LayerType        NumberOfLearnables    NumberOfOperations    ParameterMemory (MB)    StateMemory (MB)    NumberOfMACs    ArithmeticIntensity
    __________________    _________________    __________________    __________________    ____________________    ________________    ____________    ___________________

    "conv1"               "2-D Convolution"           1792                4.413e+07             0.0068359                 0             2.2065e+07           25.739       
    "fire2-squeeze1x1"    "2-D Convolution"           1040               6.4225e+06             0.0039673                 0             3.2113e+06           12.748       
    "fire2-expand1x1"     "2-D Convolution"           1088               6.4225e+06             0.0041504                 0             3.2113e+06           12.748       
    "fire2-expand3x3"     "2-D Convolution"           9280               5.7803e+07                0.0354                 0             2.8901e+07           111.12       
    "fire3-squeeze1x1"    "2-D Convolution"           2064               1.2845e+07             0.0078735                 0             6.4225e+06           14.158       
    "fire3-expand1x1"     "2-D Convolution"           1088               6.4225e+06             0.0041504                 0             3.2113e+06           12.748       
    "fire3-expand3x3"     "2-D Convolution"           9280               5.7803e+07                0.0354                 0             2.8901e+07           111.12       
    "fire4-squeeze1x1"    "2-D Convolution"           4128               6.4225e+06              0.015747                 0             3.2113e+06           24.791       
    "fire4-expand1x1"     "2-D Convolution"           4224               6.4225e+06              0.016113                 0             3.2113e+06           24.791       
    "fire4-expand3x3"     "2-D Convolution"          36992               5.7803e+07               0.14111                 0             2.8901e+07           178.07       
    "fire5-squeeze1x1"    "2-D Convolution"           8224               1.2845e+07              0.031372                 0             6.4225e+06           27.449       
    "fire5-expand1x1"     "2-D Convolution"           4224               6.4225e+06              0.016113                 0             3.2113e+06           24.791       
    "fire5-expand3x3"     "2-D Convolution"          36992               5.7803e+07               0.14111                 0             2.8901e+07           178.07       
    "fire6-squeeze1x1"    "2-D Convolution"          12336               4.8169e+06              0.047058                 0             2.4084e+06            33.51       
    "fire6-expand1x1"     "2-D Convolution"           9408               3.6127e+06              0.035889                 0             1.8063e+06           32.109       
    "fire6-expand3x3"     "2-D Convolution"          83136               3.2514e+07               0.31714                 0             1.6257e+07           125.07       
      ⋮

Compare Metrics for Floating-Point and Quantized Neural Network

This example shows how to estimate the metrics for a floating-point and quantized neural network.

Load the pretrained network. net is a SqueezeNet convolutional neural network that has been retrained using transfer learning to classify images in the MerchData data set.

load squeezedlnetmerch
net

net = 
  dlnetwork with properties:

         Layers: [67×1 nnet.cnn.layer.Layer]
    Connections: [74×2 table]
     Learnables: [52×3 table]
          State: [0×3 table]
     InputNames: {'data'}
    OutputNames: {'prob'}
    Initialized: 1

  View summary with summary.

Unzip and load the MerchData images as an image datastore. Define an augmentedImageDatastore object to resize the data for the network, and split the data into calibration and validation data sets to use for quantization.

unzip('MerchData.zip');
imds = imageDatastore('MerchData', ...
    'IncludeSubfolders',true, ...
    'LabelSource','foldernames');
[calData, valData] = splitEachLabel(imds, 0.7,'randomized');
aug_calData = augmentedImageDatastore([227 227],calData);
aug_valData = augmentedImageDatastore([227 227],valData);

Create a dlquantizer object and specify the network to quantize. Set the execution environment to MATLAB. When you use the MATLAB execution environment, quantization is performed using the fi fixed-point data type which requires a Fixed-Point Designer™ license.

quantObj = dlquantizer(net,'ExecutionEnvironment','MATLAB');

Use the calibrate function to exercise the network with sample inputs and collect range information.

calResults = calibrate(quantObj,aug_calData);

Use the quantize method to quantize the network object and return a simulatable quantized network.

qNet = quantize(quantObj)

qNet = 
  Quantized dlnetwork with properties:

         Layers: [67×1 nnet.cnn.layer.Layer]
    Connections: [74×2 table]
     Learnables: [52×3 table]
          State: [0×3 table]
     InputNames: {'data'}
    OutputNames: {'prob'}
    Initialized: 1

  View summary with summary.
  Use the quantizationDetails function to extract quantization details.

Use the estimateNetworkMetrics function to estimate metrics for the floating-point and quantized networks.

[dataTableFloat,dataTableQuantized] = estimateNetworkMetrics(net,qNet)

dataTableFloat=26×8 table
        LayerName             LayerType        NumberOfLearnables    NumberOfOperations    ParameterMemory (MB)    StateMemory (MB)    NumberOfMACs    ArithmeticIntensity
    __________________    _________________    __________________    __________________    ____________________    ________________    ____________    ___________________

    "conv1"               "2-D Convolution"           1792                4.413e+07             0.0068359                 0             2.2065e+07           25.739       
    "fire2-squeeze1x1"    "2-D Convolution"           1040               6.4225e+06             0.0039673                 0             3.2113e+06           12.748       
    "fire2-expand1x1"     "2-D Convolution"           1088               6.4225e+06             0.0041504                 0             3.2113e+06           12.748       
    "fire2-expand3x3"     "2-D Convolution"           9280               5.7803e+07                0.0354                 0             2.8901e+07           111.12       
    "fire3-squeeze1x1"    "2-D Convolution"           2064               1.2845e+07             0.0078735                 0             6.4225e+06           14.158       
    "fire3-expand1x1"     "2-D Convolution"           1088               6.4225e+06             0.0041504                 0             3.2113e+06           12.748       
    "fire3-expand3x3"     "2-D Convolution"           9280               5.7803e+07                0.0354                 0             2.8901e+07           111.12       
    "fire4-squeeze1x1"    "2-D Convolution"           4128               6.4225e+06              0.015747                 0             3.2113e+06           24.791       
    "fire4-expand1x1"     "2-D Convolution"           4224               6.4225e+06              0.016113                 0             3.2113e+06           24.791       
    "fire4-expand3x3"     "2-D Convolution"          36992               5.7803e+07               0.14111                 0             2.8901e+07           178.07       
    "fire5-squeeze1x1"    "2-D Convolution"           8224               1.2845e+07              0.031372                 0             6.4225e+06           27.449       
    "fire5-expand1x1"     "2-D Convolution"           4224               6.4225e+06              0.016113                 0             3.2113e+06           24.791       
    "fire5-expand3x3"     "2-D Convolution"          36992               5.7803e+07               0.14111                 0             2.8901e+07           178.07       
    "fire6-squeeze1x1"    "2-D Convolution"          12336               4.8169e+06              0.047058                 0             2.4084e+06            33.51       
    "fire6-expand1x1"     "2-D Convolution"           9408               3.6127e+06              0.035889                 0             1.8063e+06           32.109       
    "fire6-expand3x3"     "2-D Convolution"          83136               3.2514e+07               0.31714                 0             1.6257e+07           125.07       
      ⋮

dataTableQuantized=26×8 table
        LayerName             LayerType        NumberOfLearnables    NumberOfOperations    ParameterMemory (MB)    StateMemory (MB)    NumberOfMACs    ArithmeticIntensity
    __________________    _________________    __________________    __________________    ____________________    ________________    ____________    ___________________

    "conv1"               "2-D Convolution"           1750                4.413e+07               0.001852                0             2.2065e+07            25.74       
    "fire2-squeeze1x1"    "2-D Convolution"            987               6.4225e+06             0.00098705                0             3.2113e+06           12.751       
    "fire2-expand1x1"     "2-D Convolution"           1038               6.4225e+06               0.001173                0             3.2113e+06            12.75       
    "fire2-expand3x3"     "2-D Convolution"           8784               5.7803e+07              0.0085602                0             2.8901e+07           111.33       
    "fire3-squeeze1x1"    "2-D Convolution"           2001               1.2845e+07              0.0019541                0             6.4225e+06            14.16       
    "fire3-expand1x1"     "2-D Convolution"           1062               6.4225e+06              0.0011959                0             3.2113e+06           12.749       
    "fire3-expand3x3"     "2-D Convolution"           8850               5.7803e+07              0.0086231                0             2.8901e+07            111.3       
    "fire4-squeeze1x1"    "2-D Convolution"           3878               6.4225e+06              0.0037899                0             3.2113e+06           24.838       
    "fire4-expand1x1"     "2-D Convolution"           4084               6.4225e+06               0.004261                0             3.2113e+06           24.817       
    "fire4-expand3x3"     "2-D Convolution"          34573               5.7803e+07               0.033338                0             2.8901e+07           180.76       
    "fire5-squeeze1x1"    "2-D Convolution"           7900               1.2845e+07              0.0076256                0             6.4225e+06           27.487       
    "fire5-expand1x1"     "2-D Convolution"           4056               6.4225e+06              0.0042343                0             3.2113e+06           24.823       
    "fire5-expand3x3"     "2-D Convolution"          34547               5.7803e+07               0.033313                0             2.8901e+07           180.79       
    "fire6-squeeze1x1"    "2-D Convolution"          11734               4.8169e+06               0.011328                0             2.4084e+06           33.793       
    "fire6-expand1x1"     "2-D Convolution"           9191               3.6127e+06              0.0093145                0             1.8063e+06           32.234       
    "fire6-expand3x3"     "2-D Convolution"          77015               3.2514e+07               0.073997                0             1.6257e+07           131.25       
      ⋮

Compare the parameter memory requirements of the layers supported by estimateNetworkMetrics for the floating-point and quantized networks.

totalMemoryFloat = sum(dataTableFloat.("ParameterMemory (MB)"));
totalMemoryQuantized = sum(dataTableQuantized.("ParameterMemory (MB)"));
percentReduction = (totalMemoryFloat - totalMemoryQuantized)*100/totalMemoryFloat

percentReduction = 
76.0673

Input Arguments

net — Neural network

dlnetwork object | DAGNetwork object | SeriesNetwork object | taylorPrunableNetwork object | RegressionNeuralNetwork object | CompactRegressionNeuralNetwork object | ClassificationNeuralNetwork object | CompactClassificationNeuralNetwork object

Neural network, specified as one of these values:

• dlnetwork object

• DAGNetwork object

• SeriesNetwork object

• taylorPrunableNetwork object

• RegressionNeuralNetwork (Statistics and Machine Learning Toolbox) object

• CompactRegressionNeuralNetwork (Statistics and Machine Learning Toolbox) object

• ClassificationNeuralNetwork (Statistics and Machine Learning Toolbox) object

• CompactClassificationNeuralNetwork (Statistics and Machine Learning Toolbox) object

estimateNetworkMetrics supports both floating-point and quantized networks.

net1,net2,…,netN — Neural networks

dlnetwork object | DAGNetwork object | SeriesNetwork object | taylorPrunableNetwork object | RegressionNeuralNetwork object | CompactRegressionNeuralNetwork object | ClassificationNeuralNetwork object | CompactClassificationNeuralNetwork object

Neural networks, specified as a comma-separated list of any of the following values:

• dlnetwork objects

• DAGNetwork objects

• SeriesNetwork objects

• taylorPrunableNetwork objects

• RegressionNeuralNetwork (Statistics and Machine Learning Toolbox) objects

• CompactRegressionNeuralNetwork (Statistics and Machine Learning Toolbox) objects

• ClassificationNeuralNetwork (Statistics and Machine Learning Toolbox) objects

• CompactClassificationNeuralNetwork (Statistics and Machine Learning Toolbox) objects

estimateNetworkMetrics supports both floating-point and quantized networks.

Output Arguments

dataTable — Estimated layer metrics

table

Estimated layer metrics, returned as a table containing these metrics:

• LayerName — Name of layer.

• LayerType — Type of layer.

• NumberOfLearnables — Number of non-zero learnable parameters (weights and biases) in the network.

• NumberOfOperations — Total number of multiplications and additions.

• ParameterMemory (MB) — Memory required to store all of the learnable parameters.

• StateMemory (MB) — Memory required to store all of the state parameters. For layers without state parameters, this value is zero.

• NumberOfMACs — Number of multiply-accumulate operations.

• ArithmeticIntensity (FLOP/B) — Amount of reuse of data fetched from memory, measured as the number of floating-point operations performed per the bytes of memory access required to support those operations. For example, convolutional layers reuse the same weight data across computations for multiple input features, resulting in a relatively high arithmetic intensity.

[dataTable1,dataTable2,…,dataTableN] — Estimated layer metrics

tables

Estimated layer metrics, returned as tables containing the metrics for each deep neural network you input. Each table contains the same metrics as dataTable.

Limitations

• To estimate metrics for a fullyConnectedLayer layer, the InputLearnables and OutputLearnables values must be {}.

Version History

Introduced in R2022a

R2026a: Estimate memory of state parameters

The dataTable output now contains a StateMemory (MB) metric. This metric estimates the memory required to store all of the state parameters for each layer. For layers without state parameters, this value is zero.

In previous releases, the dataTable output contained a StateMemory metric that returned the number of states for each layer when your network contained at least one layer with a state parameter. The StateMemory (MB) metric replaces the StateMemory metric.

R2022b: Calculate number of state parameters

When your network contains at least one layer with a state parameter, the dataTable output contains a StateMemory metric. The StateMemory metric counts the number of states for each layer.