analyzeNetwork

Analyze deep learning network architecture

Syntax

analyzeNetwork(layers)

analyzeNetwork(layers,TargetUsage=target)

analyzeNetwork(layers,X1,...,Xn,TargetUsage="dlnetwork")

analyzeNetwork(net)

analyzeNetwork(net,X1,...,Xn)

Description

Use analyzeNetwork to visualize and understand the architecture of a network, check that you have defined the architecture correctly, and detect problems before training. Problems that analyzeNetwork detects include missing or unconnected layers, incorrectly sized layer inputs, an incorrect number of layer inputs, and invalid graph structures.

Tip

To interactively visualize, analyze, and train a network, use deepNetworkDesigner(net). For more information, see Deep Network Designer.

analyzeNetwork(layers) analyzes the network layers given by the layer array or layer graph layers and also detects errors and issues for trainNetwork workflows. The function displays an interactive visualization of the network architecture and provides detailed information about the network layers. The layer information includes the layer type, the size and format of the layer activations, and the size and number of learnable and state parameters.

Each activation dimension has one of the following labels: S (spatial), C (channel), B (batch), T (time or sequence), or U (unspecified).

analyzeNetwork(layers,TargetUsage=target) analyzes the network layers given by the layer array or layer graph layers for the specified target workflow. Use this syntax when analyzing the layers for dlnetwork workflows.

analyzeNetwork(layers,X1,...,Xn,TargetUsage="dlnetwork") analyzes the layer array or layer graph layers using the example networks inputs X1,...,Xn. The software propagates the example inputs through the network to determine the size and format of layer activations, the size and number of learnable and state parameters, and the total number of learnables. Use this syntax to analyze a network that has one or more inputs that are not connected to an input layer.

analyzeNetwork(net) analyzes the SeriesNetwork, DAGNetwork, or dlnetwork object net.

analyzeNetwork(net,X1,...,Xn) analyzes the dlnetwork object net using example networks inputs X1,...,Xn. The software propagates the example inputs through the network to determine the size and format of layer activations, the size and number of learnable and state parameters, and the total number of learnables. Use this syntax to analyze an uninitialized dlnetwork that has one or more inputs that are not connected to an input layer.

Examples

Analyze Trained Network

Load a pretrained GoogLeNet convolutional neural network.

net = googlenet;

Analyze the network. analyzeNetwork displays an interactive plot of the network architecture and a table containing information about the network layers.

analyzeNetwork(net)

Investigate the network architecture using the plot to the left. Select a layer in the plot. The selected layer is highlighted in the plot and in the layer table.

In the table, view layer information such as layer properties, layer type, and sizes of the layer activations and learnable parameters. The activations of a layer are the outputs of that layer. Each activation dimension has one of the following labels:

• S — Spatial

• C — Channel

• B — Batch observations

• T — Time or sequence

• U — Unspecified

View the dimension labels to understand how data propagates through the network and how the layers modify the size and layout of activations.

Select a deeper layer in the network. Notice that activations in deeper layers are smaller in the spatial dimensions (the first two dimensions) and larger in the channel dimension (the last dimension). Using this structure enables convolutional neural networks to gradually increase the number of extracted image features while decreasing the spatial resolution.

The Deep Learning Network Analyzer shows the total number of learnable parameters in the network, to one decimal place. To see the exact number of learnable parameters, pause on total learnables. To show the number of learnable parameters in each layer, click the arrow in the top-right corner of the layer table and select Number of Learnables. To sort the layer table by column value, hover the mouse over the column heading and click the arrow that appears. For example, you can determine which layer contains the most parameters by sorting the layers by the number of learnable parameters.

Fix Errors in Network Architecture

Create a simple convolutional network with some skip connections. Create a layer array containing the main branch of the network.

layers = [
    imageInputLayer([32 32 3])
    convolution2dLayer(5,16,Padding="same")
    reluLayer(Name="relu_1")
    convolution2dLayer(3,16,Padding="same",Stride=2)
    reluLayer
    additionLayer(2,Name="add_1")
    convolution2dLayer(3,16,Padding="same",Stride=2)
    reluLayer
    additionLayer(3,Name="add_2")
    fullyConnectedLayer(10)
    softmaxLayer
    classificationLayer];

lgraph = layerGraph(layers);

Convert the layer array to a layer graph and add the skip connections. One of the skip connections contains a single 1-by-1 convolutional layer conv_skip.

layer = convolution2dLayer(1,16,Stride=2,Name="conv_skip");
lgraph = addLayers(lgraph,layer);
lgraph = connectLayers(lgraph,"relu_1","add_1/in2");
lgraph = connectLayers(lgraph,"add_1","add_2/in2");

Analyze the network architecture using the analyzeNetwork function. The function finds issues with three layers in the network.

analyzeNetwork(lgraph)

Investigate and fix the errors in the network. In this example, the following issues cause the errors:

• The layer conv_skip is not connected to the rest of the network. It should be a part of the shortcut connection between the add_1 and add_2 layers. To fix this error, connect add_1 to conv_skip and conv_skip to add_2.

• The add_2 layer is specified to have three inputs, but the layer only has two inputs. To fix the error, specify the number of inputs as 2.

• All the inputs to an addition layer must have the same size, but the add_1 layer has two inputs with different sizes. Because the conv_2 layer has a Stride value of 2, this layer downsamples the activations by a factor of two in the first two dimensions (the spatial dimensions). To resize the input from the relu2 layer so that it has the same size as the input from relu_1, remove the downsampling by removing the Stride value of the conv_2 layer.

Apply these modifications to the layer graph construction from the beginning of this example and create a new layer graph.

layers = [
    imageInputLayer([32 32 3])
    convolution2dLayer(5,16,Padding="same")
    reluLayer(Name="relu_1")
    convolution2dLayer(3,16,Padding="same")
    reluLayer
    additionLayer(2,Name="add_1")
    convolution2dLayer(3,16,Padding="same",Stride=2)
    reluLayer
    additionLayer(2,Name="add_2")
    fullyConnectedLayer(10)
    softmaxLayer
    classificationLayer];

lgraph = layerGraph(layers);

layer = convolution2dLayer(1,16,Stride=2,Name="conv_skip");
lgraph = addLayers(lgraph,layer);
lgraph = connectLayers(lgraph,"relu_1","add_1/in2");
lgraph = connectLayers(lgraph,"add_1","conv_skip");
lgraph = connectLayers(lgraph,"conv_skip","add_2/in2");

Analyze the new architecture. The new network does not contain any errors and is ready to be trained.

analyzeNetwork(lgraph)

Analyze Layers for Custom Training Loop

Create a layer array for a custom training loop. For custom training loop workflows, the network must not have an output layer.

layers = [
    imageInputLayer([28 28 1],Normalization="none")
    convolution2dLayer(5,20,Padding="same")
    batchNormalizationLayer
    reluLayer
    convolution2dLayer(3,20,Padding="same")
    batchNormalizationLayer
    reluLayer
    convolution2dLayer(3,20,Padding="same")
    batchNormalizationLayer
    reluLayer
    fullyConnectedLayer(10)
    softmaxLayer];

Analyze the layer array using the analyzeNetwork function and set the TargetUsage option to "dlnetwork".

analyzeNetwork(layers,TargetUsage="dlnetwork")

Here, the function does not report any issues with the layer array.

Analyze Network Using Example Inputs

To analyze a network that has an input that is not connected to an input layer, you can provide example network inputs to the analyzeNetwork function. You can provide example inputs when you analyze dlnetwork objects, or when you analyze Layer arrays or LayerGraph objects for custom training workflows by setting the TargetUsage option to "dlnetwork".

Define the network architecture. Construct a network with two branches. The network takes two inputs, with one input per branch. Connect the branches using an addition layer.

numFilters = 24;
inputSize = [64 64 3];

layersBranch1 = [
    convolution2dLayer(3,6*numFilters,Padding="same",Stride=2)
    groupNormalizationLayer("all-channels")
    reluLayer
    convolution2dLayer(3,numFilters,Padding="same")
    groupNormalizationLayer("channel-wise")
    additionLayer(2,Name="add")
    reluLayer
    fullyConnectedLayer(10)
    softmaxLayer];

layersBranch2 = [
    convolution2dLayer(1,numFilters,Name="conv_branch")
    groupNormalizationLayer("all-channels",Name="gn_branch")];

lgraph = layerGraph(layersBranch1);
lgraph = addLayers(lgraph,layersBranch2);
lgraph = connectLayers(lgraph,"gn_branch","add/in2");

Create the dlnetwork. Because this network contains an unconnected input, create an uninitialized dlnetwork object by setting the Initialize option to false.

net = dlnetwork(lgraph,Initialize=false);

View the network input names.

net.InputNames

ans = 1×2 cell
    {'conv_1'}    {'conv_branch'}

Create example network inputs of the same size and format as typical inputs for this network using random dlarray objects. For both inputs, use a batch size of 32. Use an input of size 64-by-64 with three channels for the input to the layer "input". Use an input of size 64-by-64 with 18 channels for the input to the layer "conv_branch".

X1 = dlarray(rand([inputSize 32]),"SSCB");
X2 = dlarray(rand([32 32 18 32]),"SSCB");

Analyze the network. Provide the example inputs in the same order as the InputNames property of the dlnetwork. You must provide an example input for all network inputs, including inputs that are connected to an input layer.

analyzeNetwork(net,X1,X2)

Input Arguments

layers — Network layers

Layer array | LayerGraph object

Network layers, specified as a Layer array or a LayerGraph object.

For a list of built-in layers, see List of Deep Learning Layers.

net — Deep learning network

SeriesNetwork object | DAGNetwork object | dlnetwork object

Deep learning network, specified as a SeriesNetwork, DAGNetwork, or dlnetwork object.

target — Target workflow

"trainNetwork" (default) | "dlnetwork"

Target workflow, specified as one of the following:

• "trainNetwork" — Analyze layer graph for usage with the trainNetwork function. For example, the function checks that the layer graph has an output layer and no disconnected layer outputs.

• "dlnetwork" — Analyze layer graph for usage with dlnetwork objects. For example, the function checks that the layer graph does not have any output layers.

X1,...,Xn — Example network inputs

dlarray

Example network inputs, specified as formatted dlarray objects. The software propagates the example inputs through the network to determine the size and format of layer activations, the size and number of learnable and state parameters, and the total number of learnables.

Use example inputs when you want to analyze a network that does not have any input layers or that has inputs that are not connected to an input layer.

The order in which you must specify the example inputs depends on the type of network you are analyzing:

• Layer array — Provide example inputs in the same order that the layers that require inputs appear in the Layer array.

• LayerGraph — Provide example inputs in the same order as the layers that require inputs appear in the Layers property of the LayerGraph.

• dlnetwork — Provide example inputs in the same order as the inputs are listed in the InputNames property of the dlnetwork.

If a layer has multiple unconnected inputs, then example inputs for that layer must be specified separately in the same order as they appear in the layer’s InputNames property.

You must specify one example input for each input to the network, even if that input is connected to an input layer.

Version History

Introduced in R2018a