How do I determine the output size of a fullyConnectedLayer?

Hello! I was working on turing this Siamese Network example into a 1D network to compare signals. I am trying to reduce how many convultional layers this network is using as my signals are very short. I was wondering how the number 4096 was determined for the output of the fullyConnectedLayer?

I am a beginner so a simplified explanation would be very helpful.

Thank you!

Answer 1

[Raunak Gupta]

Hi,

You can use analyzeNetwork for inspecting what is the input and output size from each layer. This can help you determine valid inputs to the layer. Let’s assume you define a network as mentioned here you may use

analyzeNetwork(lgraph) 

which will open the Deep Learning Network Analyzer and in that the activations column show the output size from a layer. Here since the output just before fullyConnectedLayer is having a size of 5x5x256 and we want the representation in 4096-dimensional feature vector hence the output size of fullyConnectedLayer is 4096.

If you remove some layers from the network, it will still work but will have less weight compared to previous model and may not be able to provide as accurate results as the previous one.

[Daniel Bevan]

Thank you for your answer! Unfortutnantly I am still confused as to where the number "4096" came from. 5x5x256 is 6400, not 4096.

Am I missing something?

[Raunak Gupta]

Hi Daniel,

We can take output based on the number of features we want at the end. So in this case the output feature vector required is having size 4096 that is why it is used. Be sure that the second last layer output will be multiplied with weight matrix of size 6400x4096 to get appropriate result (a feature vector of size 4096). Also if the last fully connected layer had let's say 1024 nodes then the layer would have been giving 1024 size vector as output and in that case the above mentioned matrix would have a size of 6400x1024.

Hope this clarifies your doubt!