How to manually calculate a Neural Network output?

Asked by Jeff Chang on 2 May 2018

Latest activity Answered by Shounak Mitra on 8 Oct 2018

Accepted Answer by JESUS DAVID ARIZA ROYETH

117 views (last 30 days)

Dear everyone,

I am exploring the Neural Network Toolbox and would like to manually calculate output by hand. I used one of the example provided by Matlab with the following code. Unfortunately, my output is incorrect. Does anyone know why? Thanks

%%below is the sample code from Matlab
[x, y] = crab_dataset;
size(x)    % 6 x 200
size(y)    % 2 x 200
setdemorandstream(491218382);
net = patternnet(10);
[net, tr_info] = train(net, x, y);
testX = x(:, tr_info.testInd);
testT = y(:, tr_info.testInd);
testY = net(testX);
testIndices = vec2ind(testY);
[error_rate, conf_mat] = confusion(testT, testY);
fprintf('Percentage Correct Classification   : %f%%\n', 100*(1 - error_rate));
fprintf('Percentage Incorrect Classification : %f%%\n', 100*error_rate);
%%Manually calculate the output by hand
% nFeatures = 6
% nSamples = 200
% nHiddenNode = 10
% nClass = 2
% input layer => x      (6x200)
% hidden layer => h    = sigmoid(w1.x + b1)
%                      = (10x6)(6x200) + (10x1) 
%                      = (10x200) 
%
% output layer => yhat = w2.h + b2             
%                      = (2x200)  
w1 = net.IW{1};   % (10x6)
w2 = net.LW{2};   % (2x10)
b1 = net.b{1};    % (10x1)
b2 = net.b{2};    % (2x1)
h = sigmoid(w1*x + b1);
yhat = w2*h + b2;
[testY' yhat']
[vec2ind(testY)' vec2ind(yhat)']

0 Comments

Products

3 Answers

Link

Answer by JESUS DAVID ARIZA ROYETH on 2 May 2018

Accepted Answer

you missed several normalization parameters, here I leave the solution :

[x, y] = crab_dataset;
size(x)    % 6 x 200
size(y)    % 2 x 200
setdemorandstream(491218382);
net = patternnet(10);
[net, tr_info] = train(net, x, y);
xoffset=net.inputs{1}.processSettings{1}.xoffset;
gain=net.inputs{1}.processSettings{1}.gain;
ymin=net.inputs{1}.processSettings{1}.ymin;
w1 = net.IW{1};   % (10x6)
w2 = net.LW{2};   % (2x10)
b1 = net.b{1};    % (10x1)
b2 = net.b{2};
% Input 1
y1 = bsxfun(@times,bsxfun(@minus,x,xoffset),gain);
y1 = bsxfun(@plus,y1,ymin);
% Layer 1
a1 = 2 ./ (1 + exp(-2*(repmat(b1,1,size(x,2)) + w1*y1))) - 1;
% output
n=repmat(b2,1,size(x,2)) + w2*a1;
nmax = max(n,[],1);
n = bsxfun(@minus,n,nmax);
num = exp(n);
den = sum(num,1);
den(den == 0) = 1;
y2 = bsxfun(@rdivide,num,den);%y2==outputnet == net(x)

1 Comment

Jeff Chang on 2 May 2018

Thank you very much. I realize that the 3 things that I missed were:

I should normalize the input to [-1, 1]
I should use the tanh activation (instead of the sigmoid activation) on the hidden layer
I should apply softmax on the output layer

Link

Answer by Jeff Chang on 2 May 2018

Edited by Jeff Chang on 2 May 2018

Beside, is it possible to use deepDreamImage() to visualize the hidden layer in this example?

0 Comments

Link

Answer by Shounak Mitra on 8 Oct 2018

Unfortunately, using deepDreamImage() is not possible in this case.

You are now following this question

Jeff Chang (view profile)