Image Classification Returning Different Results on Different Computers

2 views (last 30 days)
Sofia Brown
Sofia Brown on 27 Mar 2021
Commented: Sofia Brown on 5 Sep 2021
I am running a neural network for image classification using the Matlab deep learning toolbox. The algorithm contains three instances of convolution2dLayer, and it trains using TrainNetwork. If I run the exact same algorithm (identical code) on a different computer, my algorithm returns vastly different results: one computer yields a very high accuracy, around 99%, while the other never learns at all (the learning curve never increases above 50%). The only difference between the two computers is the Matlab version. The 99% accurate computer is on Matlab 2019a (Deep Learning Toolbox, version 12.1), while the 50% accurate computer is on Matlab 2019b (Deep Learning Toolbox, version 13.0). Is it expected that these two versions, which are so close together, would return such different results? I also find it surprising that the older version is better, but as machine learning is a blackbox, perhaps this is entirely possible? Are there any other reasons the results could be so different?

Answers (1)

Mahesh Taparia
Mahesh Taparia on 30 Mar 2021
This is an unexpected behaviour. Check if all the data preprocessing steps are same, also the training parameters, number of epochs in both the network, weight initialization and other hyperparameters should be same. If the problem still remains, then share your code and relevant information, it will be helpful.
Sofia Brown
Sofia Brown on 5 Sep 2021
Thank you very much for this response. I am sorry for my months long delay in writing back - I took a break from this project for the summer. I hope you are still able to be of assistance.
So, if I understand correctly, I just need to initialize a random seed value at the beginning of my code, like this (Using, say, 0 as the random seed):
rootFolder = 'TrainingAll5Sets';
categories = {'0deg', 'eighthdeg'};
%imds = imageDatastore(fullfile(rootFolder, categories), 'LabelSource', 'foldernames');
imds = imageDatastore(fullfile(rootFolder, categories), 'LabelSource', 'foldernames','FileExtensions','.png');
%Define Layers
layers = [
imageInputLayer([256 320 1])
%Set training options - use default options from 7.15.20
options = trainingOptions('sgdm', ...
'InitialLearnRate',0.00001, ...
'MaxEpochs',300, ...
'Shuffle','every-epoch', ...
'Verbose',false, ...
[net, info] = trainNetwork(imds, layers, options);
I have not changed any parts of the code except line 4. By just making this small change, will it generate the same random weights during each run, as you mentioned? I'm not sure I completetly understand how rng works in relation to the rest of the code and if I need to incorporate it anywhere else.
Thank you!

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by