Why RMSE doens't decrease in deeplearning toolbox?

Hi, I'm stduying Matlab deep learning toolbox, and confused about using regression layer.

I want to make regression model for 8 input and 1 output. Input and output are all feature.

How can I make model which RMSE goes dereasing?

Should I control hyperparameter to fix this situation?

Plz help my project. Thanks.

(data file is uploaded)

clc;clear;
filename = "training_set2.csv";
tbl = readtable(filename,'TextType','string');
head(tbl)

edges = 0.41:0.01:0.54; %discretize data
responses=cell2mat(table2cell(tbl(:,"FM")));
tbl(:,"FM") = cell2table(num2cell(discretize(responses,edges)));

labelName = "FM";
numObservations = size(tbl,1);
numObservationsTrain = floor(0.8*numObservations);
numObservationsValidation = floor(0.15*numObservations);
numObservationsTest = numObservations - numObservationsTrain - numObservationsValidation;



idx = randperm(numObservations);
idxTrain = idx(1:numObservationsTrain);
idxValidation = idx(numObservationsTrain+1:numObservationsTrain+numObservationsValidation);
idxTest = idx(numObservationsTrain+numObservationsValidation+1:end);

tblTrain = cell2mat(table2cell(tbl(idxTrain,1:9)));
responseTrain = cell2mat(table2cell(tbl(idxTrain,10)));
tblValidation = cell2mat(table2cell(tbl(idxValidation,1:9)));
responseValidation = cell2mat(table2cell(tbl(idxValidation,10)));
tblTest = cell2mat(table2cell(tbl(idxTest,1:9)));
responseTest = cell2mat(table2cell(tbl(idxTest,10)));

numFeatures  = size(tbl,2) - 1;
numClasses = 1;



layers = [
    featureInputLayer(numFeatures)
    fullyConnectedLayer(50)
    batchNormalizationLayer
    reluLayer
    fullyConnectedLayer(numClasses)
    softmaxLayer
    regressionLayer]; 

miniBatchSize = 1;

options = trainingOptions('adam', ...
    'InitialLearnRate',0.0001, ...
    'MiniBatchSize',miniBatchSize, ...
    'MaxEpochs',50,...
    'Shuffle','every-epoch', ...
    'Plots','training-progress', ...
    'ValidationData',{tblValidation,responseValidation}, ...
    'Verbose',false);


net = trainNetwork(tblTrain,responseTrain,layers,options);

Answer 1

[Angelo Yeo]

The training environment is not ideal to use deep neural networks. A few comments:

1. Why did you discretize the output, "FM", and use softmaxLayer and regressionLayer at the same time? There is no point to discretize "FM" but to use "regressionLayer".
2. The number of samples is too small. The total number of data is 152. Which is too small to train a neural network.
3. Also, it is hard to distinguish between the samples. Most of features are identical or correlated.

I want to recommend you take the following courses for a better understanding for theories behind Deep Learning.

Deep Learning Onramp | Self-Paced Online Courses - MATLAB & Simulink (mathworks.com)

Deep Learning with MATLAB | Self-Paced Online Courses - MATLAB & Simulink (mathworks.com)