Predicted values are all the same in CNN regression

Hello, I am new to CNN, and I am trying to use regression using CNN on 1D data. My data has 105 samples, each with 458 data points. However, whenever I train my network and use that to predict values, all the prediction values turn out to be the same. I have tried using different solvers, filter sizes, or pooling layers, but none of it seems to produce any difference. I am including my code here:

XTrain = xlsread('1stderv_preprocessed_training_sample_spectra.xlsx');
height = 458;
width = 1;
channels = 1;
samples = 105;
CNN_TrainingData = reshape(XTrain,[height, width, channels, samples]);
YTrain = xlsread('trng ref data_caffeine-35x3_105 sample.xls');
CNN_TrainingLabels = YTrain;
layers = [
    imageInputLayer([height, width, channels])

    convolution2dLayer([5 1],100, 'stride',5)
    batchNormalizationLayer
    reluLayer
    
    averagePooling2dLayer([50 1],'Stride',2)
    
    dropoutLayer(0.3)
    fullyConnectedLayer(100)
    fullyConnectedLayer(50)
    fullyConnectedLayer(20)
    fullyConnectedLayer(5)
    fullyConnectedLayer(1)
    regressionLayer];

miniBatchSize  = 5;
validationFrequency = floor(numel(YTrain)/miniBatchSize);
options = trainingOptions('sgdm', ...
    'MiniBatchSize',miniBatchSize, ...
    'MaxEpochs',50, ...
    'InitialLearnRate',1e-4, ...
    'LearnRateSchedule','piecewise', ...
    'LearnRateDropFactor',0.1, ...
    'LearnRateDropPeriod',45);
            
net = trainNetwork(CNN_TrainingData,CNN_TrainingLabels,layers,options);
YPrediction = predict(net,CNN_TestingData);

Please help me out, I'm at my wits' end.