Predict sequential data using a trained deep learning neural network

Hello,

ich have multiple time series data, e.g

X1 = [300 x 3] with labels Y1 = [300 x 1] (300 data points and 3 features)

X2 = [350 x 3] with labels Y2 = [350 x 1]

X3 = [200 x 3] with labels Y3 = [200 x 1]

I have trained a neural network (1d-CNN) and now want to use "predict" to predict the labels of these time series (X1, X2, X3).

https://de.mathworks.com/help/deeplearning/ref/seriesnetwork.predict.html#namevaluepairarguments

However, the "MiniBatchSize"-parameter iritates me.

"When making predictions with sequences of different lengths, the mini-batch size can impact the amount of padding added to the input data which can result in different predicted values. Try using different values to see which works best with your network. To specify mini-batch size and padding options, use the 'MiniBatchSize' and 'SequenceLength' options, respectively."

What is done by this padding? I mean, I just want to predict the labels to the inputs. If i got it right, if I select mini-batch size = 2, MATLAB select two of the time series (e.g. X1 and X2) and extend the shorter one to the length of the larger one (here X1_new = [350 x 3]. Then it calculates the error. But why is this done? Also strange is, that the predicted output for X1 is [300 x 1]. So why the extention to 350 is needed?

Kind regards

Paul

Answer 1

[Aneela]

Hi @Paul Muster

Neural networks require inputs of uniform size within a mini-batch to efficiently process the data in parallel. "Padding" ensures that all sequences in a mini-batch have the same length.

• The shorter sequence in a mini-batch is padded with zeros(or any specified value) to match the length of the longest sequence in that batch.
• Afte predictions are made, the padding is ignored, the network is designed to output predictions that correspond to original sequence lengths.
• Therfore, you still receive a [300X1] prediction for "X1" example.

Different mini-batch sizes can lead to variations in numerical precision and stability, which affect the final predicted values. The amount of padding applied can vary with differnent mini-batch sizes, which influence the network's predictions due to differences in how much extra data the network sees during processing.

Refer to the following MathWorks documentation for more information on "MiniBatchSize":

https://www.mathworks.com/help/deeplearning/ref/trainingoptions.html?searchHighlight=mini%20batch%20size&s_tid=srchtitle_support_results_9_mini%20batch%20size#:~:text=Size%20of%20the,%22rmsprop%22).