Bayesian neural network for nonlinear model identification

Hello everyone,

I’m interested in using Bayesian neural networks to model a dynamical system, similar to the examples presented here: https://www.mathworks.com/help/ident/nonlinear-model-identification.html.

So far, the only relevant resource I’ve found is this page: https://www.mathworks.com/help/deeplearning/ug/train-bayesian-neural-network.html, which demonstrates how to train a Bayesian neural network to predict the rotation of handwritten digits.

Would the methodology and functions shown in that example be suitable for my case? Specifically, can they be adapted to train a model using time series data—i.e., sequences of input and output over time?

Thank you in advance for your support!

Marco

[Mathieu NOE]

hello

I would say probably yes but it would be nice if you could share some data

[Marco Pizzoli]

Hi Mathieu,

Thank you first of all for such a quick response.

Actually, I don't have any data of my own available.

However, to clarify what I would like to do, we can consider the data in the example given at https://www.mathworks.com/help/ident/ug/nonlinear-arx-models-with-custom-regressors.html. Thus, the input and output data that can be loaded with

load icEngine

According to the example:

• The file icEngine.mat contains one data set with 1500 input-output samples collected at the a sampling rate of 0.04 seconds. The input u(t) is the voltage [V] controlling a By-Pass Idle Air Valve (BPAV), and the output y(t) is the engine speed [RPM/100].

So, what I would like to do would be to identify a Bayesian neural network model capable of estimating and simulating the output y(t) (engine speed) based on the input u(t) (voltage).

Thank you in advance!

Answer 1

[Gayathri]

Hi @Marco Pizzoli,

Yes, the methodology used to train a Bayesian Neural Network (BNN) for predicting the rotation of handwritten digits can be adapted for modeling a dynamical system with time series data.

Time series data is inherently sequential and often has dependencies across time steps. The BNN example might not be optimized for capturing these dependencies and might not be suitable for time-dependent patterns. You may need to incorporate specialized layers or architectures, such as Recurrent Neural Networks (RNNs) or Long Short-Term Memory networks (LSTMs), to better handle sequential data.

You can also train a LSTM with Bayesion Optimization which might help you achieve your end goal. Please refer to this link, which will help you chose training configurations for LSTM using Bayesian Optimization:

https://www.mathworks.com/help/deeplearning/ug/experiment-with-training-configurations-for-sequence-regression.html

[Marco Pizzoli]

Hi Gayathri,

Thank you first of all for your response.

Regarding your suggestions, I am already aware of the methodology presented on the link page you shared with me. It is very interesting, but I don't think it is for me. What I would like to implement, would be an LSTM network that is able to estimate/simulate time histories, while also providing the uncertainty of the predictions.

To put it simply, I would like to train a network with a similar architecture:

Networklayers = [sequenceInputLayer(featureDimension) ...
                 lstmLayer(numHiddenUnits) ...
                 bayesFullyConnectedLayer(numResponses, Sigma1 = sigma1, Sigma2 = sigma2)];

Would that make sense?

Thank you again and best regards,

Marco

[Gayathri]

Yes, this should likely work.

[Marco Pizzoli]

I will try it and let you know as soon as I have some results.

Thank you :)