model non-autonomous system with neural ode

Hi!

The Neural ODE method models the dynamical equation. While Mathworks provides many related examples, I have noticed that these cases seem to focus only on modeling autonomous systems. How should one model a non-autonomous system? In this case, the neural computation timesteps[t0,t1] would no longer be fixed, but it appears that this parameter cannot be modified.For example, define the differential equation to be:

At this time, the selected time steps during training should be [t01,t11],[t02,t12],[t03,t13],...

So is there any other possible approach to solve this problem?

Thanks a lot!

Answer 1

[Torsten]

You can easily convert your non-autonomous system into an autonomous one by defining an additional ODE y2 as

dy/dt = y2,    y(0) = y0
dy2/dt = 1,    y2(0) = 0  

[k]

I got it,thanks for your detailed explanation.

[k]

Hi, Torsten!@Torsten

Following your advice, I tested the differential equation, and it worked effectively. However, when I attempted more complex dynamical forms, I found that the network training did not perform well—the loss failed to decrease after a period of training. I have tried to adjust the number of neurons and learning rate , but it didn't yield improvements. Could you provide some suggestions for improvement or adjustments? Thank you very much!

The new dynamical equation was and .The analytical solution to this ordinary differential equation should be:

I convert the original non-autonomous system into an autonomous one:

The code I tested is as follows，

clear;
clc;
% https://uk.mathworks.com/help/deeplearning/ug/dynamical-system-modeling-using-neural-ode.html
% using neuralODELayer
% Generate training data
x0 = [-2; -8];
trueModel = @(t,y) [1;3*y(1)^2];
numTimeSteps = 2000;
T = 4;
odeOptions = odeset(RelTol=1.e-7);
t = linspace(0, T, numTimeSteps);
[~, xTrain] = ode45(trueModel, t, x0, odeOptions);
plot(xTrain(:,1),xTrain(:,2));
xTrain = xTrain';
% Rearrange the single solution [x(1),x(2),...,x(end)] into subsequences 
% x(t) and [x(t+1), ..., x(t+40)].
neuralOdeTimesteps = 40;
dt = t(2);
% timesteps = (0:neuralOdeTimesteps)*dt;
timesteps = t(1:41);
input = xTrain(:,1:end-neuralOdeTimesteps);
numObs = numTimeSteps - neuralOdeTimesteps;
targets = cell(numObs,  1);
for i = 1:numObs
    targets{i} = xTrain(:,i + (1:neuralOdeTimesteps));
end
% Design neural ODE.
stateSize = size(xTrain,1);
hiddenSize = 128;
odeNet = [
    fullyConnectedLayer(hiddenSize)
    tanhLayer
    fullyConnectedLayer(stateSize)];
odeNet = dlnetwork(odeNet,Initialize=false);
odeLayer = neuralODELayer(odeNet,timesteps,GradientMode="adjoint");
net = dlnetwork([featureInputLayer(stateSize); odeLayer]);
% Train neural ODE.
opts = trainingOptions("adam", ...
    Plots="training-progress", ...
    ExecutionEnvironment="auto", ...
    InputDataFormats="CB",...
    TargetDataFormats="CTB");
trainednet = trainnet(input,targets,net,"l2loss",opts);
odeNet = trainednet.Layers(2).Network;
xPred1 = dlode45(odeNet, t, dlarray(x0,"CB"));
plot(xPred1(1,:),xPred1(2,:),xTrain(1,:),xTrain(2,:));

[Torsten]

Sorry, but I have no experience with training neural networks - you could open a new question.

[k]

That's fine.Thanks for your reply.