Solving ODE using Deep Learning

Hi all,

I am trying to understand how to solve ODE using Deep Learning, and code it using MATLAB, based on this tutorial:

https://www.mathworks.com/help/deeplearning/ug/solve-odes-using-a-neural-network.html

When I modified the code to solve a Lotka-Volterra model:

https://www.mathworks.com/help/matlab/math/numerical-integration-of-differential-equations.html

I could not get the loss to converge. I think it is because the tutorial uses sgdmupdate optimizer. If I want to change it to adam optimizer, how can I change the code?

Answer 1

[Antoni Woss]

To use the adam optimizer in this custom training loop example, you can follow the example set out in the documentation page for the adamupdate function - https://uk.mathworks.com/help/deeplearning/ref/adamupdate.html.

Note that the adamupdate function has some different required input arguments and return arguments so you will need to map the differences to the ODE example you are trying to solve. For example, initializing empty averageGrad and averageSqGrad outside the custom training loop so that you can update it at each call to adamupdate. Here is a snippet just showing where these quantites would be used.

averageGrad = [];
averageSqGrad = [];
...
[net,averageGrad,averageSqGrad] = adamupdate(net,gradients,averageGrad,averageSqGrad,iteration);