how to merge two networks trained on different dataset ?

Hi

I am trainning LSTM network for time series prediction.

My training data size is 160 and number of channels are 19.

Training this data on single machine takes a lot of time. Therefore, divided the data set to train on different machine.

This way I can optimize training hyperparameters quickly rather than waiting for long time.

My question is: when I train LSTM network on different machines with different data set. Is there a way to merge these trained network.

If not, what is the ideal method to optimize the training time and process.

-Chetan

[Hiro Yoshino]

Hyperparameters are the parameters that cannot be trained through trainings.

Can you make this point a bit clearer?

To reduce the training time, the first thing you should try is parallelization but the models with LSTM are limited in that the processing cannot be parallelized along the time axis.

Here's the toolbox with which you can make things parallel: https://jp.mathworks.com/products/parallel-computing.html

Also, if you want to tune the hyperparameters in a pallalel way: https://jp.mathworks.com/help/deeplearning/ref/experimentmanager-app.html

This app might be a help for you.

[Chetan Thakur]

@Hiro Yoshino

Thanks for the comment.

I do not wish to train the hyper parameter. My question was how do I train the LSTM network on two different machines on datasets(e.g. divided in half) and then combine these two trained network into one as if the network is trained on entire dataset.

I asked this because my machine is not powerfull enough and I want to take advantage of other machines with GPU to train my network.

Each machine has only one GPU.

As you said I do use parallel computing toolbox and experiment manager. But this does not speed up the training processes.

I hope you understand my requirement. If you need additional information please feel free to ask.

Looking forward to hear from you.

-Chetan

[Hiro Yoshino]

@Chetan Thakur

NN is just a big chunk of network parameters where linear and non-linear calculations take place. So if you want to, you can add the value together and devid it by the number of models but I wonder if this "network" works as you expect.

This is an example how to read the parameter values:

net.Layers(2).Bias