Deep Learning: Training Network with "parallel" option using only CPUs

Hi,

I am trying to train a network using the follow parameters:

miniBatchSize = 10;
clear NewNetIn3D

valFrequency = floor(numel(imdsTrain)/miniBatchSize);

options = trainingOptions('sgdm', ...
    'MiniBatchSize',miniBatchSize, ...
    'MaxEpochs',6, ...
    'InitialLearnRate',1e-5,...
    'Shuffle','never',...
    'ExecutionEnvironment','parallel',...
    'Verbose',false, ...
    'Plots','training-progress');

net = trainNetwork(imdsTrain,LabelsTrain,LayersBMC,options);

Since my graphic card is not super, I am trying to run the code using multiple CPUs, but the parallel option always go with multiple GPUs and then crashes. Is there any way to restrict the paralel pool to use only CPUs? If I define the option 'cpu' it works, but with only one core.

Answer 1

[Joss Knight]

Even with a weak graphics card you will usually see better performance than on multiple CPUs. However, to try it out, after you start MATLAB, type

setenv CUDA_VISIBLE_DEVICES -1

[Brunno Machado de Campos]

Hi, thank you!

Your line worked perfectly.

I believe my problem with GPU is the dedicated memory. I am running with 800 3D MagRessoImages and when the trainning starts with GPUs it stops imediately with a memory related error.

[Joss Knight]

The correct solution is to reduce the MiniBatchSize training option until you have enough memory.

[shivan artosh]

Hello Mr. Knight

i need to use multiple CPU instead of hardware resource: single CPU in order to speed the network up, so do you have any suggestion for me?

thanks

[Joss Knight]

Yes, use 'ExecutionEnvironment', 'parallel' as the opening poster is doing.

[shivan artosh]

thanks sir

if i use 'ExecutionEnvironnment', 'parallel' it will help to speed the network up?

my computer features are: 8 GB HD graphic, core i7 8th generation

[Joss Knight]

Give it a go and see if it helps!

[shivan artosh]

i did it but i got this error:

Error using trainingOptions (line 285)
'ExecutionEnvironnment' is not an option for solver 'sgdm'.

[Joss Knight]

Looks like you have a typo since there are only three 'n's in Environment.

[shivan artosh]

so, how to solve this issue?

[Joss Knight]

Type ExecutionEnvironment instead of ExecutionEnvironnment.

[Joss Knight]

By the way it's my bad because my earlier comment had the typo.

[shivan artosh]

oh no problem sir

is there a way to showdown the 'parallel' command in the system, because after multiple CPU i again use single CPU but there is a little bit on my learning curve.

through the running on single CPU i found this on command window and it seems affected on my model:

IdleTimeout has been reached.
Parallel pool using the 'local' profile is shutting down.

[Joss Knight]

That's nothing. You had a bunch of MATLABs running to do your parallel training, you stopped using them, so eventually they were terminated.