How to write objective function for Bayesian Optimization (bayesopt)?

Dear all,

I am using `bayesopt` method and I need to provide an objective function.

I have studied the https://uk.mathworks.com/help/stats/bayesian-optimization-objective-functions.html on how to write an objective function.

Where can I find more example of objective function in MATLAB?

Can anyone guide me how to write objective function for Bayesian Optimization?

Thank you

Answer 1

[Alan Weiss]

There is an example here:

https://www.mathworks.com/help/stats/bayesian-optimization-plot-functions.html#d118e31663

The link you provided really has the information you need to write an objective function. Your objective takes the table that MATLAB passes, and returns a real scalar value as the objective. The fields in the table are the optimization variables. In the example, the optimization variables are box and sigma, and the function works with x.box and x.sigma.

If you want to provide pseudocode for your objective, maybe we can help you turn it into the correct form.

Alan Weiss

MATLAB mathematical toolbox documentation

[Eric]

Hi, thank you for the reply.

I have prepared some codes here. I aim to apply cvpartition from k fold loss in bayesopt objective function.

I have created CV partition using 5 fold loss where t is the target.

k = 5;
cv = cvpartition(t, 'KFold', k, 'Stratify', true);

Then I have called bayesopt with my objective function called MyObjFunc.

function rootMSE = JoeBestObjFunc(x, t, cv, numHiddenLayers, numHiddenNeurons, trainFcn, transferFcn, lr, momentum)

% 2. Defines the network architecture and training options.
hiddenLayerSize = numHiddenNeurons * ones(1, numHiddenLayers); 
net = patternnet(hiddenLayerSize, char(trainFcn));
net.trainParam.epochs = 500;
net.trainParam.max_fail = 6; 
% Update other properties based on the training functions

% Update Activation Function of Layers
for i = 1:numHiddenLayers 
    net.layers{i}.transferFcn = char(transferFcn); 
end

%% Train Network, Save the Error
% create an array with empty rootMSE error
rootMSEs = zeros(cv.NumTestSets, 1);

for i = 1 : cv.NumTestSets
   %% 3. Trains and validates the network.
   trainedNet = train(net, x(:, cv.training(i)), t(:,cv.training(i))); 
   %% Evaluate on validation set and compute Classification Error
   % TODO how to count the error
end

%% finally count mean value of rootMSE
rootMSE = mean(rootMSEs);

end

Inside my objective function above, I have passed the cv variable for training the neural network.

I am not sure how to apply the cv variable in my objective function above.

1) Is my objective function layout correct?

2) How do I train, save and count the validation error from 5 different subsets?

I am not sure how to return a single validation error from 5 different subsets.

%% Train Network, Save the Error
% create an array with empty rootMSE error
rootMSEs = zeros(cv.NumTestSets, 1);

for i = 1 : cv.NumTestSets
   %% 3. Trains and validates the network.
   trainedNet = train(net, x(:, cv.training(i)), t(:,cv.training(i))); 
   %% Evaluate on validation set and compute Classification Error
   % TODO how to count the error
end

%% finally count mean value of rootMSE
rootMSE = mean(rootMSEs);

Thank you very much for your time, patience & knowledge.

[Alan Weiss]

I recently learned that there is an existing documentation example on this subject:

https://www.mathworks.com/help/deeplearning/examples/deep-learning-using-bayesian-optimization.html

The following example shows exactly how to write the objective function for cross-validation loss:

https://www.mathworks.com/help/stats/bayesian-optimization-case-study.html

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation