predict

R2026b

Predict responses using neighborhood component analysis (NCA) regression model

Syntax

ypred = predict(mdl,X)

Description

ypred = predict(mdl,X) computes the predicted response values, ypred, corresponding to rows of X, using the model mdl.

Examples

Tune NCA Model for Regression Using loss and predict

Load the sample data.

load robotarm.mat

The robotarm data set contains 7168 training observations and 1024 test observations with 32 features of a robot arm simulator [1][2]. The data set in this example is a preprocessed version of the original data set. The data set used in this example is a preprocessed version of the original data set. The data was preprocessed by subtracting values given by a linear regression fit from the original data, followed by normalization of all features to unit variance.

Perform neighborhood component analysis (NCA) feature selection for regression with the default $$\lambda$$ (regularization parameter) value.

rng(1,"twister") % For reproducibility 
nca = fsrnca(Xtrain,ytrain,FitMethod="exact",Solver="lbfgs");

Plot the selected values.

figure
plot(nca.FeatureWeights,"ro")
xlabel("Feature Index")
ylabel("Feature Weight")
grid on

More than half of the feature weights are nonzero. Use the test set as a measure of performance and compute the loss using the selected features.

L = loss(nca,Xtest,ytest)

L = 
0.0837

Compute the predicted response values for the test set and plot them versus the actual response.

ypred = predict(nca,Xtest);
figure
plot(ypred,ytest,"bo")
axis square
grid on
xlim([-4 4])
ylim([-4 4])
xlabel("Predicted Response")
ylabel("Actual Response")

A perfect fit versus the actual values forms a 45-degree straight line. In this plot, the predicted and actual response values seem to be scattered around this line.

Try improving the performance. Tune the regularization parameter $$\lambda$$ for feature selection using 5-fold cross-validation. Tuning $$\lambda$$ means finding the $$\lambda$$ value that produces the minimum regression loss.

First, partition the data into five folds. For each fold, cvpartition assigns 4/5th of the data as a training set, and 1/5th of the data as a test set.

n = length(ytrain);
cvp = cvpartition(length(ytrain),KFold=5);
numvalidsets = cvp.NumTestSets;

Assign the $$\lambda$$ values for the search. Multiplying response values by a constant increases the loss function term by a factor of the constant. Therefore, including the std(ytrain) factor in the $$\lambda$$ values balances the default loss function ("mad", mean absolute deviation) term and the regularization term in the objective function. In this example, the std(ytrain) factor is 1 because the loaded sample data is a preprocessed version of the original data set.

lambdavals = linspace(0,50,20)*std(ytrain)/n;

Create an array to store the loss values.

lossvals = zeros(length(lambdavals),numvalidsets);

Train the NCA model for each $$\lambda$$ value using the training set in each fold. Compute the regression loss for the corresponding test set in the fold using the NCA model, and record the loss value. Repeat this process for each $$\lambda$$ value and each fold.

for i = 1:length(lambdavals)
    for k = 1:numvalidsets
        X = Xtrain(cvp.training(k),:);
        y = ytrain(cvp.training(k),:);
        Xvalid = Xtrain(cvp.test(k),:);
        yvalid = ytrain(cvp.test(k),:);

        nca = fsrnca(X,y,FitMethod="exact", ...
             Solver="minibatch-lbfgs",Lambda=lambdavals(i), ...
             GradientTolerance=1e-4,IterationLimit=30);
        
        lossvals(i,k) = loss(nca,Xvalid,yvalid,LossFunction="mse");
    end
end

Compute the average loss obtained from the folds for each $$\lambda$$ value.

meanloss = mean(lossvals,2);

Plot the mean loss versus the $$\lambda$$ values.

figure
plot(lambdavals,meanloss,"ro-")
xlabel("Lambda")
ylabel("Loss (MSE)")
grid on

Find the $$\lambda$$ value that gives the minimum loss value.

[~,idx] = min(meanloss)

idx = 
17

bestlambda = lambdavals(idx)

bestlambda = 
0.0059

bestloss = meanloss(idx)

bestloss = 
0.0590

Fit the NCA feature selection model for regression using the best $$\lambda$$ value.

nca2 = fsrnca(Xtrain,ytrain,FitMethod="exact", ...
    Solver="lbfgs",Lambda=bestlambda);

Plot the selected features.

figure
plot(nca2.FeatureWeights,"ro")
xlabel("Feature Index")
ylabel("Feature Weight")
grid on

Most of the feature weights are zero. fsrnca identifies the four most relevant features.

Compute the loss for the test set.

L = loss(nca2,Xtest,ytest)

L = 
0.0571

Tuning the regularization parameter $$\lambda$$ eliminates more of the irrelevant features and improves the performance.

Plot the predicted versus the actual response values in the test set.

ypred = predict(nca2,Xtest);
figure
plot(ypred,ytest,"bo")
axis square
grid on
xlim([-4 4])
ylim([-4 4])
xlabel("Predicted Response")
ylabel("Actual Response")

The predicted response values seem to be closer to the actual values.

Input Arguments

mdl — Neighborhood component analysis model for regression

FeatureSelectionNCARegression object

Neighborhood component analysis model for regression, specified as a FeatureSelectionNCARegression object.

X — Predictor variable values

table | n-by-p matrix

Predictor variable values, specified as a table or an n-by-p matrix, where n is the number of observations and p is the number of predictor variables used to train mdl. By default, each row of X corresponds to one observation, and each column corresponds to one variable.

For a numeric matrix:

• The variables in the columns of X must have the same order as the predictor variables that trained mdl.

• If you train mdl using a table (for example, Tbl), and Tbl contains only numeric predictor variables, then X can be a numeric matrix. To treat numeric predictors in Tbl as categorical during training, identify categorical predictors by using the CategoricalPredictors name-value argument of fsrnca. If Tbl contains heterogeneous predictor variables (for example, numeric and categorical data types), and X is a numeric matrix, then predict throws an error.

For a table:

• X must contain all the predictors used to train the model.

• predict does not support multicolumn variables or cell arrays other than cell arrays of character vectors.

• If you train mdl using a table (for example, Tbl), then all predictor variables in X must have the same variable names and data types as the variables that trained mdl (stored in mdl.PredictorNames). However, the column order of X does not need to correspond to the column order of Tbl. Also, Tbl and X can contain additional variables (response variables, observation weights, and so on), but predict ignores them.

• If you train mdl using a numeric matrix, then the predictor names in mdl.PredictorNames must be the same as the corresponding predictor variable names in X. To specify predictor names during training, use the CategoricalPredictors name-value argument of fsrnca. All predictor variables in X must be numeric vectors. X can contain additional variables (response variables, observation weights, and so on), but predict ignores them.

Data Types: table | single | double

Output Arguments

ypred — Predicted response values

n-by-1 vector

Predicted response values, returned as an n-by-1 vector, where n is the number of observations.

Version History

Introduced in R2016b