Main Content

idGaussianProcess

Gaussian process regression mapping function for nonlinear ARX models (requires Statistics and Machine Learning Toolbox)

Description

An idGaussianProcess object implements a Gaussian process regression model, and is a nonlinear mapping function for estimating nonlinear ARX models. This mapping object, which is also referred to as a nonlinearity, incorporates RegressionGP (Statistics and Machine Learning Toolbox) objects that the mapping function creates using Statistics and Machine Learning Toolbox™. The mapping object contains three components: a linear component that uses a combination of linear weights, an offset, and a nonlinear component.

Diagram of an idGaussianProcess object, with an offset, nonlinear function, and linear function contributing to an output.

Mathematically, idGaussianProcess is a function that maps m inputs X(t) = [x(t1),x2(t),…,xm(t)]T to a scalar output y(t) using the following relationship:

y(t)=y0+(Χ(t)X¯)TPL+G(Χ(t),θ))

Here,

  • X(t) is an m-by-1 vector of inputs, or regressors, with mean Χ¯.

  • y0 is the output offset, a scalar.

  • P is an m-by-p projection matrix, where m is the number of regressors and p is the number of linear weights. m must be greater than or equal to p.

  • L is a p-by-1 vector of weights.

  • G(X,θ) is the regressive Gaussian process that constitutes the nonlinear component of the idGaussianProcess object. G has a mean of zero and a covariance that the user specifies by choosing a kernel, and can be expressed generally as

    G(X)=GP(0,K(Xtest,Xtrain,θ))

The Gaussian process G can be defined more precisely as the predicted value of a test input for a given value of θ. In this case, the relationship becomes:

G(X)=K(Xtest,Xtrain)[K(Xtrain,Xtrain)+σn2I]1Ytrain

Here:

  • K(Xtest,Xtrain) is the covariance kernel function.

  • Xtrain is a matrix representing the set of training inputs.

  • Xtest is a matrix representing the set of test inputs.

  • Ytrain is the vector of outputs from the training set.

  • σn is the standard deviation of the additive measurement noise.

For more information about creating Gaussian process regression models, see fitrgp (Statistics and Machine Learning Toolbox).

Use idGaussianProcess as the value of the OutputFcn property of an idnlarx model. For example, specify idGaussianProcess when you estimate an idnlarx model with the following command.

sys = nlarx(data,regressors,idGaussianProcess)
When nlarx estimates the model, it essentially estimates the parameters of the idGaussianProcess function.

You can configure the idGaussianProcess function to disable components and fix parameters. To omit the linear component, set LinearFcn.Use to false. To omit the offset, set Offset.Use to false. To specify known values for the linear function and the offset, set their Value attributes directly and set the corresponding Free attributes to False. To modify the estimation options, set the option property in EstimationOptions. For example, to change the fit method to 'exact', use G.EstimationOptions.FitMethod = 'exact'. Use evaluate to compute the output of the function for a given vector of inputs.

Creation

Description

G = idGaussianProcess creates an idGaussianProcess object G with the kernel function 'SquaredExponential' and default kernel parameters. The number of inputs is determined during model estimation and the number of outputs is 1.

example

G = idGaussianProcess(kernel) specifies a specific kernel.

G = idGaussianProcess(kernel,kernelParameters) initializes the parameters of the specified kernel to the values in kernelParameters.

G = idGaussianProcess(kernel,kernelParameters,UseLinearFcn) specifies whether the function uses a linear function as a subcomponent.

G = idGaussianProcess(kernel,kernelParameters,UseLinearFcn,UseOffset) specifies whether the function uses an offset term y0 parameter.

Input Arguments

expand all

Kernel covariance function, specified as character array or string. For information about the options, see Kernel (Covariance) Function in fitrgp (Statistics and Machine Learning Toolbox).

This argument sets the G.NonlinearFcn.KernelFunction property.

Initial values for the kernel parameters, specified as a vector. The size of the vector and the values depend on the choice of kernel. For more information, see Kernel Parameters in fitrgp (Statistics and Machine Learning Toolbox).

This argument sets the G.NonlinearFcn.Parameters property.

Option to use the linear function subcomponent, specified as true or false. This argument sets the value of the G.LinearFcn.Use property.

Option to use an offset term, specified as true or false. This argument sets the value of the G.Offset.Use property.

Properties

expand all

Input signal information for signals used for estimation, specified as vectors of m property-specific values, where m is the number of input signals. The Input properties for each input signal are as follows:

  • Name — Names of the input signals, specified as a 1-by-m string or character array, where m is the number of inputs

  • Mean — Mean of the input signals, specified as a numeric scalar

  • Range — Ranges of the input signals, specified as a 2-by-m numeric array that contains the minimum and maximum values

Output signal information, specified as property-specific values. The Output properties are as follows:

  • Name — Name of the output signal, specified as a string or a character array

  • Mean — Mean of the output signal, specified as a numeric scalar

  • Range — Range of the output signal, specified as a 2-by-1 numeric array that contains the minimum and maximum values

Parameters of the linear function, specified as follows:

  • Use — Option to use the linear function in the idGaussianProduct model, specified as a scalar logical. The default value is true.

  • Value — Linear weights that compose L', specified as a 1-by-p vector.

  • InputProjection — Input projection matrix P, specified as an m-by-p matrix, that transforms the detrended input vector of length m into a vector of length p.

  • Free — Option to update entries of Value during estimation, specified as a 1-by-p logical vector. The software honors the Free specification only if the starting value of Value is finite. The default value is true.

  • Minimum — Minimum bound on Value, specified as a 1-by-p vector. If Minimum is specified with a finite value and the starting value of Value is finite, then the software enforces that minimum bound during model estimation.

  • Maximum — Maximum bound on Value, specified as a 1-by-p vector. If Maximum is specified with a finite value and the starting value of Value is finite, then the software enforces that maximum bound during model estimation.

  • SelectedInputIndex — Indices of idGaussianProcess inputs (see Input.Name) that are used as inputs to the linear function, specified as an 1-by-nr integer vector, where nr is the number of inputs. For nonlinear ARX models, the RegressorUsage property determines these indices..

Parameters of the offset term, specified as follows:

  • Use — Option to use the offset in the idGaussianProcess model, specified as a scalar logical. The default value is true.

  • Value — Offset value, specified as a scalar.

  • Free — Option to update Value during estimation, specified as a scalar logical. The software honors the Free specification of false only if the value of Value is finite. The default value is true.

  • Minimum — Minimum bound on Value, specified as a numeric scalar or –Inf. If Minimum is specified with a finite value and the value of Value is finite, then the software enforces that minimum bound during model estimation. The default value is -Inf.

  • Maximum — Maximum bound on Value, specified as a numeric scalar or Inf. If Maximum is specified with a finite value and the starting value of Value is finite, then the software enforces that maximum bound during model estimation. The default value is Inf.

Parameters of the nonlinear function, specified as follows:

  • KernelFunction — Kernel covariance kernel function, specified as one of the values listed in the kernel argument description. For more information about these options, see Kernel (Covariance) Function in fitrgp (Statistics and Machine Learning Toolbox).

  • Parameters — Parameters of the idGaussianProcess kernel function, specified as a vector. The size of the vector and the values depend on the choice of KernelFunction. For more information, see Kernel Parameters in fitrgp (Statistics and Machine Learning Toolbox).

  • Free — Option to estimate parameters, specified as a logical scalar. If all the parameters have finite values, such as when the idGaussianProcess object corresponds to a previously estimated model, then setting Free to false causes the parameters of the nonlinear function S(X) to remain unchanged during estimation. The default value is true.

  • SelectedInputIndex — Indices of idGaussianProcess inputs (see Input.Name) that are used as inputs to the nonlinear function, specified as an 1-by-nr integer vector, where nr is the number of inputs. For nonlinear ARX models, the RegressorUsage property determines these indices.

Estimation options for the nonlinear block of the idGaussianProcess model, specified as follows. For more information on any of these options, see fitrgp (Statistics and Machine Learning Toolbox).

  • FitMethod — Method to use for estimating the parameters of the idGaussianProcess nonlinear model, specified as one of the items in the following table.

    Option Description
    'auto'

    Software selects the method automatically (default)

    'exact'

    Exact Gaussian process regression

    'sd'

    Subset of data points approximation

    'sr'

    Subset of regressors approximation

    'fic'

    Fully independent conditional approximation

  • ActiveSetMethod — Active set selection method, specified as one of the items in the following table.

    Option Description
    'random'

    Random selection (default)

    'sgma'

    Sparse greedy matrix approximation

    'entropy'Differential entropy-based selection
    'likelihood'

    Subset of regressors log likelihood-based selection

  • SparseFitRegularization — Regularization standard deviation for the sparse methods subset of regressors ('sr') and the fully independent conditional approximation ('fic'), specified as a positive scalar value.

  • Optimizer — Optimizer to use for parameter estimation, specified as one of the items in the following table.

    Option Description
    'quasinewton'Dense, symmetric rank-1-based, quasi-Newton approximation to the Hessian (default)
    'lbfgs'LBFGS-based quasi-Newton approximation to the Hessian
    'fminsearch'Unconstrained nonlinear optimization using the simplex search method of Lagarias et al. [see fitrgp (Statistics and Machine Learning Toolbox)]
    'fminunc'Unconstrained nonlinear optimization (requires an Optimization Toolbox™ license)
    'fmincon'Constrained nonlinear optimization (requires an Optimization Toolbox license)
  • OptimizerOptions — Options for the optimizer, specified as a structure or object. When Optimizer is set or changed, the software automatically updates the value of OptimizerOptions to match the defaults for the corresponding optimizer. Use the properties of the OptimizerOptions option set to change the values from their defaults.

Examples

collapse all

Load the input/output data from twotankdata and construct an iddata object z.

load twotankdata u y
z = iddata(y,u,0.8,'timeunit','hours');

Create an idGaussianProduct mapping object g that uses a Matern kernel with the parameter 3/2.

g = idGaussianProcess('Matern32');

Estimate a nonlinear ARX model that uses g as the output function.

sys = nlarx(z,[4 4 1],g)
sys = 
Nonlinear ARX model with 1 output and 1 input
  Inputs: u1
  Outputs: y1

Regressors:
  Linear regressors in variables y1, u1
  List of all regressors

Output function: Gaussian process function using a Matern32 kernel.
Sample time: 0.8 hours

Status:                                          
Estimated using NLARX on time domain data "z".   
Fit to estimation data: 97.08% (prediction focus)
FPE: 2.945e-05, MSE: 2.92e-05

Display the postestimation properties of g.

disp(sys.OutputFcn.Input)
Function inputs

     Name: {1x8 cell}
     Mean: [0.2700 0.2699 0.2697 0.2696 6.4286 6.4286 6.4286 6.4286]
    Range: [2x8 double]
disp(sys.outputFcn.Offset)
Output Offset: initialized to 0.27
      Use: 1
    Value: 0.2702
     Free: 1
disp(sys.outputFcn.NonlinearFcn)
GP kernel and its parameters

        KernelFunction: 'Matern32'
            Parameters: '<Kernel parameters>'
                  Free: 1
    SelectedInputIndex: [1 2 3 4 5 6 7 8]

Compare the output of sys with the measured output z.

compare(z,sys)

Figure contains an axes object. The axes object contains 2 objects of type line. These objects represent z (y1), sys: 92.49%.

The nonlinear model shows a good fit to the estimation data.

See Also

| (Statistics and Machine Learning Toolbox) | (Statistics and Machine Learning Toolbox) | | | | | | | |

Topics

Introduced in R2021b