Maximum Likelihood Estimation of regARIMA Models

Innovation Distribution

For regression models with ARIMA time series errors in Econometrics Toolbox™, ε_t = σz_t, where:

• ε_t is the innovation corresponding to observation t.

• σ is the constant variance of the innovations. You can set its value using the Variance property of a regARIMA model.

• z_t is the innovation distribution. You can set the distribution using the Distribution property of a regARIMA model. Specify either a standard Gaussian (the default) or standardized Student’s t with ν > 2 or NaN degrees of freedom.

  Note

  If ε_t has a Student’s t distribution, then

  $$z_t=T_{\nu }\sqrt{\frac{\nu -2}{\nu }},$$

  where T_ν is a Student’s t random variable with ν > 2 degrees of freedom. Subsequently, z_t is t-distributed with mean 0 and variance 1, but has the same kurtosis as T_ν. Therefore, ε_t is t-distributed with mean 0, variance σ, and has the same kurtosis as T_ν.

estimate builds and optimizes the likelihood objective function based on ε_t by:

1. Estimating c and β using MLR

2. Inferring the unconditional disturbances from the estimated regression model, $${\widehat{u}}_t=y_t-\widehat{c}-X_t\widehat{\beta }$$

3. Estimating the ARIMA error model, $${\widehat{u}}_t={{\rm H}}^{-1}(L){\rm N}(L){\epsilon }_t,$$ where H(L) is the compound autoregressive polynomial and N(L) is the compound moving average polynomial

4. Inferring the innovations from the ARIMA error model, $${\widehat{\epsilon }}_t={\widehat{{\rm H}}}^{-1}(L)\widehat{{\rm N}}(L){\widehat{u}}_t$$

5. Maximizing the loglikelihood objective function with respect to the free parameters

Note

If the unconditional disturbance process is nonstationary (i.e., the nonseasonal or seasonal integration degree is greater than 0), then the regression intercept, c, is not identifiable. estimate returns a NaN for c when it fits integrated models. For details, see Intercept Identifiability in Regression Models with ARIMA Errors.

estimate estimates all parameters in the regARIMA model set to NaN. estimate honors any equality constraints in the regARIMA model, i.e., estimate fixes the parameters at the values that you set during estimation.

Loglikelihood Functions

Given its history, the innovations are conditionally independent. Let H_t denote the history of the process available at time t, where t = 1,...,T. The likelihood function of the innovations is

$$f({\epsilon }_1,\mathrm{...},{\epsilon }_T\text{|}{H}_{T-1})={\displaystyle \prod }_{t=1}^{T}f({\epsilon }_t{\text{|H}}_{t-1}),$$

where f is the standard Gaussian or t probability density function.

The exact form of the loglikelihood objective function depends on the parametric form of the innovation distribution.

• If z_t is standard Gaussian, then the loglikelihood objective function is

  $$logL=-\frac{T}{2}\log (2\pi )-\frac{T}{2}\log {\sigma }^2-\frac{1}{2{\sigma }^2}{\displaystyle \sum _{t=1}^T{\epsilon }_t^2}.$$

• If z_t is a standardized Student’s t, then the loglikelihood objective function is

  $$logL=T\log \left[\frac{\Gamma \left(\frac{\nu +1}{2}\right)}{\Gamma \left(\frac{\nu }{2}\right)\sqrt{\pi (\nu -2)}}\right]-\frac{T}{2}{\sigma }^2-\frac{\nu +1}{2}{\displaystyle \sum _{t=1}^{T}l}og\left[1+\frac{{\epsilon }_t^2}{{\sigma }^2(\nu -2)}\right].$$

estimate performs covariance matrix estimation for maximum likelihood estimates using the outer product of gradients (OPG) method.