Monte Carlo Simulation of Regression Models with ARIMA Errors

What Is Monte Carlo Simulation?

Monte Carlo simulation is the process of generating independent, random draws from a specified probabilistic model. When simulating time series models, one draw (or realization) is an entire sample path of specified length N, y₁, y₂,...,y_N. When you generate a large number of draws, say M, you generate M sample paths, each of length N.

Note

Some extensions of Monte Carlo simulation rely on generating dependent random draws, such as Markov Chain Monte Carlo (MCMC). The simulate function in Econometrics Toolbox™ generates independent realizations.

Some applications of Monte Carlo simulation are:

• Demonstrating theoretical results

• Forecasting future events

• Estimating the probability of future events

Generate Monte Carlo Sample Paths

The time series portion of the model specifies the dynamic evolution of the unconditional disturbance process over time through a conditional mean structure. To perform Monte Carlo simulation of regression models with ARIMA errors:

1. Specify presample innovations or unconditional disturbances (or use default presample data).

2. Generate an uncorrelated innovation series from a probability distribution.

3. Filter the innovations through the ARIMA error model to obtain the simulated unconditional disturbances.

4. Use the regression model, predictor data, and simulated unconditional disturbances to obtain the responses.

For example, consider simulating N responses from the regression model with ARMA(2,1) errors:

$$\begin{array}{l}{y}_t=X_t\beta +u_t\\ u_t={\varphi }_1{u}_{t-1}+{\varphi }_2{u}_{t-2}+{\epsilon }_t+{\theta }_1{\epsilon }_{t-1},\end{array}$$

where ε_t is Gaussian with mean 0 and variance σ². Given presample unconditional disturbances (u₀ and u_–1) and innovations (ε₀), following these steps:

1. Generate N independent innovations from the Gaussian distribution:

   $$\{{\widehat{\epsilon }}_1,{\widehat{\epsilon }}_2,\mathrm{...},{\widehat{\epsilon }}_N\}.$$

2. Filter the innovations recursively to obtain the unconditional disturbances:

   1. $${\widehat{u}}_1={\varphi }_1{u}_0+{\varphi }_2{u}_{-1}+{\widehat{\epsilon }}_1+{\epsilon }_0$$

   2. $${\widehat{u}}_2={\varphi }_1{\widehat{u}}_1+{\varphi }_2{u}_0+{\widehat{\epsilon }}_2+{\widehat{\epsilon }}_1$$

   3. $${\widehat{u}}_3={\varphi }_1{\widehat{u}}_2+{\varphi }_2{\widehat{u}}_1+{\widehat{\epsilon }}_3+{\widehat{\epsilon }}_2$$

   4. ...

   5. $${\widehat{u}}_N={\varphi }_1{\widehat{u}}_{N-1}+{\varphi }_2{\widehat{u}}_{N-2}+{\widehat{\epsilon }}_N+{\widehat{\epsilon }}_{N-1}.$$

3. Obtain simulated responses using the unconditional disturbances, regression model, and the predictors:

   $${\widehat{y}}_t=X_t\beta +{\widehat{u}}_t.$$

Econometrics Toolbox automates this process with simulate. Pass in a fully specified regression model with ARIMA errors (regARIMA), the number of responses to simulate, and, optionally, the number of paths and presample data, and simulate simulates the responses.

Note

Econometrics Toolbox treats the predictors in the regression model as fixed, nonstochastic series. Therefore, in order to generate Monte Carlo sample paths of the response, you need to know the values of the predictors.

Monte Carlo Error

Using many simulated paths, you can estimate various features of the model. However, Monte Carlo estimation is based on a finite number of simulations. Therefore, Monte Carlo estimates are subject to some amount of error. You can reduce the amount of Monte Carlo error in your simulation study by increasing the number of sample paths, M, that you generate from your model.

For example, to estimate the probability of a future event:

1. Generate M sample paths from your model.

2. Estimate the probability of the future event using the sample proportion of the event occurrence across M simulations,

   $$\widehat{p}=\frac{\#timeseventoccursinMdraws}{M}.$$

3. Calculate the Monte Carlo standard error for the estimate,

   $$se=\sqrt{\frac{\widehat{p}(1-\widehat{p})}{M}}.$$

You can reduce the Monte Carlo error of the probability estimate by increasing the number of realizations. If you know the desired precision of your estimate, you can solve for the number of realizations needed to achieve that level of precision.