Main Content

Stochastic Process Characteristics

What Is a Stochastic Process?

A time series yt is a collection of observations on a variable indexed sequentially over several time points t = 1, 2,...,T. Time series observations y1, y2,...,yT are inherently dependent. From a statistical modeling perspective, this means it is inappropriate to treat a time series as a random sample of independent observations.

The goal of statistical modeling is finding a compact representation of the data-generating process for your data. The statistical building block of econometric time series modeling is the stochastic process. Heuristically, a stochastic process is a joint probability distribution for a collection of random variables. By modeling the observed time series yt as a realization from a stochastic process y={yt;t=1,...,T}, it is possible to accommodate the high-dimensional and dependent nature of the data. The set of observation times T can be discrete or continuous. Figure 1-1, Monthly Average CO2 displays the monthly average CO2 concentration (ppm) recorded by the Mauna Loa Observatory in Hawaii from 1980 to 2012 [3].

Figure 1-1, Monthly Average CO2

Time series plot showing Monthly Average CO2 Concentration over Moana Loa with the years 1980 through 2012 on the x axis and CO2 Concentration in parts per million on the y axis.

Stationary Processes

Stochastic processes are weakly stationary or covariance stationary (or simply, stationary) if their first two moments are finite and constant over time. Specifically, if yt is a stationary stochastic process, then for all t:

  • E(yt) = μ < ∞.

  • V(yt) = σ2 < ∞.

  • Cov(yt, yt–h) = γh for all lags h0.

Does a plot of your stochastic process seem to increase or decrease without bound? The answer to this question indicates whether the stochastic process is stationary. “Yes” indicates that the stochastic process might be nonstationary. In Figure 1-1, Monthly Average CO2, the concentration of CO2 is increasing without bound which indicates a nonstationary stochastic process.

Linear Time Series Model

Wold’s theorem [2] states that you can write all weakly stationary stochastic processes in the general linear form

yt=μ+i=1ψiεti+εt.

Here, εt denotes a sequence of uncorrelated (but not necessarily independent) random variables from a well-defined probability distribution with mean zero. It is often called the innovation process because it captures all new information in the system at time t.

Unit Root Process

A linear time series model is a unit root process if the solution set to its characteristic equation contains a root that is on the unit circle (i.e., has an absolute value of one). Subsequently, the expected value, variance, or covariance of the elements of the stochastic process grows with time, and therefore is nonstationary. If your series has a unit root, then differencing it might make it stationary.

For example, consider the linear time series model yt=yt1+εt, where εt is a white noise sequence of innovations with variance σ2 (this is called the random walk). The characteristic equation of this model is z1=0, which has a root of one. If the initial observation y0 is fixed, then you can write the model as yt=y0+i=1tεi. Its expected value is y0, which is independent of time. However, the variance of the series is 2, which grows with time making the series unstable. Take the first difference to transform the series and the model becomes dt=ytyt1=εt. The characteristic equation for this series is z=0, so it does not have a unit root. Note that

  • E(dt)=0, which is independent of time,

  • V(dt)=σ2, which is independent of time, and

  • Cov(dt,dts)=0, which is independent of time for all integers 0 < s < t.

Figure 1-1, Monthly Average CO2 appears nonstationary. What happens if you plot the first difference dt = yt – yt–1 of this series? Figure 1-2, Monthly Difference in CO2 displays the dt. Ignoring the fluctuations, the stochastic process does not seem to increase or decrease in general. You can conclude that dt is stationary, and that yt is unit root nonstationary. For details, see Differencing.

Figure 1-2, Monthly Difference in CO2

Time series plot showing Monthly Difference of Average CO2 Concentration over Moana Loa with the years 1980 through 2012 on the x axis and Difference in CO2 Concentration in parts per million on the y axis.

Lag Operator Notation

The lag operator L operates on a time series yt such that Liyt=yti.

An mth-degree lag polynomial of coefficients b1, b2,...,bm is defined as

B(L)=(1+b1L+b2L2++bmLm).

In lag operator notation, you can write the general linear model using an infinite-degree polynomial ψ(L)=(1+ψ1L+ψ2L2+),

yt=μ+ψ(L)εt.

You cannot estimate a model that has an infinite-degree polynomial of coefficients with a finite amount of data. However, if ψ(L) is a rational polynomial (or approximately rational), you can write it (at least approximately) as the quotient of two finite-degree polynomials.

Define the q-degree polynomial θ(L)=(1+θ1L+θ2L2++θqLq) and the p-degree polynomial ϕ(L)=(1+ϕ1L+ϕ2L2++ϕpLp). If ψ(L) is rational, then

ψ(L)=θ(L)ϕ(L).

Thus, by Wold’s theorem, you can model (or closely approximate) every stationary stochastic process as

yt=μ+θ(L)ϕ(L)εt,

which has p + q coefficients (a finite number).

Characteristic Equation

A degree p characteristic polynomial of the linear time series model yt=ϕ1yt1+ϕ2yt2+...+ϕpytp+εt is

ϕ(a)=apϕ1ap1ϕ2ap2...ϕp.

It is another way to assess that a series is a stationary process. For example, the characteristic equation of yt=0.5yt10.02yt2+εt is ϕ(a)=a20.5a+0.02.

The roots of the homogeneous characteristic equation ϕ(a)=0 (called the characteristic roots) determine whether the linear time series is stationary. If every root in ϕ(a) lies inside the unit circle, then the process is stationary. Roots lie within the unit circle if they have an absolute value less than one. This is a unit root process if one or more roots lie inside the unit circle (i.e., have absolute value of one). Continuing the example, the characteristic roots of ϕ(a)=0 are a={0.4562,0.0438}. Since the absolute values of these roots are less than one, the linear time series model is stationary.

References

[1] Box, G. E. P., G. M. Jenkins, and G. C. Reinsel. Time Series Analysis: Forecasting and Control. 3rd ed. Englewood Cliffs, NJ: Prentice Hall, 1994.

[2] Wold, H. A Study in the Analysis of Stationary Time Series. Uppsala, Sweden: Almqvist & Wiksell, 1938.

[3] Tans, P., and R. Keeling. (2012, August). “Trends in Atmospheric Carbon Dioxide.” NOAA Research. Retrieved October 5, 2012 from https://gml.noaa.gov/ccgg/trends/mlo.html.

Related Examples

More About