📂Probability Theory

가우스 과정

Definition ¹

For every finite subset $S = \left\{ X_{t_{k}} \right\}_{k=1}^{n} \subset \left\{ X_{t} \right\}$ of a stochastic process $\left\{ X_{t} \right\}$, all linear combinations $S$ of the elements $$\sum_{k=1}^{n} a_{k} X_{t_{k}} \qquad , \left\{ a_{k} \right\}_{k=1}^{n} \subset \mathbb{R}$$ are said to follow a multivariate normal distribution. If $\left\{ X_{t} \right\}$ satisfies this condition, it is called a Gaussian process.

Explanation

To non-specialists, the definition might appear excessively mathematical, but intuitively it is not much different from a Wiener process. Of course, strictly speaking, according to the definition, a Wiener process is a Gaussian process, but the converse does not hold.

A geometric Brownian motion, for instance, does not constitute a Gaussian process because it follows a log-normal distribution at each time point.

Applications

In Bayesian contexts, it is used as the prior distribution itself. In machine learning, it is often utilized for generating random data, among other uses. For example, in the context of $\left[ x_{1} , x_{n} \right]$, a widely used method to create a Gaussian process is through forming a covariance matrix $\Sigma$ with a Gaussian kernel $k = k(u,v)$ and sampling from a multivariate normal distribution. $$\begin{align*} k \left( u, v \right) =& \exp \left( - {\frac{ 1 }{ 2 }} \left( u - v \right)^{2} \right) \\ \Sigma =& \begin{bmatrix} k \left( x_{1} , x_{1} \right) & k \left( x_{1} , x_{2} \right) & \cdots & k \left( x_{1} , x_{n} \right) \\ k \left( x_{2} , x_{1} \right) & k \left( x_{2} , x_{2} \right) & \cdots & k \left( x_{2} , x_{n} \right) \\ \vdots & \vdots & \ddots & \vdots \\ k \left( x_{n} , x_{1} \right) & k \left( x_{n} , x_{2} \right) & \cdots & k \left( x_{n} , x_{n} \right) \end{bmatrix} \end{align*}$$

The following is a sample code in Julia to demonstrate how to sample from a Gaussian process. The reason for adding a very small $\varepsilon I$ in the middle of $\Sigma$ is to prevent numerical errors.

using Distributions, LinearAlgebra, Plots

k(u, v) = exp(-abs2(u - v) / 2)
x = LinRange(-5, 5, 100)
Σ = [k(x[i],x[j],1) for i ∈ 1:100, j ∈ 1:100]
ϵ = 1e-6
Σ += ϵ*I
plot(x, [rand(MvNormal(zeros(100), Σ)) for _ in 1:10],
    legend = :none, title = "Gaussian Process", lw = 2, alpha = .5)

See Also

• Gaussian spatial process