Hyperparameters, What Are They?
Terminology
Bayesian Statistics 1
In the Bayesian paradigm, the following is referred to as Bayesian Hierarchical Model:
- (1) It is assumed that data $y_{1} , \cdots , y_{n}$ is obtained according to parameters $\theta_{1} , \cdots , \theta_{n}$: $$y_{1} , \cdots , y_{n} | \theta_{1} , \cdots , \theta_{n} \sim p \left( y_{k} | \theta_{k} \right)$$
- (2) Parameters $\theta_{1} , \cdots , \theta_{n}$ themselves are considered to be obtained by Hyper parameter $\lambda$: $$\theta_{1} , \cdots , \theta_{n} | \lambda \sim \pi \left( \theta_{k} | \lambda \right)$$
- (3) $\lambda$ also has its own explaining distribution: $$\lambda \sim \pi (\lambda)$$
Machine Learning
In machine learning techniques, weights or biases related to results according to data are referred to as Parameters. Constants decided by the user in methods and algorithms to obtain these parameters are called Hyper parameters.
Description
Regardless of the context in which the phrase Hyper parameter is used, it is utilized to mean a parameter that determines Parameters.
See Also
Parameter
김달호. (2013). R과 WinBUGS를 이용한 베이지안 통계학 ↩︎