📂Data Science

Hyperparameters, What Are They?

Terminology

Bayesian Statistics ¹

In the Bayesian paradigm, the following is referred to as Bayesian Hierarchical Model:

• (1) It is assumed that data $y_{1} , \cdots , y_{n}$ is obtained according to parameters $\theta_{1} , \cdots , \theta_{n}$: $$y_{1} , \cdots , y_{n} | \theta_{1} , \cdots , \theta_{n} \sim p \left( y_{k} | \theta_{k} \right)$$
• (2) Parameters $\theta_{1} , \cdots , \theta_{n}$ themselves are considered to be obtained by Hyper parameter $\lambda$: $$\theta_{1} , \cdots , \theta_{n} | \lambda \sim \pi \left( \theta_{k} | \lambda \right)$$
• (3) $\lambda$ also has its own explaining distribution: $$\lambda \sim \pi (\lambda)$$

Machine Learning

In machine learning techniques, weights or biases related to results according to data are referred to as Parameters. Constants decided by the user in methods and algorithms to obtain these parameters are called Hyper parameters.

Description

Regardless of the context in which the phrase Hyper parameter is used, it is utilized to mean a parameter that determines Parameters.

See Also

Parameter