📂Machine Learning

딥러닝에서 레이어란?

Definition

In deep learning, a linear transformation $L^{mn} : \mathbb{R}^{n} \to \mathbb{R}^{m}$ is called a layer.

Generalization

In deep learning, for a fixed $\mathbf{b} \in \mathbb{R}^{m}$, an affine transformation $\mathbf{x} \mapsto L^{mn}(\mathbf{x}) + \mathbf{b}$ is also called a layer.

Description

In other words, a layer refers to a linear vector function. On the other hand, a non-linear scalar function is called an activation function.

The reason it is called a layer is that when we visualize multiple compositions, it appears as though layers are stacked on top of each other.

$$L^{n_{m}n_{m-1}} \circ \cdots \circ L^{n_{2}n_{1}} : \begin{bmatrix} x_{1} \\ \vdots \\ x_{n_{1}}\end{bmatrix} \mapsto \begin{bmatrix} y_{1} \\ \vdots \\ \vdots \\ y_{n_{2}}\end{bmatrix} \mapsto \begin{bmatrix} z_{1} \\ \vdots \\ \vdots \\ \vdots \\ z_{n_{3}}\end{bmatrix} \mapsto \cdots \mapsto \begin{bmatrix} v_{1} \\ \vdots \\ \vdots \\ v_{n_{m-1}}\end{bmatrix} \mapsto \begin{bmatrix} w_{1} \\ \vdots \\ w_{n_{m}}\end{bmatrix}$$

The composition of a layer and an activation function is referred to as an artificial neural network. Specifically, the composition of a linear function and a step function is called a (single-layer) perceptron. A deep neural network is formed by “many” compositions of layers and activation functions. Approximating a function using a deep neural network is termed as deep learning.

Types

• Linear Layer
• Convolutional Layer
• Pooling Layer