logo

Multilayer Perceptron (MLP), Fully Connected Neural Network (FCNN) 📂Machine Learning

Multilayer Perceptron (MLP), Fully Connected Neural Network (FCNN)

Definition

Let $L_{i} : \mathbb{R}^{n_{i}} \to \mathbb{R}^{n_{i+1}}$ be referred to as a fully connected layer. Let $\sigma : \mathbb{R} \to \mathbb{R}$ be referred to as an activation function. The composition of these is called a multilayer perceptron.

$$ \operatorname{MLP}(\mathbf{x}) = T_{N} \circ \overline{\sigma} \circ T_{N-1} \circ \overline{\sigma} \circ \cdots \circ T_{1} (\mathbf{x}) $$

Here, $\overline{\sigma}$ is a function that applies $\sigma$ to each component.

$$ \overline{\sigma}(\mathbf{x}) = \begin{bmatrix} \sigma(x_{1}) \\ \sigma(x_{2}) \\ \vdots \\ \sigma(x_{n}) \end{bmatrix} \qquad \text{where } \mathbf{x} = \begin{bmatrix} x_{1} \\ x_{2} \\ \vdots \\ x_{n} \end{bmatrix} $$

Explanation

It is called a multilayer perceptron because it is a composition of single-layer perceptrons multiple times, primarily using fully connected layers, and is also referred to as a fully connected neural network. These terms refer to the same neural network.