📂Linear Algebra

In Linear Algebra, What is a Norm?

Definition

Let us define a vector space over $V$ as $\mathbb{F}$.

$\left\| \cdot \right\| : V \to \mathbb{F}$ is defined as a norm on $V$ if it satisfies the following three conditions with respect to $\mathbf{u}, \mathbf{v} \in V$ and $k \in \mathbb{F}$:

• (i) Positive definiteness: $\left\| \mathbf{u} \right\| \ge 0$ and $\mathbf{u} = \mathbb{0} \iff \left\| \mathbf{u} \right\| = 0$
• (ii) Homogeneity: $\left\|k \mathbf{u} \right\| = | k | \left\| \mathbf{u} \right\|$
• (iii) Triangle inequality: $\left\| \mathbf{u} + \mathbf{v}\right\| \le \left\|\mathbf{v} \right\| + \left\| \mathbf{u} \right\|$

Explanation

The norm is a concept that starts from the absolute value and abstracts it. In Korean, there isn’t a direct translation, so it’s read as it is pronounced. Personally, I think it was awkwardly translated, so I try to pronounce it as close to [nɔ:m] as possible.

In linear algebra, the definition of a norm is as above. (In other words, a norm might be defined differently in other fields.) As can be seen, the essential conditions for defining a norm are those that make “measurement” or “comparison” possible. Although these concepts are intuitively defined when thinking about three-dimensional space $\mathbb{R}^3$, abstraction becomes necessary when considering complex numbers, for example. There are many types of norms in the world, and a norm in a vector space does not necessarily have to be unique. As long as these definitions are satisfied, there can be an infinite number of ways to think about norms.

Introducing the following norms for the vector space $\mathbb{C}^n$ and its vector $\mathbf{u} = ( u_1 , u_2 , \cdots , u_n ) \in \mathbb{C}^n$:

Manhattan Norm

$$\left\| \mathbf{u} \right\|_1 = \sum_{k=1}^{n} |u_k|$$

Also called the $\mathcal{l}^1$ norm, the Manhattan norm is used to define distances in taxicab geometry. The name “Manhattan” comes from the actual Manhattan city, designed to represent the actual travel distance rather than the simple straight-line distance. Though not exactly the same concept, the illustration helps understand why this norm was named after Manhattan. In the image, it corresponds to the blue line, and if one side of the green square is taken as $1$, the distance between A and B becomes $6+2 = 8$.

Euclidean Norm

$$\left\| \mathbf{u} \right\|_2 = \sqrt{\sum_{k=1}^{n} |u_k|^2}$$

The Euclidean norm is the concept of distance and magnitude that we are familiar with, obtained regardless of dimension as the square root of the sum of the squares of absolute values. In the image, it corresponds to the red line, and as is well known, the distance between A and B is $\sqrt{6^2 + 2^2} =6.32…$.

$\infty$-Norm, Maximum Norm

$$\left\| \mathbf{u} \right\|_\infty = \max_{1\le k \le n} |u_k|$$

Also known as the Supremum Norm, it simply takes the maximum value.

$p$-Norm

$$\left\| \mathbf{u} \right\|_p = \left( \sum_{k=1}^{n} |u_k|^p \right) ^ {{1} \over {p} }$$

$p$ can be equal to or greater than $1$ and does not necessarily have to be a natural number. The Manhattan norm and the Euclidean norm are special examples of the $p$-norm, corresponding to the $1$-norm and $2$-norm respectively. Especially, if $p = \infty$, it becomes the Maximum Norm, covering all the notations mentioned above.

It may or may not be noteworthy, but one interesting point is that the shape of the $p$-norm is similar to that of the $p$-moment in statistics. While there are differences such as the presence of absolute values or fixing the median at 0, just by looking at the shape, the $1$-norm reminds one of the mean, and the $2$-norm suggests variance.