📂Linear Algebra

Norm of Linear Transformations

Definition¹

Define the norm of the linear transformation $T \in L(\mathbb{R}^{n}, \mathbb{R}^{m})$ as follows.

$$ \begin{equation} \| T \| := \sup \limits_{\| \mathbf{x} \| = 1} \| T(\mathbf{x}) \| \end{equation} $$

Explanation

(a) From (a) we have the following equality, so $\| T \|$ can be interpreted as the ratio by which $T$ changes the magnitude of elements of $\mathbb{R}^{n}$ when mapping them into $\mathbb{R}^{m}$. In other words, no matter how much the magnitude changes, it is on the order of $\| T \|$.

$$ \dfrac{|T(\mathbf{x})|}{\| \mathbf{x} \|} \le \| T \| $$

Also, by the definition, $\| T \|$ is the smallest value among $\lambda$ that satisfy the following.

$$ \| T (\mathbf{x}) \| \le \lambda \| \mathbf{x} \| , \quad \forall \mathbf{x} \in \mathbb{R}^{n} $$

It is straightforward to verify that $\| T \|$ satisfies the properties of a norm.

• $\| T \| \ge 0$
• $\| T \| = 0 \iff T = 0$
• $\| c T \| = | c | \| T \|$
• $\| T_{1} + T_{2} \| \le \| T_{1} \| + \| T_{2} \|$

Then, since we can equip $L(\mathbb{R}^{n}, \mathbb{R}^{m})$ with the [distance] as follows (../1840), $L(\mathbb{R}^{n}, \mathbb{R}^{m})$ becomes a metric space.

$$ d(T_{1}, T_{2}) = \| T_{1} - T_{2} \|,\quad T_{1},T_{2} \in L(\mathbb{R}^{n}, \mathbb{R}^{m}) $$

Definition $(1)$ and statement (a) are necessary and sufficient conditions.

$$ \| T \| := \sup \limits_{\| \mathbf{x} \| = 1} \| T(\mathbf{x}) \| \implies \| T(\mathbf{x}) \| \le \| T \| \| \mathbf{x} \|,\quad \forall \mathbf{x} \in \mathbb{R}^{n} $$

$$ \| T \| := \min \left\{ K : \| T(\mathbf{x}) \| \le K \| \mathbf{x} \|,\quad \forall \mathbf{x} \in \mathbb{R}^{n} \right\} \implies \| T \| = \sup \limits_{\| \mathbf{x} \| = 1} \| T(\mathbf{x}) \| $$

Theorem

• (a) If $T \in L(\mathbb{R}^{n}, \mathbb{R}^{m})$, then the following holds. $$ \| T(\mathbf{x}) \| \le \| T \| \| \mathbf{x} \|,\quad \forall \mathbf{x} \in \mathbb{R}^{n} $$

• (b) If $T \in L(\mathbb{R}^{n}, \mathbb{R}^{m})$, then $\| T \| < \infty$ and $T$ is uniformly continuous.

• (c) If $T_{1} \in L(\mathbb{R}^{n}, \mathbb{R}^{m})$ and $T_{2} \in L(\mathbb{R}^{m}, \mathbb{R}^{k})$, then the following holds. $$ \|T_{2}\circ T_{1} \| \le \| T_{2} \| \| T_{1} \| $$

Proof

(a)

Let $\mathbf{x} \ne \mathbf{0}$. Since $T$ is a linear transformation, the following holds.

$$ \begin{align*} \dfrac{ \| T(\mathbf{x}) \|}{\| \mathbf{x} \|} &= \dfrac{1}{\| \mathbf{x} \|}|T(\mathbf{x})| \\ &= \left\| \dfrac{1}{\| \mathbf{x} \|} T(\mathbf{x}) \right\| \\ &= \left\| T\left( \dfrac{\mathbf{x}}{\| \mathbf{x} \|} \right) \right\| \end{align*} $$

Then, because $\left\| \dfrac{\mathbf{x}}{\| \mathbf{x} \|} \right\| = 1$, by the definition of $\| T \|$ we obtain the following.

$$ \begin{align*} && \dfrac{ \| T(\mathbf{x}) \|}{\| \mathbf{x} \|} &\le \| T \| \\ \implies && \| T(\mathbf{x}) \| &\le \| T \| \| \mathbf{x} \|,\quad \forall \mathbf{x} \in \mathbb{R}^{n} \end{align*} $$

■

(b)

Let $\left\{ \mathbf{e}_{1}, \dots, \mathbf{e}_{n} \right\}$ be the standard basis of $\mathbb{R}^{n}$. Then for $\mathbf{x} \in \mathbb{R}^{n}$ with $\| \mathbf{x} \| \le 1$ the following holds.

$$ \mathbf{x} = \sum c_{i}\mathbf{e}_{i} \quad \text{and} \quad | c_{i} | \le 1 $$

Since $T$ is a linear transformation, we have the following.

$$ \| T (\mathbf{x}) \| = \left\| T \left( \sum _{i=1}^{n} c_{i} \mathbf{e}_{i} \right) \right\| = \left\| \sum _{i=1}^{n} c_{i} T \left(\mathbf{e}_{i} \right) \right\| \le \sum _{i=1}^{n} | c_{i} | \| T (\mathbf{e}_{i}) \| \le \sum _{i=1}^{n} \| T (\mathbf{e}_{i}) \| $$

Hence we obtain:

$$ \| T \| \le \sum _{i=1}^{n} \| T (\mathbf{e}_{i}) \| < \infty $$

By (a), for $\mathbf{x}, \mathbf{y} \in \mathbb{R}^{n}$ the following holds.

$$ |T(\mathbf{x}) - T(\mathbf{y})| \le \| T \| \| \mathbf{x} - \mathbf{y} \| $$

Let $\varepsilon > 0$ be given, and put $\delta = \dfrac{\varepsilon}{\| T \|}$. Then the following holds, so $T$ is uniformly continuous.

$$ \| \mathbf{x} - \mathbf{y} \| < \delta \implies |T(\mathbf{x}) - T(\mathbf{y})| \le \| T \| \| \mathbf{x} - \mathbf{y} \| = \| T \| \dfrac{\varepsilon}{\| T \|} = \varepsilon $$

(c)

By (a) we have the following.

$$ \| (T_{2}\circ T_{1}) (\mathbf{x}) \| = \| T_{2} (T_{1} (\mathbf{x})) \| \le \| T_{2} \| |T_{1} (\mathbf{x}) | \le \| T_{2} \| \|T_{1}\| \| \mathbf{x} \| $$

Therefore, by the definition of $\| T_{2} \circ T_{1} \|$, we obtain:

$$ \| T_{2} \circ T_{1} \| \le \| T_{2} \| \|T_{1}\| $$