📂Matrix Algebra

Frobenius Norm

Definition ¹

For a matrix $A = \left( a_{ij} \right) \in \mathbb{C}^{m \times n}$, the matrix norm $\left\| \cdot \right\|_{F}$ is defined as follows and is called the Frobenius norm. $$ \left\| A \right\|_{F} = \sqrt{ \sum_{ij} \left| a_{ij} \right|^{2} } = \sqrt{ \text{Tr} \left( A^{\ast} A \right) } $$

Explanation

The Frobenius norm is also called the Hilbert–Schmidt norm. $n = 1$, i.e. in the space of $m$-dimensional vectors it coincides with the Euclidean norm, so it can be regarded as the natural generalization of the Euclidean norm.

Although the name Frobenius may sound grandiose, the definition itself is not difficult — think of it simply. It is often simply written as “norm” and denoted $\| \cdot \|$, and the expression Frobenius$(\| \cdot \|_{F})$ may be understood as emphasizing that the object is a matrix.

Similarly, the inner product of matrices is also called the Frobenius inner product.

$$ \braket{A, B}_{F} := \sum\limits_{i = 1}^{m} \sum\limits_{j = 1}^{n} \overline{a_{ij}}b_{ij}, \qquad A, B \in M_{n \times n}(\mathbb{C}) $$

$$ \| A \|_{F} = \sqrt{\braket{A, A}_{F}} = \sqrt{\sum_{i,j} |a_{ij}|^{2}} = \sqrt{\Tr(A^{\ast}A)} $$

Properties

Let $A \in M_{n \times n}(\mathbb{C})$ and $\mathbf{x} \in \mathbb{C}^{n}$. The following hold. (The norms and inner products for vectors and matrices are distinguished by the subscript $_{F}$.)

(a) For the eigenvalues $\lambda_{i}$ of $A$, $\| A \|_{F} = \sqrt{\sum_{i} |\lambda_{i}|^{2}}$

(b) $\| A \mathbf{x} \| \le \| A \|_{F} \| \mathbf{x} \|$

(c) $\| A B \|_{F} \le \| A \|_{F} \| B \|_{F}$

Proof

(a)

$A^{\ast}A$ is a Hermitian matrix, so unitary diagonalization $A^{\ast}A = Q^{\ast} \Lambda Q$ is possible. If $\lambda_{i}$ are the eigenvalues of $A$, since the eigenvalues of $A^{\ast}A$ are $| \lambda_{i} |^{2}$](../3762), we obtain the following. By the cyclic property of the trace,

$$ \begin{align*} \| A \|_{F} &= \sqrt{\Tr(A^{\ast}A)} = \sqrt{\Tr(Q^{\ast} \Lambda Q)} \\ &= \sqrt{\Tr(\Lambda Q Q^{\ast})} = \sqrt{\Tr(\Lambda)} \\ &= \sqrt{\sum_{i} |\lambda_{i}|^{2}} \end{align*} $$

■

(b)

By the properties of inner products and norms, the cyclic property of the trace, and the Cauchy–Schwarz inequality, the following holds.

$$ \begin{align*} \| A \mathbf{x} \| &= \sup\limits_{\| \mathbf{y} \| = 1} | \braket{\mathbf{y}, A \mathbf{x}} | \\ &= \sup\limits_{\| \mathbf{y} \| = 1} | \mathbf{y}^{\ast} A \mathbf{x} | \\ &= \sup\limits_{\| \mathbf{y} \| = 1} \Tr( | \mathbf{y}^{\ast} A \mathbf{x} | ) \\ &= \sup\limits_{\| \mathbf{y} \| = 1} \Tr( | \mathbf{x} \mathbf{y}^{\ast} A | ) \\ &= \sup\limits_{\| \mathbf{y} \| = 1} \Tr( | \braket{\mathbf{y} \mathbf{x}^{\ast}, A}_{F} | ) \\ &= \sup\limits_{\| \mathbf{y} \| = 1} | \braket{\mathbf{y} \mathbf{x}^{\ast}, A}_{F} | \\ &\le \sup\limits_{\| \mathbf{y} \| = 1} \| \mathbf{y} \mathbf{x}^{\ast} \|_{F} \| A \|_{F} \\ \end{align*} $$

Here, by the properties of matrix norms, the following holds.

$$ \begin{align*} \| \mathbf{y} \mathbf{x}^{\ast} \|_{F} &= \sqrt{\Tr \left( (\mathbf{y} \mathbf{x}^{\ast})^{\ast} \mathbf{y} \mathbf{x}^{\ast} \right)} \\ &= \sqrt{\Tr \left( \mathbf{x} \mathbf{y}^{\ast} \mathbf{y} \mathbf{x}^{\ast} \right)} \\ &= \sqrt{\Tr \left( \mathbf{x} \| \mathbf{y} \|^{2} \mathbf{x}^{\ast} \right)} \\ &= \sqrt{\| \mathbf{y} \|^{2} \Tr \left( \mathbf{x} \mathbf{x}^{\ast} \right)} \\ &= \sqrt{\| \mathbf{y} \|^{2} \Tr \left( \| \mathbf{x} \|^{2} \right)} \\ &= \sqrt{\| \mathbf{y} \|^{2} \| \mathbf{x} \|^{2} } \\ &= \| \mathbf{y} \| \| \mathbf{x} \| \end{align*} $$

Therefore we obtain

$$ \| A \mathbf{x} \| \le \sup\limits_{\| \mathbf{y} \| = 1} \| \mathbf{y} \| \| \mathbf{x} \| \| A \|_{F} = \| A \|_{F} \| \mathbf{x} \| $$

■

(c)

For $\| AB \|_{F}$ the following holds.

$$ \| AB \|_{F} = \sqrt{\Tr \left( (AB)^{\ast} AB \right)} = \sqrt{\Tr \left( B^{\ast} A^{\ast} AB \right)} $$

$A^{\ast}A$ is a Hermitian matrix, so unitary diagonalization $A^{\ast}A = Q^{\ast} D Q$ is possible. Let $C = QB$, then we obtain

$$ \| AB \|_{F} = \sqrt{\Tr \left( B^{\ast}Q^{\ast} D QB \right)} = \sqrt{\Tr \left( C^{\ast} D C \right)} $$

$\Tr(C^{\ast} D C)$ is as follows. If $\lambda_{i}$ are the eigenvalues of $A$, then

$$ \Tr(C^{\ast} D C) = \Tr \left( \begin{bmatrix} -\mathbf{c}_{1}^{\ast}- \\ \vdots \\ -\mathbf{c}_{n}^{\ast}- \end{bmatrix} \begin{bmatrix} \lambda_{1}^{2} & & \\ & \ddots & \\ & & \lambda_{n}^{2} \end{bmatrix} \begin{bmatrix} \overset{|}{\underset{|}{\mathbf{c}_{1}}} & \cdots & \overset{|}{\underset{|}{\mathbf{c}_{n}}} \end{bmatrix} \right) = \sum\limits_{i} \lambda_{i}^{2} \| \mathbf{c}_{i} \|^{2} $$

On the other hand, since $\| A \|_{F} = \sqrt{\sum\limits_{i} |\lambda_{i}|^{2}}$ and $\| B \|_{F} = \sqrt{\Tr (C^{\ast}Q Q^{\ast} C)} = \sqrt{\sum\limits_{i} \| \mathbf{c}_{i} \|}$, we obtain the following.

$$ \| AB \|_{F} = \sqrt{\sum\limits_{i} |\lambda_{i}|^{2} \| \mathbf{c}_{i} \|^{2}} \le \sqrt{\sum\limits_{i} |\lambda_{i}|^{2}} \sqrt{\sum\limits_{i} \| \mathbf{c}_{i} \|^{2}} = \| A \|_{F} \| B \|_{F} $$

■