Inner Product of Matrices (Frobenius Inner Product)
Definition
The inner product or dot product of two matrices $m \times n$, $X = [x_{ij}]$, $Y=[Y]_{ij}$ is defined as follows.
$$ X \cdot Y = \braket{X, Y} = \sum_{i=1}^{m}\sum\limits_{j=1}^{n} x_{ij}y_{ij} $$
In the case where the components are complex numbers in a complex matrix, it is defined as follows.
$$ X \cdot Y = \braket{X, Y} = \sum_{i=1}^{m}\sum\limits_{j=1}^{n} \overline{x}_{ij}y_{ij} $$
Here, $\overline{x}$ denotes the complex conjugate.
Explanation
Since the inner product (dot product) of vectors is “multiplying each component of two vectors and summing them all,” defining the inner product of matrices as above becomes a natural generalization of the inner product of vectors. It can be denoted as below and is also called the Frobenius inner product, which is simply named after a person without any special meaning.
$$ \braket{X, Y}_{F} $$
It’s not extremely commonly used, and it’s rather a matter of notational preference. Consider it as something used to emphasize that $X$ and $Y$ are matrices.
Norm
If there is an inner product, you can naturally define a norm, so the (Frobenius) norm of a matrix is defined as follows.
$$ \| X \|_{F} := \sqrt{\braket{X, X}_{F}} = \sqrt{\sum\limits_{i,j} (x_{ij})^{2}} = \sqrt{\Tr (X^{\mathsf{T}}X)} $$
As explained above, the subscript $_{F}$ is often omitted. The last equality holds by the following formula.
Basic Properties
For complex matrices $X$, $Y$, $Z$, and complex number $\alpha, \beta$,
- Linearity:
- $\braket{\alpha X, \beta Y} = \overline{\alpha}\beta \braket{X, Y}$
- $\braket{X + Y, Z} = \braket{X, Z} + \braket{Y, Z}$
- $\braket{X, Y + Z} = \braket{X, Y} + \braket{X, Z}$
- Conjugate Symmetry: $\braket{X, Y} = \overline{\braket{Y, X}}$
Formula
The following holds for $(1)$. $$ X \cdot Y = \sum_{i, j} x_{ij}y_{ij} = \Tr (X Y^{\mathsf{T}}) = \Tr (X^{\mathsf{T}}Y) $$ In the case of complex matrices, $$ X \cdot Y = \sum_{i, j} \overline{x}_{ij}y_{ij} = \Tr (X Y^{\ast}) = \Tr (X^{\ast}Y) $$ Here, $^{\ast}$ denotes the conjugate transpose.
Proof
$(1)$
Expanding in terms of index $i$,
$$ \sum_{i=1}^{m}\sum\limits_{j=1}^{n} x_{ij}y_{ij} = \sum\limits_{j=1}^{n} x_{1j}y_{1j} + \sum\limits_{j=1}^{n} x_{2j}y_{2j} + \cdots + \sum\limits_{j=1}^{n} x_{nj}y_{nj} $$
The first term is the element in the $XY^{\mathsf{T}}$th row, $1$th column of $1$.
$$ [AB^{\mathsf{T}}]_{ij} = \left[ \sum_{k} a_{ik}b_{jk}\right] $$
The second term is $[XY^{\mathsf{T}}]_{22}$, and the $i$th term is $[XY^{\mathsf{T}}]_{ii}$. Therefore, summarizing,
$$ \sum_{i=1}^{m}\sum\limits_{j=1}^{n} x_{ij}y_{ij} = \sum\limits_{i} [XY^{\mathsf{T}}]_{ii} $$
This coincides with the definition of trace. Since $\Tr(A) = \Tr(A^{\mathsf{T}})$, we finally obtain:
$$ X \cdot Y = \braket{X, Y} = \sum_{i, j} x_{ij}y_{ij} = \Tr (X Y^{\mathsf{T}}) = \Tr (X^{\mathsf{T}}Y) $$
■