📂Matrix Algebra

Matrix Inner Product

Definition: Inner Product of Two Column Vectors¹

The inner product of two column vectors of size $n \times 1$, $\mathbf{u}$, $\mathbf{v}$ $\in \mathbb{R}^{n}$ is defined as follows.

$$ \begin{equation} \mathbf{u} \cdot \mathbf{v} := \mathbf{u}^{T}\mathbf{v}=u_{1}v_{1} + u_{2}v_{2} + \cdots + u_{n}v_{n} \label{EuclideanIP} \end{equation} $$

In the case where $\mathbf{u}$, $\mathbf{v}$ $\in \mathbb{C}^{n}$, it is as follows.

$$ \mathbf{u} \cdot \mathbf{v} := \mathbf{u}^{\ast}\mathbf{v}=u^{\ast}_{1}v_{1}^{\ } + u_{2}^{\ast}v_{2}^{\ } + \cdots + u_{n}^{\ast}v_{n}^{\ } $$

Here, $\mathbf{u}$ is the conjugate transpose of $\mathbf{u}$. Two vectors $\mathbf{u}$, $\mathbf{v}$ are said to be orthogonal to each other if they satisfy the following equation, and it is denoted as $\mathbf{u} \perp \mathbf{v}$.

$$ \mathbf{u} \cdot \mathbf{v} = 0 $$

The norm or length of the column vector $\mathbf{v}$ is defined as follows.

$$ \left\| \mathbf{v} \right\| := \sqrt{\mathbf{v} \cdot \mathbf{v}} $$

A vector with norm $1$ is called a unit vecter. The distance between two vectors $\mathbf{u}$, $\mathbf{v}$ is represented as $d(\mathbf{u}. \mathbf{v})$ and defined as follows.

$$ d(\mathbf{u}, \mathbf{v}) := \left\| \mathbf{u} - \mathbf{v} \right\| = \sqrt{(\mathbf{u}-\mathbf{v}) \cdot (\mathbf{u}-\mathbf{v})} = \sqrt{(\mathbf{u}-\mathbf{v})^{\ast} (\mathbf{u}-\mathbf{v})} $$

Explanation

In coordinate space, the inner product of two vectors is nothing but its representation as a matrix multiplication and its expansion to complex numbers. Therefore, $\eqref{EuclideanIP}$ is called the Euclidean inner product or standard inner product. Thus, the notation $\cdot$ is used for the inner product, but the general notation for the inner product is as follows.

$$ \left\langle \mathbf{u}, \mathbf{v} \right\rangle $$

By definition, in the case of real matrices, $\mathbf{u} \cdot \mathbf{v} = \mathbf{v} \cdot \mathbf{u}$ holds, and in the case of complex matrices, $\mathbf{u} \cdot \mathbf{v} = \overline{\mathbf{v} \cdot \mathbf{u}}$ holds.

The core concept of the inner product is ‘multiply the same components and sum them all,’ which generalized for $n\times n$ matrix is as follows.

Properties

Let $A$ be a $n\times n$ real matrix and $\mathbf{u},\mathbf{v}$ a $n\times 1$ real matrix. Then, the following equation holds.

$$ \begin{align*} A \mathbf{u} \cdot \mathbf{v} &= \mathbf{u} \cdot A^{T} \mathbf{v} \\ \mathbf{u} \cdot A \mathbf{v} &= A^{T} \mathbf{u} \cdot \mathbf{v} \end{align*} $$

In the case of complex matrices, the following equation holds.

$$ \begin{align*} A \mathbf{u} \cdot \mathbf{v} &= \mathbf{u} \cdot A^{\ast} \mathbf{v} \\ \mathbf{u} \cdot A \mathbf{v} &= A^{\ast} \mathbf{u} \cdot \mathbf{v} \end{align*} $$

Proof

Since the method of proving the four formulas is the same, only the proof of the first formula is introduced.

By the properties of the transpose matrix, the following holds.

$$ \begin{align*} A \mathbf{u} \cdot \mathbf{v} &= \left( A \mathbf{u} \right)^{T} \mathbf{v} \\ &= \left( \mathbf{u}^{T} A^{T} \right) \mathbf{v} \\ &= \mathbf{u}^{T} \left( A^{T} \mathbf{v} \right) \\ &= \mathbf{u} \cdot A^{T} \mathbf{v} \end{align*} $$

■

See Also

General Definition of Inner Product

General Definition of Norm

General Definition of Distance

Relationship between Inner Product, Norm, and Distance