Operations and Notation Table of Vectors and Matrices
Overview
This document summarizes various notations and operations for vectors and matrices.
Vector
Vectors are usually denoted in lower case bold type, and unless otherwise stated, they refer to a $n\times 1$ matrix, that is, a column vector. The $i$rd component of vector $\mathbf{x}$ is represented as $x_{i}$.
$$ \mathbf{x}=\begin{bmatrix} x_{1} \\ x_{2} \\ \vdots \\ x_{n} \end{bmatrix},\quad\mathbf{y}=\begin{bmatrix} y_{1} \\ y_{2} \\ \vdots \\ y_{n} \end{bmatrix} $$
Row vectors are denoted as $\mathbf{x}^{T}$, and ${}^{T}$ is called the transpose.
$$ \mathbf{x}^{T}=\begin{bmatrix} x_{1} & x_{2} & \cdots & x_{n} \end{bmatrix},\quad\mathbf{y}^{T}=\begin{bmatrix} y_{1} & y_{2} & \cdots & y_{n} \end{bmatrix} $$
Norm of Vectors
The norm of a vector is defined as follows and is the same concept as the magnitude of a vector in three-dimensional space.
$$ \left\| \mathbf{x} \right\| =\sqrt{\sum \limits _{i=1} ^{n} x_{i}^{2} },\quad \left\| \mathbf{x} \right\|^{2}=\sum \limits _{i=1} ^{n} x_{i}^{2} $$
Inner Product of Two Vectors
The matrix multiplication of row and column vectors of the same size is called the inner product. The result of the inner product is a scalar.
$$ \begin{align*} \mathbf{x}^{T}\mathbf{y}&=\begin{bmatrix} x_{1} & x_{2} & \cdots & x_{n} \end{bmatrix}\begin{bmatrix} y_{1} \\ y_{2} \\ \vdots \\ y_{n} \end{bmatrix} \\ &=x_{1}y_{1}+x_{2}y_{2}+\cdots+x_{n}y_{n} \\ &=\sum \limits_{i=1}^{n} x_{i}y_{i} \end{align*} $$
The norm is a scalar, so $\mathbf{x}^{T}\mathbf{y}=\mathbf{y}^{T}\mathbf{x}$ holds, and if expressed as an inner product, it is as follows.
$$ \left\| \mathbf{x} \right\| =\sqrt{\mathbf{x}^{T} \mathbf{x} },\quad \left\| \mathbf{x} \right\|^{2} =\mathbf{x}^{T} \mathbf{x} $$
Outer Product of Two Vectors
The matrix multiplication of a column vector and a row vector as follows is called the outer product of two vectors (not the cross product). The result of the outer product is a $n \times n$ matrix.
$$ \begin{align*} \mathbf{x}\mathbf{y}^{T} &= \begin{bmatrix} x_{1} \\ x_{2} \\ \vdots \\ x_{n} \end{bmatrix} \begin{bmatrix} y_{1} & y_{2} & \cdots & y_{n} \end{bmatrix} \\ &= \begin{bmatrix} x_{1}y_{1} & x_{1}y_{2} & \cdots & x_{1}y_{n} \\ x_{2}y_{1} & x_{2}y_{2} & \cdots & x_{2}y_{n} \\ \vdots & \vdots & \ddots & \vdots \\ x_{n}y_{1} & x_{n}y_{2} & \cdots & x_{n}y_{n} \end{bmatrix} \\ &= \begin{bmatrix} x_{1}\begin{bmatrix} y_{1} & y_{2} & \cdots & y_{n} \end{bmatrix} \\ x_{2}\begin{bmatrix} y_{1} & y_{2} & \cdots & y_{n} \end{bmatrix} \\ \vdots \\ x_{n}\begin{bmatrix} y_{1} & y_{2} & \cdots & y_{n} \end{bmatrix} \end{bmatrix} \\ &= \begin{bmatrix} x_{1}\mathbf{y}^{T} \\ x_{2}\mathbf{y}^{T} \\ \vdots \\ x_{n}\mathbf{y}^{T} \end{bmatrix} \end{align*} $$
The commutative law does not hold.
$$ \mathbf{x} \mathbf{y}^{T} \ne \mathbf{y} \mathbf{x}^{T} $$
It can also be seen that the trace of the outer product is equal to the inner product.
$$ \mathbf{x}^{T}\mathbf{y} =\sum \limits _{i=1} ^{n} x_{i}y_{i} =\text{Tr}\left( \mathbf{x}\mathbf{y}^{T} \right) =\text{Tr}\left( \mathbf{y}\mathbf{x}^{T} \right) $$
Therefore, the following equation holds.
$$ \begin{align*} \left\| \mathbf{x} \right\| &=\sqrt{\sum \limits _{i=1} ^{n} x_{i}^{2}}=\sqrt{\mathbf{x}^{T}\mathbf{x}}=\sqrt{\text{Tr}\left( \mathbf{x}\mathbf{x}^{T} \right)} \\ \left\| \mathbf{x} \right\|^{2} &=\sum \limits _{i=1} ^{n} x_{i}^{2}=\mathbf{x}^{T}\mathbf{x}=\text{Tr}\left( \mathbf{x}\mathbf{x}^{T} \right) \end{align*} $$
Matrix
Matrices are usually denoted in uppercase, and an arbitrary matrix is typically denoted as a $m\times n$ matrix. It refers to the form stacked with row vectors vertically or arranged with column vectors horizontally. In this document, a matrix is treated as a row vector of column vectors. If the $i$rd column vector is denoted as $\mathbf{x}_{i} = \begin{bmatrix} x_{1i} & x_{2i} & \cdots & x_{mi} \end{bmatrix}^{T}$,