Tensor Product of Vector Spaces
Buildup1
- For convenience, this exposition is developed for the complex number space $\mathbb{C}$, but it is applicable to $\mathbb{R}$ or any vector space as well.
Let’s denote the set of functions from a finite set $\Gamma$ to the complex number space as described by $\mathbb{C}^{\Gamma}$.
$$ \mathbb{C}^{\Gamma} = \left\{ f : \Gamma \to \mathbb{C} \right\} $$
Let’s set $\Gamma$ as $\mathbf{n} = \left\{ 1, 2, \dots, n \right\}$. A function that sends each $1 \le i \le n$ to a complex number $z_{i} \in \mathbb{C}$ is denoted by $(z_{1}, \dots, z_{n})$, which belongs to $\mathbb{C}^{\mathbf{n}}$ and is also equivalent to a vector in the complex number ordered pair set $\mathbb{C}^{n}$.
$$ (z_{1}, \dots, z_{n}) : i \mapsto z_{i} $$
$$ \mathbb{C}^{n} := \mathbb{C}^{\mathbf{n}} = \left\{ (z_{1}, \dots, z_{n}) \vert z_{i} \in \mathbb{C} \right\} $$
Hence, $v \in \mathbb{C}^{\Gamma}$ can be seen both as a function similar to $v : i \mapsto z_{i}$ and as an ordered pair like $v = (z_{1}, \dots, z_{\left| \Gamma \right|})$.
For finite sets $\Gamma_{1}$ and $\Gamma_{2}$, the tensor product of two vector spaces $\mathbb{C}^{\Gamma_{1}}$ and $\mathbb{C}^{\Gamma_{2}}$ is defined as the function space (vector space) $\mathbb{C}^{\Gamma_{1} \times \Gamma_{2}}$ created from the product space $\Gamma_{1} \times \Gamma_{2}$ of $\Gamma_{1}$ and $\Gamma_{2}$.
Definition2
For finite sets $\Gamma_{1}$ and $\Gamma_{2}$, the tensor product of two vector spaces $\mathbb{C}^{\Gamma_{1}}$ and $\mathbb{C}^{\Gamma_{2}}$ is defined as follows.
$$ \mathbb{C}^{\Gamma_{1}} \otimes \mathbb{C}^{\Gamma_{2}} := \mathbb{C}^{\Gamma_{1} \times \Gamma_{2}} $$
Here, $\Gamma_{1} \times \Gamma_{2}$ is the product space of $\Gamma_{1}$ and $\Gamma_{2}$.
Explanation
For a simple example, let’s consider $\Gamma_{1} = \mathbf{2} = \left\{ 1, 2 \right\}$ and $\Gamma_{2} = \mathbf{3} = \left\{ 1, 2, 3 \right\}$. And let’s assume $\Gamma$ as their product space.
$$ \Gamma = \Gamma_{1} \times \Gamma_{2} = \left\{ (1,1), (1,2), (1,3), (2,1), (2,2), (2,3) \right\} $$
Let’s denote the elements of this as follows.
$$ e_{i} \otimes e_{j} = (i, j) $$
Following the logic from the overview, $v \in \mathbb{C}^{\Gamma}$ can be seen as a function like $(i,j) \mapsto \alpha_{ij}$ and also as an ordered pair like $\left( \alpha_{11}, a_{12}, a_{13}, a_{21}, a_{22}, a_{23} \right)$. Therefore, $\mathbb{C}^{\Gamma}$ is a vector space with $\left\{ e_{i} \otimes e_{j} : 1 \le i \le 2, 1 \le j \le 3 \right\}$ as its basis.
$$ \begin{align*} \mathbb{C}^{\Gamma} &= \left\{ \sum\limits_{i,j} \alpha_{i,j} e_{i} \otimes e_{j} : \alpha_{ij} \in \mathbb{C} \right\} \\ &= \left\{ \left( \alpha_{11}, a_{12}, a_{13}, a_{21}, a_{22}, a_{23} \right) : \alpha_{ij} \in \mathbb{C} \right\} \end{align*} $$
Hence, it is isomorphic to $\mathbb{C}^{6}$.
$$ \mathbb{C}^{\Gamma} = \mathbb{C}^{2} \otimes \mathbb{C}^{3} \cong \mathbb{C}^{6} $$
Grouping $\mathbb{C}$ into a product space can be thought of as increasing the position of variables, and grouping into a tensor product as increasing the position of indices.
$$ z_{1} \in \mathbb{C}\qquad (z_{1},z_{2}) \in \mathbb{C} \times \mathbb{C}\qquad (z_{1}, z_{2}, z_{3}) \in \mathbb{C}\times \mathbb{C} \times \mathbb{C} $$
$$ (z_{1}, z_{2}) \in \mathbb{C}^{2} \qquad (z_{11}, z_{12}, z_{21}, z_{22}) \in \mathbb{C}^{2} \otimes \mathbb{C}^{2} \\[1em] (z_{111}, z_{112}, z_{121}, z_{122}, z_{211}, z_{212}, z_{221}, z_{222}) \in \mathbb{C}^{2} \otimes \mathbb{C}^{2} \otimes \mathbb{C}^{2} $$
Each $e_{i} \otimes e_{j}$ corresponds to the standard basis vector of $\mathbb{C}^{6}$ as follows.
$$ e_{1} \otimes e_{1} = \begin{bmatrix}1 \\ 0 \\ 0 \\ 0 \\ 0 \\ 0\end{bmatrix} \quad e_{1} \otimes e_{2} = \begin{bmatrix}0 \\ 1 \\ 0 \\ 0 \\ 0 \\ 0\end{bmatrix} \quad e_{1} \otimes e_{3} = \begin{bmatrix}0 \\ 0 \\ 1 \\ 0 \\ 0 \\ 0\end{bmatrix} \\[2em] e_{2} \otimes e_{1} = \begin{bmatrix}0 \\ 0 \\ 0 \\ 1 \\ 0 \\ 0\end{bmatrix} \quad e_{2} \otimes e_{2} = \begin{bmatrix}0 \\ 0 \\ 0 \\ 0 \\ 1 \\ 0\end{bmatrix} \quad e_{2} \otimes e_{3} = \begin{bmatrix}0 \\ 0 \\ 0 \\ 0 \\ 0 \\ 1\end{bmatrix} $$
When expressed as a Kronecker product of matrices, it looks like this.
$$ e_{1} \otimes e_{1} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \otimes \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} = \begin{bmatrix} 1 \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \\[1.5em] 0 \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix} \quad e_{1} \otimes e_{2} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \otimes \begin{bmatrix} 0 \\ 1 \\ 0\end{bmatrix} = \begin{bmatrix} 1 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \\[1.5em] 0 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix} \\[2em] e_{1} \otimes e_{3} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \otimes \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 1 \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \\[1.5em] 0 \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 1 \\ 0 \\ 0 \\ 0 \end{bmatrix} \quad e_{2} \otimes e_{1} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \otimes \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \\[1.5em] 1 \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \\ 1 \\ 0 \\ 0 \end{bmatrix} \\[2em] e_{2} \otimes e_{2} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \otimes \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \\[1.5em] 1 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 1 \\ 0 \end{bmatrix} \quad e_{2} \otimes e_{3} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \otimes \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \\[1.5em] 1 \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \\ 1 \end{bmatrix} $$
Moreover, from this, it can be understood that the following holds.
$$ \mathbb{C} \otimes \mathbb{C}^{n} \cong \mathbb{C}^{n} \qquad \mathbb{C} \otimes \mathbb{C} \cong \mathbb{C} $$
Properties
- $\mathbb{C}^{n} \otimes \mathbb{C}^{m}$ is a vector space with respect to the following two operations:
- $(x_{1} \otimes y_{1}) + (x_{2} \otimes y_{2}) = (x_{1} + x_{2}) \otimes (y_{1} + y_{2})$
- $\alpha (x \otimes y) = (\alpha x) \otimes y = x \otimes (\alpha y)$
- $\mathbb{C}^{n} \otimes \mathbb{C}^{m} \cong \mathbb{C}^{nm}$
- $\dim (\mathbb{C}^{n} \otimes \mathbb{C}^{m}) = \dim(\mathbb{C}^{n}) \cdot \dim(\mathbb{C}^{m}) = nm$
Generalization
For finite sets $\Gamma_{i} (1 \le i \le r)$, the tensor product of vector spaces $\mathbb{C}^{\Gamma_{i}}$ is defined as follows.
$$ \mathbb{C}^{\Gamma_{1}} \otimes \cdots \otimes \mathbb{C}^{\Gamma_{r}} := \mathbb{C}^{\Gamma} = \mathbb{C}^{\Gamma_{1} \times \cdots \times \Gamma_{r}},\quad \Gamma = \Gamma_{1} \times \cdots \times \Gamma_{r} $$
The basis vectors corresponding to $(j_{1}, \dots j_{r}) \in \prod\limits_{i} \Gamma_{i}$ are denoted as follows: $( 1 \le j_{i} \le \left| \Gamma_{i} \right|)$
$$ e_{j_{1}} \otimes \cdots \otimes e_{j_{r}} $$
Then, the tensor product is a vector space like the following. If the cardinality of $\Gamma_{i}$ is denoted by $n_{i} = \left| \Gamma_{i} \right|$,
$$ \begin{align*} \mathbb{C}^{\Gamma_{1}} \otimes \cdots \otimes \mathbb{C}^{\Gamma_{r}} &= \left\{ \sum\limits_{(j_{1}, \dots j_{r}) \in \prod\limits_{i} \Gamma_{i}} \alpha_{j_{1}, \dots, j_{r}} e_{j_{1}} \otimes \cdots \otimes e_{j_{r}}\right\} \\ &= \left\{ (\alpha_{1,\dots,1},\ \dots, \alpha_{n_{1},\ \dots, n_{r}}) : \alpha \in \mathbb{C} \right\} \\ &\cong \mathbb{C}^{\left| \Gamma \right|} \end{align*} $$
General Vector Spaces
Given a finite-dimensional vector space $V_{1}, \dots, V_{r}$, by selecting a basis $\mathcal{B}_{i} = \left\{ v_{1}, v_{2}, \dots, v_{\dim V_{i}} \right\}$ of vector space $V_{i}$, one can obtain the following bijection $f_{i}$.
$$ \begin{align*} f _{i}: & V_{i} \to \mathbb{C}^{\dim V_{i}} \\ & \sum z_{j}v_{j} \mapsto (z_{1}, \dots, z_{\dim V_{i}}) \end{align*} $$
Then, the tensor product of $V_{i}$ is defined as follows.
$$ \bigotimes\limits_{i=1}^{r} V_{i} = V_{1} \otimes \cdots \otimes V_{r} := \mathbb{C}^{\dim V_{1}} \otimes \cdots \otimes \mathbb{C}^{\dim V_{r}} $$