logo

Tensor Product of Vector Spaces 📂Linear Algebra

Tensor Product of Vector Spaces

Buildup1

  • For convenience, this exposition is developed for the complex number space C\mathbb{C}, but it is applicable to R\mathbb{R} or any vector space as well.

Let’s denote the set of functions from a finite set Γ\Gamma to the complex number space as described by CΓ\mathbb{C}^{\Gamma}.

CΓ={f:Γ→C} \mathbb{C}^{\Gamma} = \left\{ f : \Gamma \to \mathbb{C} \right\}

Let’s set Γ\Gamma as n={1,2,…,n}\mathbf{n} = \left\{ 1, 2, \dots, n \right\}. A function that sends each 1≤i≤n1 \le i \le n to a complex number zi∈Cz_{i} \in \mathbb{C} is denoted by (z1,…,zn)(z_{1}, \dots, z_{n}), which belongs to Cn\mathbb{C}^{\mathbf{n}} and is also equivalent to a vector in the complex number ordered pair set Cn\mathbb{C}^{n}.

(z1,…,zn):i↦zi (z_{1}, \dots, z_{n}) : i \mapsto z_{i}

Cn:=Cn={(z1,…,zn)∣zi∈C} \mathbb{C}^{n} := \mathbb{C}^{\mathbf{n}} = \left\{ (z_{1}, \dots, z_{n}) \vert z_{i} \in \mathbb{C} \right\}

Hence, v∈CΓv \in \mathbb{C}^{\Gamma} can be seen both as a function similar to v:i↦ziv : i \mapsto z_{i} and as an ordered pair like v=(z1,…,z∣Γ∣)v = (z_{1}, \dots, z_{\left| \Gamma \right|}).

For finite sets Γ1\Gamma_{1} and Γ2\Gamma_{2}, the tensor product of two vector spaces CΓ1\mathbb{C}^{\Gamma_{1}} and CΓ2\mathbb{C}^{\Gamma_{2}} is defined as the function space (vector space) CΓ1×Γ2\mathbb{C}^{\Gamma_{1} \times \Gamma_{2}} created from the product space Γ1×Γ2\Gamma_{1} \times \Gamma_{2} of Γ1\Gamma_{1} and Γ2\Gamma_{2}.

Definition2

For finite sets Γ1\Gamma_{1} and Γ2\Gamma_{2}, the tensor product of two vector spaces CΓ1\mathbb{C}^{\Gamma_{1}} and CΓ2\mathbb{C}^{\Gamma_{2}} is defined as follows.

CΓ1⊗CΓ2:=CΓ1×Γ2 \mathbb{C}^{\Gamma_{1}} \otimes \mathbb{C}^{\Gamma_{2}} := \mathbb{C}^{\Gamma_{1} \times \Gamma_{2}}

Here, Γ1×Γ2\Gamma_{1} \times \Gamma_{2} is the product space of Γ1\Gamma_{1} and Γ2\Gamma_{2}.

Explanation

For a simple example, let’s consider Γ1=2={1,2}\Gamma_{1} = \mathbf{2} = \left\{ 1, 2 \right\} and Γ2=3={1,2,3}\Gamma_{2} = \mathbf{3} = \left\{ 1, 2, 3 \right\}. And let’s assume Γ\Gamma as their product space.

Γ=Γ1×Γ2={(1,1),(1,2),(1,3),(2,1),(2,2),(2,3)} \Gamma = \Gamma_{1} \times \Gamma_{2} = \left\{ (1,1), (1,2), (1,3), (2,1), (2,2), (2,3) \right\}

Let’s denote the elements of this as follows.

ei⊗ej=(i,j) e_{i} \otimes e_{j} = (i, j)

Following the logic from the overview, v∈CΓv \in \mathbb{C}^{\Gamma} can be seen as a function like (i,j)↦αij(i,j) \mapsto \alpha_{ij} and also as an ordered pair like (α11,a12,a13,a21,a22,a23)\left( \alpha_{11}, a_{12}, a_{13}, a_{21}, a_{22}, a_{23} \right). Therefore, CΓ\mathbb{C}^{\Gamma} is a vector space with {ei⊗ej:1≤i≤2,1≤j≤3}\left\{ e_{i} \otimes e_{j} : 1 \le i \le 2, 1 \le j \le 3 \right\} as its basis.

CΓ={∑i,jαi,jei⊗ej:αij∈C}={(α11,a12,a13,a21,a22,a23):αij∈C} \begin{align*} \mathbb{C}^{\Gamma} &= \left\{ \sum\limits_{i,j} \alpha_{i,j} e_{i} \otimes e_{j} : \alpha_{ij} \in \mathbb{C} \right\} \\ &= \left\{ \left( \alpha_{11}, a_{12}, a_{13}, a_{21}, a_{22}, a_{23} \right) : \alpha_{ij} \in \mathbb{C} \right\} \end{align*}

Hence, it is isomorphic to C6\mathbb{C}^{6}.

CΓ=C2⊗C3≅C6 \mathbb{C}^{\Gamma} = \mathbb{C}^{2} \otimes \mathbb{C}^{3} \cong \mathbb{C}^{6}

Grouping C\mathbb{C} into a product space can be thought of as increasing the position of variables, and grouping into a tensor product as increasing the position of indices.

z1∈C(z1,z2)∈C×C(z1,z2,z3)∈C×C×C z_{1} \in \mathbb{C}\qquad (z_{1},z_{2}) \in \mathbb{C} \times \mathbb{C}\qquad (z_{1}, z_{2}, z_{3}) \in \mathbb{C}\times \mathbb{C} \times \mathbb{C}

(z1,z2)∈C2(z11,z12,z21,z22)∈C2⊗C2(z111,z112,z121,z122,z211,z212,z221,z222)∈C2⊗C2⊗C2 (z_{1}, z_{2}) \in \mathbb{C}^{2} \qquad (z_{11}, z_{12}, z_{21}, z_{22}) \in \mathbb{C}^{2} \otimes \mathbb{C}^{2} \\[1em] (z_{111}, z_{112}, z_{121}, z_{122}, z_{211}, z_{212}, z_{221}, z_{222}) \in \mathbb{C}^{2} \otimes \mathbb{C}^{2} \otimes \mathbb{C}^{2}

Each ei⊗eje_{i} \otimes e_{j} corresponds to the standard basis vector of C6\mathbb{C}^{6} as follows.

e1⊗e1=[100000]e1⊗e2=[010000]e1⊗e3=[001000]e2⊗e1=[000100]e2⊗e2=[000010]e2⊗e3=[000001] e_{1} \otimes e_{1} = \begin{bmatrix}1 \\ 0 \\ 0 \\ 0 \\ 0 \\ 0\end{bmatrix} \quad e_{1} \otimes e_{2} = \begin{bmatrix}0 \\ 1 \\ 0 \\ 0 \\ 0 \\ 0\end{bmatrix} \quad e_{1} \otimes e_{3} = \begin{bmatrix}0 \\ 0 \\ 1 \\ 0 \\ 0 \\ 0\end{bmatrix} \\[2em] e_{2} \otimes e_{1} = \begin{bmatrix}0 \\ 0 \\ 0 \\ 1 \\ 0 \\ 0\end{bmatrix} \quad e_{2} \otimes e_{2} = \begin{bmatrix}0 \\ 0 \\ 0 \\ 0 \\ 1 \\ 0\end{bmatrix} \quad e_{2} \otimes e_{3} = \begin{bmatrix}0 \\ 0 \\ 0 \\ 0 \\ 0 \\ 1\end{bmatrix}

When expressed as a Kronecker product of matrices, it looks like this.

e1⊗e1=[10]⊗[100]=[1[100]0[100]]=[100000]e1⊗e2=[10]⊗[010]=[1[010]0[010]]=[010000]e1⊗e3=[10]⊗[001]=[1[001]0[001]]=[001000]e2⊗e1=[01]⊗[100]=[0[100]1[100]]=[000100]e2⊗e2=[01]⊗[010]=[0[010]1[010]]=[000010]e2⊗e3=[01]⊗[001]=[0[001]1[001]]=[000001] e_{1} \otimes e_{1} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \otimes \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} = \begin{bmatrix} 1 \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \\[1.5em] 0 \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix} \quad e_{1} \otimes e_{2} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \otimes \begin{bmatrix} 0 \\ 1 \\ 0\end{bmatrix} = \begin{bmatrix} 1 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \\[1.5em] 0 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 0 \\ 1 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix} \\[2em] e_{1} \otimes e_{3} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \otimes \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 1 \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \\[1.5em] 0 \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 1 \\ 0 \\ 0 \\ 0 \end{bmatrix} \quad e_{2} \otimes e_{1} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \otimes \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \\[1.5em] 1 \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \\ 1 \\ 0 \\ 0 \end{bmatrix} \\[2em] e_{2} \otimes e_{2} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \otimes \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \\[1.5em] 1 \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 1 \\ 0 \end{bmatrix} \quad e_{2} \otimes e_{3} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \otimes \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \\[1.5em] 1 \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \\ 1 \end{bmatrix}

Moreover, from this, it can be understood that the following holds.

C⊗Cn≅CnC⊗C≅C \mathbb{C} \otimes \mathbb{C}^{n} \cong \mathbb{C}^{n} \qquad \mathbb{C} \otimes \mathbb{C} \cong \mathbb{C}

Properties

  1. Cn⊗Cm\mathbb{C}^{n} \otimes \mathbb{C}^{m} is a vector space with respect to the following two operations:
    • (x1⊗y1)+(x2⊗y2)=(x1+x2)⊗(y1+y2)(x_{1} \otimes y_{1}) + (x_{2} \otimes y_{2}) = (x_{1} + x_{2}) \otimes (y_{1} + y_{2})
    • α(x⊗y)=(αx)⊗y=x⊗(αy)\alpha (x \otimes y) = (\alpha x) \otimes y = x \otimes (\alpha y)
  2. Cn⊗Cm≅Cnm\mathbb{C}^{n} \otimes \mathbb{C}^{m} \cong \mathbb{C}^{nm}
  3. dim⁡(Cn⊗Cm)=dim⁡(Cn)⋅dim⁡(Cm)=nm\dim (\mathbb{C}^{n} \otimes \mathbb{C}^{m}) = \dim(\mathbb{C}^{n}) \cdot \dim(\mathbb{C}^{m}) = nm

Generalization

For finite sets Γi(1≤i≤r)\Gamma_{i} (1 \le i \le r), the tensor product of vector spaces CΓi\mathbb{C}^{\Gamma_{i}} is defined as follows.

CΓ1⊗⋯⊗CΓr:=CΓ=CΓ1×⋯×Γr,Γ=Γ1×⋯×Γr \mathbb{C}^{\Gamma_{1}} \otimes \cdots \otimes \mathbb{C}^{\Gamma_{r}} := \mathbb{C}^{\Gamma} = \mathbb{C}^{\Gamma_{1} \times \cdots \times \Gamma_{r}},\quad \Gamma = \Gamma_{1} \times \cdots \times \Gamma_{r}

The basis vectors corresponding to (j1,…jr)∈∏iΓi(j_{1}, \dots j_{r}) \in \prod\limits_{i} \Gamma_{i} are denoted as follows: (1≤ji≤∣Γi∣)( 1 \le j_{i} \le \left| \Gamma_{i} \right|)

ej1⊗⋯⊗ejr e_{j_{1}} \otimes \cdots \otimes e_{j_{r}}

Then, the tensor product is a vector space like the following. If the cardinality of Γi\Gamma_{i} is denoted by ni=∣Γi∣n_{i} = \left| \Gamma_{i} \right|,

CΓ1⊗⋯⊗CΓr={∑(j1,…jr)∈∏iΓiαj1,…,jrej1⊗⋯⊗ejr}={(α1,…,1, …,αn1, …,nr):α∈C}≅C∣Γ∣ \begin{align*} \mathbb{C}^{\Gamma_{1}} \otimes \cdots \otimes \mathbb{C}^{\Gamma_{r}} &= \left\{ \sum\limits_{(j_{1}, \dots j_{r}) \in \prod\limits_{i} \Gamma_{i}} \alpha_{j_{1}, \dots, j_{r}} e_{j_{1}} \otimes \cdots \otimes e_{j_{r}}\right\} \\ &= \left\{ (\alpha_{1,\dots,1},\ \dots, \alpha_{n_{1},\ \dots, n_{r}}) : \alpha \in \mathbb{C} \right\} \\ &\cong \mathbb{C}^{\left| \Gamma \right|} \end{align*}

General Vector Spaces

Given a finite-dimensional vector space V1,…,VrV_{1}, \dots, V_{r}, by selecting a basis Bi={v1,v2,…,vdim⁡Vi}\mathcal{B}_{i} = \left\{ v_{1}, v_{2}, \dots, v_{\dim V_{i}} \right\} of vector space ViV_{i}, one can obtain the following bijection fif_{i}.

fi:Vi→Cdim⁡Vi∑zjvj↦(z1,…,zdim⁡Vi) \begin{align*} f _{i}: & V_{i} \to \mathbb{C}^{\dim V_{i}} \\ & \sum z_{j}v_{j} \mapsto (z_{1}, \dots, z_{\dim V_{i}}) \end{align*}

Then, the tensor product of ViV_{i} is defined as follows.

⨂i=1rVi=V1⊗⋯⊗Vr:=Cdim⁡V1⊗⋯⊗Cdim⁡Vr \bigotimes\limits_{i=1}^{r} V_{i} = V_{1} \otimes \cdots \otimes V_{r} := \mathbb{C}^{\dim V_{1}} \otimes \cdots \otimes \mathbb{C}^{\dim V_{r}}

See Also

  1. 김영훈·허재성, 양자 정보 이론 (2020), p3 ↩︎

  2. 김영훈·허재성, 양자 정보 이론 (2020), p31 ↩︎