📂Linear Algebra

Matrix Representation of Tensor Product

선형대수

[ 펼치기 · 접기 ] 벡터공간 벡터공간 벡터공간 · 부분공간 · 합$+$ · 직합$\oplus$ · 선형결합 · 생성 · 선형 독립/종속 · 론스키안 기저 기저 · 순서기저 · 차원 · 좌표벡터 · 벡터의 좌표변환 · 선형변환의 좌표변환 잉여류 잉여류와 몫공간 · 몫공간의 기저 · 몫공간으로의 사상 · 몫공간 위의 작용소 선형변환 내용 · 내용 · 내용 고유값 내용 · 내용 · 내용 쌍대공간 내용 · 내용 · 내용 텐서 벡터공간의 텐서곱 · 곱 벡터 · 보편성질 · 선형변환의 텐서곱(행렬표현) · 쌍대공간 · 전치 내적공간 내용 · 내용 · 내용 관련 분야 행렬대수 · 바나흐 공간 · 힐베르트 공간

Buildup¹

Choose bases $\mathcal{V}, {\mathcal{V}}^{\prime}$ respectively for the finite-dimensional vector spaces $V, V^{\prime}$. Then, there exists a matrix equivalent to the linear transformation $\phi : V \to V^{\prime}$, called its matrix representation $\phi$. Now assume we have the finite-dimensional vector space $V, V^{\prime}, W, W^{\prime}$ and its ordered basis $\mathcal{V}, {\mathcal{V}}^{\prime}, \mathcal{W}, {\mathcal{W}}^{\prime}$, as well as two linear transformations $\phi : V \to V^{\prime}$ and $\psi : W \to W^{\prime}$.

$$ n = \dim V,\quad m = \dim V^{\prime},\quad p = \dim W,\quad q = \dim W^{\prime} $$

$$ \mathcal{V} = \left\{ v_{i} \right\}_{i=1}^{n},\quad {\mathcal{V}}^{\prime} = \left\{ v_{j}^{\prime} \right\}_{j=1}^{m},\quad \mathcal{W} = \left\{ w_{k} \right\}_{k=1}^{p},\quad {\mathcal{W}}^{\prime} = \left\{ w_{l}^{\prime} \right\}_{l=1}^{q} $$

There exist matrix representations for the two linear transformations $\phi$ and $\psi$ as follows.

$$ A = \begin{bmatrix} \phi \end{bmatrix}_{\mathcal{V}}^{{\mathcal{V}}^{\prime}}\in M_{m \times n} \qquad B = \begin{bmatrix} \psi \end{bmatrix}_{\mathcal{W}}^{{\mathcal{W}}^{\prime}} \in M_{q \times p} $$

Notate the ordered basis of the tensor product $V \otimes W$ as $\mathcal{V} \otimes \mathcal{W} = \left\{ v_{i} \otimes w_{k} \right\}$, and order it as follows.

$$ v_{1} \otimes w_{1}, \dots, v_{1} \times w_{p}, \\ v_{2}\otimes w_{1}, \dots, v_{2} \times w_{p}, \\ \dots \\ v_{n}\otimes w_{1}, \dots, v_{n} \times w_{p} $$

Let’s give an order to the basis $\mathcal{V}^{\prime} \otimes \mathcal{W}^{\prime} = \left\{ v_{j}^{\prime} \otimes w_{l}^{\prime} \right\}$ of $V^{\prime} \otimes W^{\prime}$ in the same manner. Then, since the tensor product of $\phi$ and $\psi$ is also the linear transformation $\phi \otimes \psi : V \otimes W \to V^{\prime} \otimes W^{\prime}$, it exists as a matrix representation as shown below.

$$ \begin{bmatrix} \phi \otimes \psi \end{bmatrix}_{\mathcal{V} \otimes \mathcal{W}}^{{\mathcal{V}}^{\prime} \otimes \mathcal{W}^{\prime}} $$

Theorem

Let the matrix representations of the two linear transformations $\phi : V \to V^{\prime}$ and $\psi : W \to W^{\prime}$ be $A = \begin{bmatrix} \phi \end{bmatrix}_{\mathcal{V}}^{{\mathcal{V}}^{\prime}}$ and $B = \begin{bmatrix} \psi \end{bmatrix}_{\mathcal{W}}^{{\mathcal{W}}^{\prime}}$, respectively. The matrix representation of the tensor product $\phi \otimes \psi : V \otimes W \to V^{\prime} \otimes W^{\prime}$ is the same as the Kronecker product of $A$ and $B$.

$$ \begin{bmatrix} \phi \otimes \psi \end{bmatrix}_{\mathcal{V} \otimes \mathcal{W}}^{{\mathcal{V}}^{\prime} \otimes \mathcal{W}^{\prime}} = A \otimes B = \begin{bmatrix} \phi \end{bmatrix}_{\mathcal{V}}^{{\mathcal{V}}^{\prime}} \otimes \begin{bmatrix} \psi \end{bmatrix}_{\mathcal{W}}^{{\mathcal{W}}^{\prime}} $$

Proof

To find the matrix representation $\begin{bmatrix} \phi \otimes \psi \end{bmatrix}_{\mathcal{V} \otimes \mathcal{W}}^{{\mathcal{V}}^{\prime} \otimes \mathcal{W}^{\prime}}$, we must see how the basis $\mathcal{V} \otimes \mathcal{W}$ of the domain is mapped by $\phi \otimes \psi$. Suppose the matrix representations of the two linear transformations are as follows.

$$ \begin{bmatrix} \phi \end{bmatrix}_{\mathcal{V}}^{{\mathcal{V}}^{\prime}} = A = [ \alpha_{ji} ] \in M_{m \times n} \qquad \begin{bmatrix} \psi \end{bmatrix}_{\mathcal{W}}^{{\mathcal{W}}^{\prime}} = B = [ \beta_{lk} ] \in M_{q \times p} $$

Which means $\phi (v_{i}) = \sum\limits_{j}\alpha_{ji}v_{j}^{\prime}$ and $\psi (w_{k}) = \sum\limits_{l}\beta_{lk}w_{k}^{\prime}$. By the definition of tensor product of linear transformations and product vector, the basis vector $v_{i} \otimes w_{k}$ is mapped as follows.

$$ \begin{align*} (\phi \otimes \psi)(v_{i} \otimes w_{k}) &= \phi (v_{i}) \otimes \psi (w_{k}) \\ &= \left( \sum\limits_{j}\alpha_{ji}v_{j}^{\prime} \right) \otimes \left( \sum\limits_{l}\beta_{lk}w_{l}^{\prime} \right) \\ &= \sum_{j,l} \alpha_{ji}\beta_{lk} v_{j}^{\prime} \otimes w_{l}^{\prime} \\ \end{align*} $$

$$ \implies \left[ (\phi \otimes \psi)(v_{i} \otimes w_{k}) \right]_{{\mathcal{V}}^{\prime} \otimes \mathcal{W}^{\prime}} = \begin{bmatrix} \alpha_{1i}\beta_{1k} \\ \alpha_{1i}\beta_{2k} \\ \vdots \\ \alpha_{1i}\beta_{qk} \\ \alpha_{2i}\beta_{1k} \\ \alpha_{2i}\beta_{2k} \\ \vdots \\ \alpha_{2i}\beta_{qk} \\ \vdots \\ \alpha_{mi}\beta_{1k} \\ \alpha_{mi}\beta_{2k} \\ \vdots \\ \alpha_{mi}\beta_{qk} \\ \end{bmatrix} $$

Thus, summarizing we get the following.

$$ \begin{align*} & \begin{bmatrix} \phi \otimes \psi \end{bmatrix}_{\mathcal{V} \otimes \mathcal{W}}^{{\mathcal{V}}^{\prime} \otimes \mathcal{W}^{\prime}} \\ &= \begin{bmatrix} \left[ \phi (v_{1}) \otimes \psi (w_{1}) \right]_{{\mathcal{V}}^{\prime} \otimes \mathcal{W}^{\prime}} & \left[ \phi (v_{1}) \otimes \psi (w_{2}) \right]_{{\mathcal{V}}^{\prime} \otimes \mathcal{W}^{\prime}} & \cdots & \left[ \phi (v_{n}) \otimes \psi (w_{p}) \right]_{{\mathcal{V}}^{\prime} \otimes \mathcal{W}^{\prime}} \end{bmatrix} \\ &= \left[ \begin{array}{cccc|c|cccc} \alpha_{11}\beta_{11} & \alpha_{11}\beta_{12} & \cdots & \alpha_{11}\beta_{1p} & \cdots & \alpha_{1n}\beta_{11} & \alpha_{1n}\beta_{12} & \cdots & \alpha_{1n}\beta_{1p} & \\ \alpha_{11}\beta_{21} & \alpha_{11}\beta_{22} & \cdots & \alpha_{11}\beta_{2p} & \cdots & \alpha_{1n}\beta_{21} & \alpha_{1n}\beta_{22} & \cdots & \alpha_{1n}\beta_{2p} & \\ \vdots & \vdots & \ddots & \vdots & \cdots & \vdots & \vdots & \ddots & \vdots \\ \alpha_{11}\beta_{q1} & \alpha_{11}\beta_{q2} & \cdots & \alpha_{11}\beta_{qp} & \cdots & \alpha_{1n}\beta_{q1} & \alpha_{1n}\beta_{q2} & \cdots & \alpha_{1n}\beta_{qp} & \\ \hline \alpha_{21}\beta_{11} & \alpha_{21}\beta_{12} & \cdots & \alpha_{21}\beta_{1p} & \cdots & \alpha_{2n}\beta_{11} & \alpha_{2n}\beta_{12} & \cdots & \alpha_{2n}\beta_{1p} & \\ \alpha_{21}\beta_{21} & \alpha_{21}\beta_{22} & \cdots & \alpha_{21}\beta_{2p} & \cdots & \alpha_{2n}\beta_{21} & \alpha_{2n}\beta_{22} & \cdots & \alpha_{2n}\beta_{2p} & \\ \vdots & \vdots & \ddots & \vdots & \cdots & \vdots & \vdots & \ddots & \vdots \\ \alpha_{21}\beta_{q1} & \alpha_{21}\beta_{q2} & \cdots & \alpha_{21}\beta_{qp} & \cdots & \alpha_{2n}\beta_{q1} & \alpha_{2n}\beta_{q2} & \cdots & \alpha_{2n}\beta_{qp} & \\ \hline \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots & \vdots \\ \hline \alpha_{m1}\beta_{11} & \alpha_{m1}\beta_{12} & \cdots & \alpha_{m1}\beta_{1p} & \cdots & \alpha_{mn}\beta_{11} & \alpha_{mn}\beta_{12} & \cdots & \alpha_{mn}\beta_{1p} & \\ \alpha_{m1}\beta_{21} & \alpha_{m1}\beta_{22} & \cdots & \alpha_{m1}\beta_{2p} & \cdots & \alpha_{mn}\beta_{21} & \alpha_{mn}\beta_{22} & \cdots & \alpha_{mn}\beta_{2p} & \\ \vdots & \vdots & \ddots & \vdots & \cdots & \vdots & \vdots & \ddots & \vdots \\ \alpha_{m1}\beta_{q1} & \alpha_{m1}\beta_{q2} & \cdots & \alpha_{m1}\beta_{qp} & \cdots & \alpha_{mn}\beta_{q1} & \alpha_{mn}\beta_{q2} & \cdots & \alpha_{mn}\beta_{qp} & \\ \end{array} \right] \\ &= \left[ \begin{array}{c} \alpha_{11} \begin{bmatrix} \beta_{11} & \beta_{12} & \cdots & \beta_{1p} \\ \beta_{21} & \beta_{22} & \cdots & \beta_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ \beta_{q1} & \beta_{q2} & \cdots & \beta_{qp} \end{bmatrix} & \cdots & \alpha_{1n}\begin{bmatrix} \beta_{11} & \beta_{12} & \cdots & \beta_{1p} \\ \beta_{21} & \beta_{22} & \cdots & \beta_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ \beta_{q1} & \beta_{q2} & \cdots & \beta_{qp} \end{bmatrix} \\ \alpha_{21} \begin{bmatrix} \beta_{11} & \beta_{12} & \cdots & \beta_{1p} \\ \beta_{21} & \beta_{22} & \cdots & \beta_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ \beta_{q1} & \beta_{q2} & \cdots & \beta_{qp} \end{bmatrix} & \cdots & \alpha_{2n}\begin{bmatrix} \beta_{11} & \beta_{12} & \cdots & \beta_{1p} \\ \beta_{21} & \beta_{22} & \cdots & \beta_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ \beta_{q1} & \beta_{q2} & \cdots & \beta_{qp} \end{bmatrix} \\ \vdots & \ddots & \vdots \\ \alpha_{m1} \begin{bmatrix} \beta_{11} & \beta_{12} & \cdots & \beta_{1p} \\ \beta_{21} & \beta_{22} & \cdots & \beta_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ \beta_{q1} & \beta_{q2} & \cdots & \beta_{qp} \end{bmatrix} & \cdots & \alpha_{mn}\begin{bmatrix} \beta_{11} & \beta_{12} & \cdots & \beta_{1p} \\ \beta_{21} & \beta_{22} & \cdots & \beta_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ \beta_{q1} & \beta_{q2} & \cdots & \beta_{qp} \end{bmatrix} \\ \end{array} \right] \\ &= \begin{bmatrix} \alpha_{11} B & \cdots & \alpha_{1n} B \\ \alpha_{21} B & \cdots & \alpha_{2n} B \\ \vdots & \ddots & \vdots \\ \alpha_{m1} B & \cdots & \alpha_{mn} B \end{bmatrix} \\ &= A \otimes B \end{align*} $$

■