Outer Product of Two Vectors
📂Matrix Algebra Outer Product of Two Vectors Definition The outer product of two column vectors u = [ u 1 ⋮ u n ] \mathbf{u} = \begin{bmatrix} u_{1} \\ \vdots \\ u_{n} \end{bmatrix} u = u 1 ⋮ u n and v = [ v 1 ⋮ v n ] \mathbf{v} = \begin{bmatrix} v_{1} \\ \vdots \\ v_{n} \end{bmatrix} v = v 1 ⋮ v n is defined as follows.
u ⊗ v = u v T = [ u 1 u 2 ⋮ u n ] [ v 1 v 2 ⋯ v n ] = [ u 1 v 1 u 1 v 2 ⋯ u 1 v n u 2 v 1 u 2 v 2 ⋯ u 2 v n ⋮ ⋮ ⋱ ⋮ u n v 1 u n v 2 ⋯ u n v n ]
\mathbf{u} \otimes \mathbf{v} =
\mathbf{u}\mathbf{v}^{\mathsf{T}} =
\begin{bmatrix}
u_{1} \\
u_{2} \\
\vdots \\
u_{n}
\end{bmatrix}
\begin{bmatrix}
v_{1} & v_{2} & \cdots & v_{n}
\end{bmatrix}=
\begin{bmatrix}
u_{1}v_{1} & u_{1}v_{2} & \cdots & u_{1}v_{n} \\
u_{2}v_{1} & u_{2}v_{2} & \cdots & u_{2}v_{n} \\
\vdots & \vdots & \ddots & \vdots \\
u_{n}v_{1} & u_{n}v_{2} & \cdots & u_{n}v_{n}
\end{bmatrix}
u ⊗ v = u v T = u 1 u 2 ⋮ u n [ v 1 v 2 ⋯ v n ] = u 1 v 1 u 2 v 1 ⋮ u n v 1 u 1 v 2 u 2 v 2 ⋮ u n v 2 ⋯ ⋯ ⋱ ⋯ u 1 v n u 2 v n ⋮ u n v n
Here, T {}^{\mathsf{T}} T is the transpose of a matrix.
Explanation There are various operations involving vectors and matrices, so care must be taken to avoid confusion.
The cross product defined in 3-dimensional space is often translated as an outer product, requiring attention. Considering the English term and its properties, cross product can be appropriately termed as a vector product or a wedge product. It can be seen as a special case of the Kronecker product (tensor product) . A ⊗ B A \otimes B A ⊗ B is when A A A is a column vector , and B B B is a row vector . It can also be seen as a special case of matrix multiplication . A × B A \times B A × B is when A A A is a column vector and B B B is a row vector. The scalar product (dot product, inner product) of two vectors results in a scalar, whereas the vector product of two vectors results in a vector. The outer product of two vectors results in a matrix (tensor ).
Operation Scalar Product (Inner Product) Vector Product Outer Product (Tensor Product) Dimension n n n -dimensional vector3 3 3 -dimensional vectorn n n -dimensional vectorNotation u ⋅ v = u T v \mathbf{u} \cdot \mathbf{v} = \mathbf{u}^{\mathsf{T}} \mathbf{v} u ⋅ v = u T v u × v \mathbf{u} \times \mathbf{v} u × v u ⊗ v = u v T \mathbf{u} \otimes \mathbf{v} = \mathbf{u}\mathbf{v}^{\mathsf{T}} u ⊗ v = u v T Result Scalar = 1 × 1 =1 \times 1 = 1 × 1 matrix 3 3 3 -dimensional vectorn × n n \times n n × n matrixValue ∑ i u i v i = u 1 v 1 + ⋯ + u n v n \sum_{i} u_{i}v_{i} = u_{1}v_{1} + \cdots + u_{n}v_{n} ∑ i u i v i = u 1 v 1 + ⋯ + u n v n [ u 2 v 3 − u 3 v 2 u 3 v 1 − u 1 v 3 u 1 v 2 − u 2 v 1 ] \begin{bmatrix} u_{2}v_{3} - u_{3}v_{2} \\ u_{3}v_{1} - u_{1}v_{3} \\ u_{1}v_{2} - u_{2}v_{1} \end{bmatrix} u 2 v 3 − u 3 v 2 u 3 v 1 − u 1 v 3 u 1 v 2 − u 2 v 1 [ u 1 v 1 u 1 v 2 ⋯ u 1 v n u 2 v 1 u 2 v 2 ⋯ u 2 v n ⋮ ⋮ ⋱ ⋮ u n v 1 u n v 2 ⋯ u n v n ] \begin{bmatrix} u_{1}v_{1} & u_{1}v_{2} & \cdots & u_{1}v_{n} \\ u_{2}v_{1} & u_{2}v_{2} & \cdots & u_{2}v_{n} \\ \vdots & \vdots & \ddots & \vdots \\ u_{n}v_{1} & u_{n}v_{2} & \cdots & u_{n}v_{n}\end{bmatrix} u 1 v 1 u 2 v 1 ⋮ u n v 1 u 1 v 2 u 2 v 2 ⋮ u n v 2 ⋯ ⋯ ⋱ ⋯ u 1 v n u 2 v n ⋮ u n v n
Generalization In fact, unlike scalar and vector products, the sizes of the two vectors involved in this operation do not need to be the same for it to be well-defined. For example, the outer product of two vectors u = [ u 1 ⋮ u n ] \mathbf{u} = \begin{bmatrix} u_{1} \\ \vdots \\ u_{n} \end{bmatrix} u = u 1 ⋮ u n and v = [ v 1 ⋮ v m ] \mathbf{v} = \begin{bmatrix} v_{1} \\ \vdots \\ v_{m} \end{bmatrix} v = v 1 ⋮ v m can be defined as follows.
u ⊗ v = [ u 1 ⋮ u n ] [ v 1 ⋯ v m ] T = [ u 1 v 1 u 1 v 2 ⋯ u 1 v m u 2 v 1 u 2 v 2 ⋯ u 2 v m ⋮ ⋮ ⋱ ⋮ u n v 1 u n v 2 ⋯ u n v m ]
\mathbf{u} \otimes \mathbf{v} = \begin{bmatrix} u_{1} \\ \vdots \\ u_{n} \end{bmatrix} \begin{bmatrix} v_{1} & \cdots & v_{m} \end{bmatrix}^{\mathsf{T}} =
\begin{bmatrix}
u_{1}v_{1} & u_{1}v_{2} & \cdots & u_{1}v_{m} \\
u_{2}v_{1} & u_{2}v_{2} & \cdots & u_{2}v_{m} \\
\vdots & \vdots & \ddots & \vdots \\
u_{n}v_{1} & u_{n}v_{2} & \cdots & u_{n}v_{m}
\end{bmatrix}
u ⊗ v = u 1 ⋮ u n [ v 1 ⋯ v m ] T = u 1 v 1 u 2 v 1 ⋮ u n v 1 u 1 v 2 u 2 v 2 ⋮ u n v 2 ⋯ ⋯ ⋱ ⋯ u 1 v m u 2 v m ⋮ u n v m
Properties Let u = [ u 1 ⋯ u n ] T \mathbf{u} = \begin{bmatrix} u_{1} & \cdots & u_{n} \end{bmatrix}^{\mathsf{T}} u = [ u 1 ⋯ u n ] T , v = [ v 1 ⋯ v n ] T \mathbf{v} = \begin{bmatrix} v_{1} & \cdots & v_{n} \end{bmatrix}^{\mathsf{T}} v = [ v 1 ⋯ v n ] T , and w = [ w 1 ⋯ w n ] T \mathbf{w} = \begin{bmatrix} w_{1} & \cdots & w_{n} \end{bmatrix}^{\mathsf{T}} w = [ w 1 ⋯ w n ] T . The following holds.
( u ⊗ v ) T = v ⊗ u (1)
(\mathbf{u} \otimes \mathbf{v})^{\mathsf{T}} = \mathbf{v} \otimes \mathbf{u}
\tag{1}
( u ⊗ v ) T = v ⊗ u ( 1 )
( 2 ) (2) ( 2 ) Linearity:
( v + w ) ⊗ u = v ⊗ u + w ⊗ u
(\mathbf{v} + \mathbf{w}) \otimes \mathbf{u} = \mathbf{v} \otimes \mathbf{u} + \mathbf{w} \otimes \mathbf{u}
( v + w ) ⊗ u = v ⊗ u + w ⊗ u
u ⊗ ( v + w ) = u ⊗ v + u ⊗ w
\mathbf{u} \otimes (\mathbf{v} + \mathbf{w}) = \mathbf{u} \otimes \mathbf{v} + \mathbf{u} \otimes \mathbf{w}
u ⊗ ( v + w ) = u ⊗ v + u ⊗ w
Let α ∈ R \alpha \in \mathbb{R} α ∈ R be a constant.
( α v ) ⊗ u = α ( v ⊗ u ) = ( α v ) ⊗ u
(\alpha \mathbf{v}) \otimes \mathbf{u} = \alpha (\mathbf{v} \otimes \mathbf{u}) = (\alpha \mathbf{v}) \otimes \mathbf{u}
( α v ) ⊗ u = α ( v ⊗ u ) = ( α v ) ⊗ u
( u ⊗ v ) w = ( v ⋅ w ) u (3)
(\mathbf{u} \otimes \mathbf{v}) \mathbf{w} = (\mathbf{v} \cdot \mathbf{w}) \mathbf{u}
\tag{3}
( u ⊗ v ) w = ( v ⋅ w ) u ( 3 )
w T ( u ⊗ v ) = ( w ⋅ u ) v T (4)
\mathbf{w}^{\mathsf{T}} (\mathbf{u} \otimes \mathbf{v}) = (\mathbf{w} \cdot \mathbf{u}) \mathbf{v}^{\mathsf{T}}
\tag{4}
w T ( u ⊗ v ) = ( w ⋅ u ) v T ( 4 )
( 5 ) (5) ( 5 ) Associativity:
When ⊗ \otimes ⊗ is extended by Kronecker product , the following holds.
( u ⊗ Kron v ) ⊗ Kron w = u ⊗ Kron ( v ⊗ Kron w )
(\mathbf{u} \otimes_{\text{Kron}} \mathbf{v}) \otimes_{\text{Kron}} \mathbf{w} = \mathbf{u} \otimes_{\text{Kron}} (\mathbf{v} \otimes_{\text{Kron}} \mathbf{w})
( u ⊗ Kron v ) ⊗ Kron w = u ⊗ Kron ( v ⊗ Kron w )
Proof ( 1 ) (1) ( 1 ) This can be seen from the properties of the transpose .
( u ⊗ v ) T = ( u v T ) T = v u T = v ⊗ u
\begin{align*}
(\mathbf{u} \otimes \mathbf{v})^{\mathsf{T}}
&= (\mathbf{u} \mathbf{v}^{\mathsf{T}})^{\mathsf{T}} \\
&= \mathbf{v} \mathbf{u}^{\mathsf{T}} \\
&= \mathbf{v} \otimes \mathbf{u} \\
\end{align*}
( u ⊗ v ) T = ( u v T ) T = v u T = v ⊗ u
( 2 ) (2) ( 2 ) This holds since the matrix multiplication is linear.
( v + w ) ⊗ u = ( v + w ) u T = v u T + w u T = v ⊗ u + w ⊗ u
\begin{align*}
(\mathbf{v} + \mathbf{w}) \otimes \mathbf{u}
&= (\mathbf{v} + \mathbf{w}) \mathbf{u}^{\mathsf{T}} \\
&= \mathbf{v} \mathbf{u}^{\mathsf{T}} + \mathbf{w} \mathbf{u}^{\mathsf{T}} \\
&= \mathbf{v} \otimes \mathbf{u} + \mathbf{w} \otimes \mathbf{u} \\
\end{align*}
( v + w ) ⊗ u = ( v + w ) u T = v u T + w u T = v ⊗ u + w ⊗ u
This holds since the transpose is linear.
u ⊗ ( v + w ) = u ( v + w ) T = u ( v T + w T ) = u v T + u w T = u ⊗ v + u ⊗ w
\begin{align*}
\mathbf{u} \otimes (\mathbf{v} + \mathbf{w})
&= \mathbf{u} (\mathbf{v} + \mathbf{w})^{\mathsf{T}} \\
&= \mathbf{u} (\mathbf{v}^{\mathsf{T}} + \mathbf{w}^{\mathsf{T}}) \\
&= \mathbf{u} \mathbf{v}^{\mathsf{T}} + \mathbf{u} \mathbf{w}^{\mathsf{T}} \\
&= \mathbf{u} \otimes \mathbf{v} + \mathbf{u} \otimes \mathbf{w} \\
\end{align*}
u ⊗ ( v + w ) = u ( v + w ) T = u ( v T + w T ) = u v T + u w T = u ⊗ v + u ⊗ w
( α v ) ⊗ u = ( α v ) u T = α ( v u T ) = α ( v ⊗ u )
\begin{align*}
(\alpha \mathbf{v}) \otimes \mathbf{u}
&= (\alpha \mathbf{v}) \mathbf{u}^{\mathsf{T}} \\
&= \alpha (\mathbf{v} \mathbf{u}^{\mathsf{T}}) \\
&= \alpha (\mathbf{v} \otimes \mathbf{u}) \\
\end{align*}
( α v ) ⊗ u = ( α v ) u T = α ( v u T ) = α ( v ⊗ u )
■
( 3 ) (3) ( 3 ) ( u ⊗ v ) w = ( u v T ) w = u ( v T w ) = u ( v ⋅ w ) = ( v ⋅ w ) u
\begin{align*}
(\mathbf{u} \otimes \mathbf{v}) \mathbf{w}
&= (\mathbf{u} \mathbf{v}^{\mathsf{T}}) \mathbf{w} \\
&= \mathbf{u} (\mathbf{v}^{\mathsf{T}} \mathbf{w}) \\
&= \mathbf{u} (\mathbf{v} \cdot \mathbf{w}) \\
&= (\mathbf{v} \cdot \mathbf{w}) \mathbf{u} \\
\end{align*}
( u ⊗ v ) w = ( u v T ) w = u ( v T w ) = u ( v ⋅ w ) = ( v ⋅ w ) u
Note that v T w \mathbf{v}^{\mathsf{T}} \mathbf{w} v T w is a scalar.
■
( 4 ) (4) ( 4 ) w T ( u ⊗ v ) = w T ( u v T ) = ( w T u ) v T = ( w ⋅ u ) v T
\begin{align*}
\mathbf{w}^{\mathsf{T}} (\mathbf{u} \otimes \mathbf{v})
&= \mathbf{w}^{\mathsf{T}} (\mathbf{u} \mathbf{v}^{\mathsf{T}}) \\
&= (\mathbf{w}^{\mathsf{T}} \mathbf{u}) \mathbf{v}^{\mathsf{T}} \\
&= (\mathbf{w} \cdot \mathbf{u}) \mathbf{v}^{\mathsf{T}} \\
\end{align*}
w T ( u ⊗ v ) = w T ( u v T ) = ( w T u ) v T = ( w ⋅ u ) v T
■