logo

Outer Product of Two Vectors 📂Matrix Algebra

Outer Product of Two Vectors

Definition

The outer product of two column vectors u=[u1un]\mathbf{u} = \begin{bmatrix} u_{1} \\ \vdots \\ u_{n} \end{bmatrix} and v=[v1vn]\mathbf{v} = \begin{bmatrix} v_{1} \\ \vdots \\ v_{n} \end{bmatrix} is defined as follows.

uv=uvT=[u1u2un][v1v2vn]=[u1v1u1v2u1vnu2v1u2v2u2vnunv1unv2unvn] \mathbf{u} \otimes \mathbf{v} = \mathbf{u}\mathbf{v}^{\mathsf{T}} = \begin{bmatrix} u_{1} \\ u_{2} \\ \vdots \\ u_{n} \end{bmatrix} \begin{bmatrix} v_{1} & v_{2} & \cdots & v_{n} \end{bmatrix}= \begin{bmatrix} u_{1}v_{1} & u_{1}v_{2} & \cdots & u_{1}v_{n} \\ u_{2}v_{1} & u_{2}v_{2} & \cdots & u_{2}v_{n} \\ \vdots & \vdots & \ddots & \vdots \\ u_{n}v_{1} & u_{n}v_{2} & \cdots & u_{n}v_{n} \end{bmatrix}

Here, T{}^{\mathsf{T}} is the transpose of a matrix.

Explanation

There are various operations involving vectors and matrices, so care must be taken to avoid confusion.

  • The cross product defined in 3-dimensional space is often translated as an outer product, requiring attention. Considering the English term and its properties, cross product can be appropriately termed as a vector product or a wedge product.
  • It can be seen as a special case of the Kronecker product (tensor product). ABA \otimes B is when AA is a column vector, and BB is a row vector.
  • It can also be seen as a special case of matrix multiplication. A×BA \times B is when AA is a column vector and BB is a row vector.

The scalar product (dot product, inner product) of two vectors results in a scalar, whereas the vector product of two vectors results in a vector. The outer product of two vectors results in a matrix (tensor).

OperationScalar Product (Inner Product)Vector ProductOuter Product (Tensor Product)
Dimensionnn-dimensional vector33-dimensional vectornn-dimensional vector
Notationuv=uTv\mathbf{u} \cdot \mathbf{v} = \mathbf{u}^{\mathsf{T}} \mathbf{v}u×v\mathbf{u} \times \mathbf{v}uv=uvT\mathbf{u} \otimes \mathbf{v} = \mathbf{u}\mathbf{v}^{\mathsf{T}}
ResultScalar =1×1=1 \times 1 matrix33-dimensional vectorn×nn \times n matrix
Valueiuivi=u1v1++unvn\sum_{i} u_{i}v_{i} = u_{1}v_{1} + \cdots + u_{n}v_{n}[u2v3u3v2u3v1u1v3u1v2u2v1]\begin{bmatrix} u_{2}v_{3} - u_{3}v_{2} \\ u_{3}v_{1} - u_{1}v_{3} \\ u_{1}v_{2} - u_{2}v_{1} \end{bmatrix}[u1v1u1v2u1vnu2v1u2v2u2vnunv1unv2unvn]\begin{bmatrix} u_{1}v_{1} & u_{1}v_{2} & \cdots & u_{1}v_{n} \\ u_{2}v_{1} & u_{2}v_{2} & \cdots & u_{2}v_{n} \\ \vdots & \vdots & \ddots & \vdots \\ u_{n}v_{1} & u_{n}v_{2} & \cdots & u_{n}v_{n}\end{bmatrix}

Generalization

In fact, unlike scalar and vector products, the sizes of the two vectors involved in this operation do not need to be the same for it to be well-defined. For example, the outer product of two vectors u=[u1un]\mathbf{u} = \begin{bmatrix} u_{1} \\ \vdots \\ u_{n} \end{bmatrix} and v=[v1vm]\mathbf{v} = \begin{bmatrix} v_{1} \\ \vdots \\ v_{m} \end{bmatrix} can be defined as follows.

uv=[u1un][v1vm]T=[u1v1u1v2u1vmu2v1u2v2u2vmunv1unv2unvm] \mathbf{u} \otimes \mathbf{v} = \begin{bmatrix} u_{1} \\ \vdots \\ u_{n} \end{bmatrix} \begin{bmatrix} v_{1} & \cdots & v_{m} \end{bmatrix}^{\mathsf{T}} = \begin{bmatrix} u_{1}v_{1} & u_{1}v_{2} & \cdots & u_{1}v_{m} \\ u_{2}v_{1} & u_{2}v_{2} & \cdots & u_{2}v_{m} \\ \vdots & \vdots & \ddots & \vdots \\ u_{n}v_{1} & u_{n}v_{2} & \cdots & u_{n}v_{m} \end{bmatrix}

Properties

Let u=[u1un]T\mathbf{u} = \begin{bmatrix} u_{1} & \cdots & u_{n} \end{bmatrix}^{\mathsf{T}}, v=[v1vn]T\mathbf{v} = \begin{bmatrix} v_{1} & \cdots & v_{n} \end{bmatrix}^{\mathsf{T}}, and w=[w1wn]T\mathbf{w} = \begin{bmatrix} w_{1} & \cdots & w_{n} \end{bmatrix}^{\mathsf{T}}. The following holds.


(uv)T=vu(1) (\mathbf{u} \otimes \mathbf{v})^{\mathsf{T}} = \mathbf{v} \otimes \mathbf{u} \tag{1}


(2)(2)Linearity: (v+w)u=vu+wu (\mathbf{v} + \mathbf{w}) \otimes \mathbf{u} = \mathbf{v} \otimes \mathbf{u} + \mathbf{w} \otimes \mathbf{u} u(v+w)=uv+uw \mathbf{u} \otimes (\mathbf{v} + \mathbf{w}) = \mathbf{u} \otimes \mathbf{v} + \mathbf{u} \otimes \mathbf{w}

Let αR\alpha \in \mathbb{R} be a constant.

(αv)u=α(vu)=(αv)u (\alpha \mathbf{v}) \otimes \mathbf{u} = \alpha (\mathbf{v} \otimes \mathbf{u}) = (\alpha \mathbf{v}) \otimes \mathbf{u}


(uv)w=(vw)u(3) (\mathbf{u} \otimes \mathbf{v}) \mathbf{w} = (\mathbf{v} \cdot \mathbf{w}) \mathbf{u} \tag{3}


wT(uv)=(wu)vT(4) \mathbf{w}^{\mathsf{T}} (\mathbf{u} \otimes \mathbf{v}) = (\mathbf{w} \cdot \mathbf{u}) \mathbf{v}^{\mathsf{T}} \tag{4}


(5)(5)Associativity:

When \otimes is extended by Kronecker product, the following holds.

(uKronv)Kronw=uKron(vKronw) (\mathbf{u} \otimes_{\text{Kron}} \mathbf{v}) \otimes_{\text{Kron}} \mathbf{w} = \mathbf{u} \otimes_{\text{Kron}} (\mathbf{v} \otimes_{\text{Kron}} \mathbf{w})

Proof

(1)(1)

This can be seen from the properties of the transpose.

(uv)T=(uvT)T=vuT=vu \begin{align*} (\mathbf{u} \otimes \mathbf{v})^{\mathsf{T}} &= (\mathbf{u} \mathbf{v}^{\mathsf{T}})^{\mathsf{T}} \\ &= \mathbf{v} \mathbf{u}^{\mathsf{T}} \\ &= \mathbf{v} \otimes \mathbf{u} \\ \end{align*}

(2)(2)

This holds since the matrix multiplication is linear.

(v+w)u=(v+w)uT=vuT+wuT=vu+wu \begin{align*} (\mathbf{v} + \mathbf{w}) \otimes \mathbf{u} &= (\mathbf{v} + \mathbf{w}) \mathbf{u}^{\mathsf{T}} \\ &= \mathbf{v} \mathbf{u}^{\mathsf{T}} + \mathbf{w} \mathbf{u}^{\mathsf{T}} \\ &= \mathbf{v} \otimes \mathbf{u} + \mathbf{w} \otimes \mathbf{u} \\ \end{align*}

This holds since the transpose is linear.

u(v+w)=u(v+w)T=u(vT+wT)=uvT+uwT=uv+uw \begin{align*} \mathbf{u} \otimes (\mathbf{v} + \mathbf{w}) &= \mathbf{u} (\mathbf{v} + \mathbf{w})^{\mathsf{T}} \\ &= \mathbf{u} (\mathbf{v}^{\mathsf{T}} + \mathbf{w}^{\mathsf{T}}) \\ &= \mathbf{u} \mathbf{v}^{\mathsf{T}} + \mathbf{u} \mathbf{w}^{\mathsf{T}} \\ &= \mathbf{u} \otimes \mathbf{v} + \mathbf{u} \otimes \mathbf{w} \\ \end{align*}

(αv)u=(αv)uT=α(vuT)=α(vu) \begin{align*} (\alpha \mathbf{v}) \otimes \mathbf{u} &= (\alpha \mathbf{v}) \mathbf{u}^{\mathsf{T}} \\ &= \alpha (\mathbf{v} \mathbf{u}^{\mathsf{T}}) \\ &= \alpha (\mathbf{v} \otimes \mathbf{u}) \\ \end{align*}

(3)(3)

(uv)w=(uvT)w=u(vTw)=u(vw)=(vw)u \begin{align*} (\mathbf{u} \otimes \mathbf{v}) \mathbf{w} &= (\mathbf{u} \mathbf{v}^{\mathsf{T}}) \mathbf{w} \\ &= \mathbf{u} (\mathbf{v}^{\mathsf{T}} \mathbf{w}) \\ &= \mathbf{u} (\mathbf{v} \cdot \mathbf{w}) \\ &= (\mathbf{v} \cdot \mathbf{w}) \mathbf{u} \\ \end{align*}

Note that vTw\mathbf{v}^{\mathsf{T}} \mathbf{w} is a scalar.

(4)(4)

wT(uv)=wT(uvT)=(wTu)vT=(wu)vT \begin{align*} \mathbf{w}^{\mathsf{T}} (\mathbf{u} \otimes \mathbf{v}) &= \mathbf{w}^{\mathsf{T}} (\mathbf{u} \mathbf{v}^{\mathsf{T}}) \\ &= (\mathbf{w}^{\mathsf{T}} \mathbf{u}) \mathbf{v}^{\mathsf{T}} \\ &= (\mathbf{w} \cdot \mathbf{u}) \mathbf{v}^{\mathsf{T}} \\ \end{align*}