📂Linear Algebra

Linear Combination, Span

Definition: Linear Combination¹

Let $\mathbf{w}$ be a vector in the vector space $V$. If $\mathbf{w}$ can be expressed as follows for vectors $\mathbf{v}_{1},\mathbf{v}_{2},\cdots ,\mathbf{v}_{r}$ in $V$ and arbitrary constants $k_{1}, k_{2}, \cdots, k_{r}$, then $\mathbf{w}$ is called a linear combination of $\mathbf{v}_{1},\mathbf{v}_{2},\cdots ,\mathbf{v}_{r}$.

$$ \mathbf{w} = k_{1}\mathbf{v}_{1} + k_{2}\mathbf{v}_{2} + \cdots + k_{r}\mathbf{v}_{r} $$

Additionally, in this case, the constants $k_{1}, k_{2}, \cdots, k_{r}$ are referred to as the coefficients of the linear combination $\mathbf{w}$.

Explanation

Though it might seem unfamiliar presented in a formulaic manner, it’s not a complex concept. The representation of vectors in a two-dimensional Cartesian coordinate system is precisely the linear combination of two unit vectors $\hat{\mathbf{x}} = (1,0)$ and $\hat{\mathbf{y}} = (0,1)$.

$$ \mathbf{v} = (v_{1}, v_{2}) = (v_{1},0)+(0,v_{2}) = v_{1}(1,0) + v_{2}(0,1) = v_{1}\hat{\mathbf{x}} + v_{2} \hat{\mathbf{y}} $$

Theorem

Let $S = \left\{ \mathbf{w}_{1}, \mathbf{w}_{2}, \dots, \mathbf{w}_{r} \right\}$ be a non-empty subset of the vector space $V$. Then the following hold.

(a) Let $W$ be the set of all possible linear combinations of elements of $S$. $W$ is a subspace of $V$.

(b) The $W$ from (a) is the smallest subspace of $V$ that includes $S$. That is, if $W^{\prime}$ is a subspace of $V$ that includes $S$, then the following equation holds.

$$ S \subset W \le W^{\prime} $$

Proof

(a)

To demonstrate that $W$ is closed under addition and scalar multiplication, one can apply the subspace test as follows.

$$ \mathbf{u} = c_{1} \mathbf{w}_{1} + c_{2} \mathbf{w}_{2} + \cdots + c_{r} \mathbf{w}_{r}, \quad \mathbf{v} = k_{1} \mathbf{w}_{1} + k_{2} \mathbf{w}_{2} + \cdots + k_{r} \mathbf{w}_{r} $$

• (A1)

  $\mathbf{u}+\mathbf{v}$ is as follows.

  $$ \mathbf{u} +\mathbf{v} = ( c_{1} + k_{1} ) \mathbf{w}_{1} + ( c_{2} + k_{2} ) \mathbf{w}_{2} + \cdots + ( c_{r} + k_{r} ) \mathbf{w}_{r} $$

  Since this is a linear combination of $\mathbf{w}_{1}, \mathbf{w}_{2}, \dots, \mathbf{w}_{r}$, $\mathbf{u} + \mathbf{v} \in W$ is true.

• (M1)

  For any constant $k$, $k\mathbf{u}$ is as follows.

  $$ k\mathbf{u} = ( k c_{1} ) \mathbf{w}_{1} + ( k c_{2} ) \mathbf{w}_{2} + \cdots + ( k c_{r} ) \mathbf{w}_{r} $$

  Since this is a linear combination of $\mathbf{w}_{1}, \mathbf{w}_{2}, \dots, \mathbf{w}_{r}$, $k\mathbf{u} \in W$ is true.

• Conclusion

  Since $W$ is closed under addition and scalar multiplication, by the subspace test, $W$ is a subspace of $V$.

  $$ W \le V $$

■

(b)

Assuming $W^{\prime}$ is a subspace of $V$ that includes $S$, since $W^{\prime}$ is closed under addition and scalar multiplication, all linear combinations of elements of $S$ are elements of $W^{\prime}$. Therefore,

$$ W \le W^{\prime} $$

■

Definition: Span

The $W$ in the theorem is referred to as the subspace of $V$ spanned by $S$. Furthermore, it is said that the vectors $\mathbf{w}_{1}, \mathbf{w}_{2}, \dots, \mathbf{w}_{r}$ span $W$, which is denoted as follows.

$$ W = \text{span}\left\{ \mathbf{w}_{1}, \mathbf{w}_{2}, \dots, \mathbf{w}_{r} \right\} \quad \text{or} \quad W = \text{span}(S) $$

Explanation

The concept of spanning is necessary to contemplate the smallest set that contains certain elements. Indeed, the above theorem highlights this point. Additionally, eliminating all redundant elements from $S$ itself would make it the basis of a vector space.

Theorem

Let $S = \left\{ \mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{r} \right\}$ and $S^{\prime} = \left\{ \mathbf{w}_{1}, \mathbf{w}_{2}, \dots, \mathbf{w}_{r} \right\}$ be non-empty subsets of the vector space $V$. Then,

$$ \text{span} \left\{ \mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{r} \right\} = \text{span} \left\{ \mathbf{w}_{1}, \mathbf{w}_{2}, \dots, \mathbf{w}_{r} \right\} $$

The necessary and sufficient condition for this to hold is that all vectors of $S$ can be expressed as linear combinations of vectors of $S^{\prime}$, and all vectors of $S^{\prime}$ can be expressed as linear combinations of vectors of $S$.