📂Linear Algebra

Linear Independence and Linear Dependence

Definition¹

Let’s denote a non-empty subset of vector space $V$ as $S = \left\{ \mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{r} \right\}$. For constants $k_{1}, k_{2}, \dots, k_{r}$, the following equation

$$k_{1} \mathbf{v}_{1} + k_{2} \mathbf{v}_{2} + \dots + k_{r} \mathbf{v}_{r} = \mathbf{0}$$

has at least one solution

$$k_{1} = 0,\ k_{2} = 0,\ \dots,\ k_{r} = 0$$

This is called a trivial solution. If the trivial solution is the only solution, then vectors $\mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{r}$ are called linearly independent. If there is at least one solution that is not trivial, it is called linearly dependent.

Explanation

A trivial solution is a solution that is immediately obvious and hence, considered of little value. This is because, as mentioned in the definition, the case of $0$ is a frequent occurrence.

From the above definition, the following theorem can be immediately derived.

Let’s denote a non-empty subset of vector space $V$ as $S = \left\{ \mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{r} \right\}$. If no vector in $S$ can be represented as a linear combination of other vectors, it is considered linearly independent. Conversely, if at least one vector can be represented as a linear combination of others, it is considered linearly dependent.

Thinking with the theorem in mind, the naming of “independent” and “dependent” makes sense. Some textbooks have the definition and theorem in reverse order.

Interestingly, the reference in the footnote, ‘Elementary Linear Algebra’, defines it as this text for the translated version, but the original version has it defined in the opposite way. Personally, I think defining it as this text is cleaner. This is because defining it in the opposite way requires separate definitions for independence/dependence for sets with only one element. The proof of the theorem is introduced later.

In simpler terms, if there are two distinct vectors such that one cannot be made identical to the other by scaling up or down, they are considered independent. For example, $(1,0)$ and $(0,1)$ cannot become equal to each other no matter how they are scaled, i.e., enlarged or reduced. To rewrite according to the definition,

$$k_{1} (1,0) + k_{2} (0,1) = \mathbf{0}$$

Rearranging the second term gives

$$k_{1}(1,0) = - k_{2}(0,1)$$

Rearranging again gives

$$(k_{1}, 0) = ( 0 , - k_{2})$$

As the only solution satisfying the above equation is $k_{1} = k_{2} = 0$, $(1,0)$ and $(0,1)$ are linearly independent. This can be proven as a theorem.

Theorem

(a) A finite set that includes the zero vector is linearly dependent.

(b) The necessary and sufficient condition for a single vector $\mathbf{v}$ to be linearly independent is $\mathbf{v} \ne \mathbf{0}$.

(c) The necessary and sufficient condition for two distinct vectors to be linearly independent is that one vector cannot be represented as a multiple of the other.

(d) Let’s denote a set containing more than two vectors as $S=\left\{ \mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{r} \right\}$. The necessary and sufficient condition for $S$ to be linearly independent is that no vector in $S$ can be represented as a linear combination of other vectors.

(e) Let’s say $T \subset S$. If $S$ is linearly independent, then $T$ is also linearly independent.

(e') Let’s say $T \subset S$. If $T$ is linearly dependent, then $S$ is also linearly dependent.

Proof

(a)

Let’s say $S=\left\{ \mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{r}, \mathbf{0} \right\}$. Then, the following equation holds true.

$$0 \mathbf{v}_{1} + 0 \mathbf{v}_{2} + \dots + 0 \mathbf{v}_{r} + 1 \mathbf{0} = \mathbf{0}$$

Therefore, by definition, $S$ is linearly dependent.

■

(b)

Applying (a) to a set with a single element proves its validity.

■

(c)

$(\Longrightarrow)$

Assume $\mathbf{v}_{1}, \mathbf{v}_{2}$ is linearly independent. Then,

$$k_{1} \mathbf{v}_{1} + k_{2} \mathbf{v}_{2} = \mathbf{0}$$

The only solution satisfying this equation is $k_{1}=k_{2}=0$, so there does not exist a constant $k$ satisfying $\mathbf{v}_{1} = -\frac{k_{2}}{k_{1}}\mathbf{v}_{2} = -k\mathbf{v}_{2}$.

$(\Longleftarrow)$

Assume $\mathbf{v}_{1}$ cannot be represented as a constant multiple of $\mathbf{v}_{2}$. That is, assume the following equation

$$\mathbf{v}_{1} = k_{2}\mathbf{v}$$

does not have a solution for $k_{2}$. Then,

$$k_{1} \mathbf{v}_{1} + k_{2} \mathbf{v}_{2} = \mathbf{0}$$

Since the only solution satisfying this equation is the trivial solution, $\mathbf{v}_{1}, \mathbf{v}_{2}$ is linearly independent.

■

(d)

$(\Longrightarrow)$

Assume $S$ is linearly independent.

$$k_{1} \mathbf{v}_{1} + k_{2} \mathbf{v}_{2} + \dots + k_{r} \mathbf{v}_{r} = \mathbf{0}$$

Since the only solution is $k_{1}=k_{2}=\cdots=k_{r}=0$,

$$\mathbf{v}_{1} = -\frac{k_{2}}{k_{1}}\mathbf{v}_{2} - \cdots - \frac{k_{r}}{k_{1}}\mathbf{v}_{r}$$

no constants $\frac{k_{2}}{k_{1}}, \dots, \frac{k_{r}}{k_{1}}$ exist satisfying this. This applies to all $\mathbf{v}_{i}$, meaning no vector can be represented as a linear combination of others.

$(\Longleftarrow)$

Assuming no vector can be represented as a linear combination of others,

$$\mathbf{v}_{1} = k_{2}\mathbf{v}_{2} + \cdots + k_{r}\mathbf{v}_{r}$$

Assuming no solution exists for $k_{2}, \dots, k_{r}$,

$$k_{1}\mathbf{v}_{1} + k_{2}\mathbf{v}_{2} + \cdots + k_{r}\mathbf{v}_{r} = \mathbf{0}$$

Since the only solution satisfying this equation is the trivial solution, $S$ is linearly independent.

■

(e)

Let’s consider the two sets $T$ and $S$ as follows.

$$T = \left\{ \mathbf{v}_{1},\ \mathbf{v}_{2}, \dots, \mathbf{v}_{r} \right\},\quad S = \left\{ \mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{r}, \mathbf{v}_{r+1}, \dots, \mathbf{v}_{n} \right\}$$

$T$ is a subset of $S$. Now, assuming $S$ is linearly independent,

$$c_{1}\mathbf{v}_{1} + c_{2}\mathbf{v}_{2} + \cdots +c_{r} \mathbf{v}_{r} + c_{r+1} \mathbf{v}_{r+1} + \cdots + c_{n} \mathbf{v}_{n} = \mathbf{0}$$

The only solution satisfying this is the trivial solution $c_{1}=c_{2} = \cdots = c_{r} = c_{r+1} = \cdots = c_{n} = 0$. Therefore, since $c_{r+1} = \cdots = c_{n} = 0$, the following equation holds true.

$$\begin{align*} && c_{1}\mathbf{v}_{1} + c_{2}\mathbf{v}_{2} + \cdots +c_{r} \mathbf{v}_{r} + c_{r+1} \mathbf{v}_{r+1} + \cdots + c_{n} \mathbf{v}_{n} =&\ \mathbf{0} \\ \implies && c_{1}\mathbf{v}_{1} + c_{2}\mathbf{v}_{2} + \cdots +c_{r} \mathbf{v}_{r} + \left( 0\mathbf{v}_{r+1} + \cdots + 0 \mathbf{v}_{n} \right) =&\ \mathbf{0} \\ \implies && c_{1}\mathbf{v}_{1} + c_{2}\mathbf{v}_{2} + \cdots +c_{r} \mathbf{v}_{r} + \mathbf{0} =&\ \mathbf{0} \\ \implies && c_{1}\mathbf{v}_{1} + c_{2}\mathbf{v}_{2} + \cdots +c_{r} \mathbf{v}_{r} =&\ \mathbf{0} \end{align*}$$

However, this equation only holds true when $c_{1} = c_{2} = \cdots = c_{r} = 0$, so $T$ is linearly independent.

■

(e')

This is a contrapositive of (e).

■