📂Matrix Algebra

Fundamental Spaces of Matrices

Explanation¹

Let’s assume that the matrix $A$ is given. Then, we can think of the following 6 spaces for $A$.

Row space of $A$, Row space of $A^{T}$

Column space of $A$, Column space of $A^{T}$

Null space of $A$, Null space of $A^{T}$

However, since the row vectors of $A$ are the column vectors of $A^{T}$, and the column vectors of $A$ are the row vectors of $A^{T}$, the row space of $A$ and the column space of $A^{T}$ are the same. For the same reason, the column space of $A$ and the row space of $A^{T}$ are the same, which allows us to think of the following 4 matrices.

Row space of $A$, Column space of $A$

Null space of $A$, Null space of $A^{T}$

These 4 spaces are collectively called the fundamental spaces of matrix $A$.

Theorem 1

For an arbitrary matrix $A$,

$$ \rank(A) = \rank(A^{T}) $$

Proof

Since the row space of $A$ and the column space of $A^{T}$ are the same, by the definition of rank,

$$ \rank(A) = \dim(\mathcal{R}(A)) = \dim(\mathcal{C}(A^{T})) = \rank(A^{T}) $$

■

Theorem 2

Let $A$ be a matrix $m \times n$.

(a) The null space of $A$ and the row space of $A$ are orthogonal complements of each other in $\mathbb{R}^{n}$.

$$ \mathcal{N}(A) \oplus \mathcal{R}(A) = \mathbb{R}^{n} $$

(b) The null space of $A^{T}$ and the column space of $A$ are orthogonal complements of each other in $\mathbb{R}^{m}$.

$$ \mathcal{N}(A^{T}) \oplus \mathcal{C}(A) = \mathbb{R}^{m} $$

Proof

Strategy: Use definitions to deduce directly. Only the proof for (a) is presented. The proof for (b) is essentially the same.

---

First, let’s show that $\mathcal{N} (A) = \mathcal{R} (A)^{\perp}$. If we say $\mathbf{x} \in \mathcal{N} (A)$, by the definition of null space, it is as follows.

$$ A \mathbf{x} = \mathbf{0} $$

Taking the dot product with $\mathbf{y} \in \mathbb{R}^{n}$, it is as follows.

$$ \mathbf{y}^{T} ( A \mathbf{x} ) = \mathbf{0} $$

By the associative law of matrix multiplication, $( \mathbf{y}^{T} A ) \mathbf{x} = \mathbf{0}$, and by the property of transpose matrices, $( A^{T} \mathbf{y} ) ^{T} \mathbf{x} = \mathbf{0}$, thus, by the definition of orthogonality, it is as follows.

$$ A^{T} \mathbf{y} \perp \mathbf{x} $$

Then, by the definition of orthogonal complement, it is as follows.

$$ \mathbf{x} \in \mathcal{R} (A)^{\perp} $$

Summarizing this content, $\mathbf{x} \in \mathcal{N} (A)\implies \mathbf{x} \in \mathcal{R} (A)^{\perp}$, so we get the following result.

$$ \mathcal{N} (A) \subset \mathcal{R} (A)^{\perp} $$

Repeating this process in reverse, we can obtain $\mathcal{N} (A) \supset \mathcal{R} (A)^{\perp}$, so we get the following result.

$$ \mathcal{N} (A) = \mathcal{R} (A)^{\perp} $$

Taking $^{\perp}$, it is as follows.

$$ \mathcal{N} (A)^{\perp} = \mathcal{R} (A) $$

Since both are subspaces of $\mathbb{R}^{n}$, we obtain the following.

$$ \mathcal{N}(A) \oplus \mathcal{R}(A) = \mathbb{R}^{n} $$

■