📂Matrix Algebra

Fundamental Spaces of Matrices

Explanation¹

Let’s assume that the matrix $A$ is given. Then, we can think of the following 6 spaces for $A$.

Row space of $A$, Row space of $A^{T}$

Column space of $A$, Column space of $A^{T}$

Null space of $A$, Null space of $A^{T}$

However, since the row vectors of $A$ are the column vectors of $A^{T}$, and the column vectors of $A$ are the row vectors of $A^{T}$, the row space of $A$ and the column space of $A^{T}$ are the same. For the same reason, the column space of $A$ and the row space of $A^{T}$ are the same, which allows us to think of the following 4 matrices.

Row space of $A$, Column space of $A$

Null space of $A$, Null space of $A^{T}$

These 4 spaces are collectively called the fundamental spaces of matrix $A$.

Theorem 1

For an arbitrary matrix $A$,

$$\rank(A) = \rank(A^{T})$$

Proof

Since the row space of $A$ and the column space of $A^{T}$ are the same, by the definition of rank,

$$\rank(A) = \dim(\mathcal{R}(A)) = \dim(\mathcal{C}(A^{T})) = \rank(A^{T})$$

■

Theorem 2

Let $A$ be a matrix $m \times n$.

(a) The null space of $A$ and the row space of $A$ are orthogonal complements of each other in $\mathbb{R}^{n}$.

$$\mathcal{N}(A) \oplus \mathcal{R}(A) = \mathbb{R}^{n}$$

(b) The null space of $A^{T}$ and the column space of $A$ are orthogonal complements of each other in $\mathbb{R}^{m}$.

$$\mathcal{N}(A^{T}) \oplus \mathcal{C}(A) = \mathbb{R}^{m}$$

Proof

Strategy: Use definitions to deduce directly. Only the proof for (a) is presented. The proof for (b) is essentially the same.

---

First, let’s show that $\mathcal{N} (A) = \mathcal{R} (A)^{\perp}$. If we say $\mathbf{x} \in \mathcal{N} (A)$, by the definition of null space, it is as follows.

$$A \mathbf{x} = \mathbf{0}$$

Taking the dot product with $\mathbf{y} \in \mathbb{R}^{n}$, it is as follows.

$$\mathbf{y}^{T} ( A \mathbf{x} ) = \mathbf{0}$$

By the associative law of matrix multiplication, $( \mathbf{y}^{T} A ) \mathbf{x} = \mathbf{0}$, and by the property of transpose matrices, $( A^{T} \mathbf{y} ) ^{T} \mathbf{x} = \mathbf{0}$, thus, by the definition of orthogonality, it is as follows.

$$A^{T} \mathbf{y} \perp \mathbf{x}$$

Then, by the definition of orthogonal complement, it is as follows.

$$\mathbf{x} \in \mathcal{R} (A)^{\perp}$$

Summarizing this content, $\mathbf{x} \in \mathcal{N} (A)\implies \mathbf{x} \in \mathcal{R} (A)^{\perp}$, so we get the following result.

$$\mathcal{N} (A) \subset \mathcal{R} (A)^{\perp}$$

Repeating this process in reverse, we can obtain $\mathcal{N} (A) \supset \mathcal{R} (A)^{\perp}$, so we get the following result.

$$\mathcal{N} (A) = \mathcal{R} (A)^{\perp}$$

Taking $^{\perp}$, it is as follows.

$$\mathcal{N} (A)^{\perp} = \mathcal{R} (A)$$

Since both are subspaces of $\mathbb{R}^{n}$, we obtain the following.

$$\mathcal{N}(A) \oplus \mathcal{R}(A) = \mathbb{R}^{n}$$

■