📂Matrix Algebra

Matrix Rank, Nullity

Theorem¹

The dimensions of the row space and column space of matrix $A$ are the same.

Proof

Let $R$ be the row echelon form matrix of $A$. Since basic row operations do not change the dimensions of the row space and column space of $A$, the following equation holds:

$$\begin{align*} \dim \big( \mathcal{R}(A) \big) &= \dim \big( \mathcal{R}(R) \big) \\ \dim \big( \mathcal{C}(A) \big) &= \dim \big( \mathcal{C}(R) \big) \end{align*}$$

Therefore, it suffices to show that the dimensions of the row space and column space of $R$ are the same. But since the row space of $R$ is generated by the rows with leading 1, and the column space of $R$ is generated by the columns with leading 1, the dimensions of the row space and column space of $R$ are the same.

■

Definition

The dimension of the row space (column space) of matrix $A$ is called rank and is denoted as follows:

$$\text{rank}(A) = \dim \mathcal{R}(A) = \dim \mathcal{C}(A)$$

The dimension of the null space of matrix $A$ is called nullity and is denoted as follows:

$$\text{nullity}(A) = \dim \mathcal{N}(A)$$

Explanation

Rank is sometimes translated as coefficient, and nullity as degeneracy.

On the other hand, $\text{rank}(A)$ can also be defined as the number of pivots when $A$ is turned into a row echelon form.

Consider the non-square matrix $m \times n$, matrix $A$. Then, the row space can be at most $n$ dimensions, and the column space can be at most $m$ dimensions. But since these two values are the same and that is the rank, the following holds:

$$\rank(A) \le \min(m,n)$$

In the case of $\rank(A) = \min(m,n)$, $A$ is said to have full rank. If it does not have full rank, it is called rank deficient.

If it is difficult to intuitively understand, think of it as a concept derived from counting the unknowns in a system of linear equations. Although the definition itself is not difficult, concepts such as null space, coefficient, and degeneracy especially complicate understanding for those who study from the original texts, making it difficult to grasp the meaning. The reason for studying these concepts is to make it easier to express the applications of linear algebra in the language of mathematics later on. When complex theories unfold, definitions like column space or null space greatly save space and cover more complex phenomena.

Also, the column space is called image as $\text{Im} (A)$. If matrix $A$ is considered in terms of a function concept, then $A \in \mathbb{R}^{m \times n}$ corresponds to function $T_{A}$, which can also be seen as $T_{A} : \mathbb{R}^{n} \to \mathbb{R}^{m}$.

The following Rank-Nullity Theorem is also easier to understand when thought of in terms of a function concept. Don’t forget that it is $\text{rank} A = \text{rank} A^{T}$.

Rank-Nullity Theorem

For matrix $A \in M_{ m \times n }(\mathbb{R})$, the following equation holds:

$$\begin{align*} \text{rank} (A) + \text{nullity} (A) &= \dim \mathbb{R}^{n} = n \\ \text{rank} (A^{T}) + \text{nullity} (A^{T}) &= \dim \mathbb{R}^{m} = m \end{align*}$$

---

Also known as the Dimension Theorem for matrices. When generalized for linear transformations, the following holds:

For vector space $V, W$ and linear transformation $T : V \to W$, the following equation holds:

$$\text{rank} (T) + \text{nullity} (T) = \dim (V)$$

Proof

Let’s say $A$ is a $m \times n$ matrix. Then, since there are $n$ columns in $A$, the homogeneous linear system $A \mathbf{x} = \mathbf{0}$ has $n$ unknowns. Therefore, ’the number of leading variables + the number of free variables = $n$’ holds. The number of leading variables is the same as the number of leading 1s, which is the same as the dimension of the row space. Moreover, the number of free variables is the same as the number of parameters, which is the same as the dimension of the null space. Therefore, the theorem holds.

■

See Also

Kernel in Abstract Algebra

The null space is denoted as $\ker A$ and is also called Kernel. This is a specialized expression in linear algebra of the general concept of kernel dealt with in abstract algebra, and it is also seen as $A$ being viewed as a function.