📂Matrix Algebra

Generalized Eigenvector

Introduction¹

The matrix eigenvalue problem is: for a given matrix $A \in M_{n \times n}(\mathbb{C})$, find a vector $\mathbf{0} \ne \mathbf{v} \in \mathbb{C}^{n}$ and a scalar $\lambda \in \mathbb{C}$ that satisfy

$$ A \mathbf{v} = \lambda \mathbf{v} \iff (A - \lambda I) \mathbf{v} = \mathbf{0} $$

Here $\lambda$ is called an eigenvalue of $A$ and $\mathbf{v}$ is called an eigenvector of $A$. If $A$ has $n$ linearly independent eigenvectors, it can be diagonalized and decomposed into a nicer form. However, a general matrix does not always have $n$ linearly independent eigenvectors. In such cases one seeks suitable alternatives and attempts other forms of decomposition; for example, when $A$ is not square, the singular value decomposition is such an alternative. Here we consider the case where $A$ is square but has fewer than $n$ linearly independent eigenvectors.

As the above equation shows, an eigenvector is also an element of the kernel $\ker (A - \lambda I)$. The fact that $A$ is not diagonalizable means the dimension of $\ker (A - \lambda I)$ is not large enough (i.e. smaller than $n$). But the kernel grows monotonically when the same transformation is applied repeatedly.

$$ \ker(A - \lambda I) \subset \ker(A - \lambda I)^{2} \subset \cdots \subset \ker(A - \lambda I)^{k} \subset \cdots $$

That is, for any natural number $k$, if we consider the set of vectors $\mathbf{v}$ that satisfy the following equation, this set contains all the ordinary eigenvectors and is a natural generalization.

$$ (A - \lambda I)^{k} \mathbf{v} = \mathbf{0} $$

Definition

Let a matrix $A \in M_{n \times n}(\mathbb{C})$ be given. For any scalar $\lambda \in \mathbb{C}$ and any natural number $k \in \mathbb{N}$, a vector $\mathbf{v}$ that satisfies the equation below is called a generalized eigenvector of $A$.

$$ (A - \lambda I)^{k} \mathbf{v} = \mathbf{0} \tag{1} $$

Explanation

This only makes sense for vectors that are $\mathbf{v} \ne \mathbf{0}$, so $(A - \lambda I)$ must not be an invertible matrix. Therefore the $\lambda$ appearing in $(1)$ is an ordinary eigenvalue. In other words, a $\lambda$ that satisfies $(1)$ is not called a generalized eigenvalue. However, as the definition suggests, there do exist generalized eigenvectors that are not ordinary eigenvectors.

For example, consider the matrix $A = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}$ given in $2 \times 2$. Its characteristic polynomial is $(\lambda - 1)^{2} = 0$, so $A$ has a single eigenvalue $\lambda = 1$ with algebraic multiplicity $2$. The corresponding (linearly independent) eigenvectors are only one in number as shown below, so $A$ is not diagonalizable by eigenvalues.

$$ \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} \implies \mathbf{x}_{1} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} $$

Now consider the case $k = 2$. One can find $\mathbf{x}_{1}$ and a vector $\mathbf{x}_{2}$ linearly independent from it as follows.

$$ \begin{align*} (A - \lambda I)^{2} \mathbf{x} = \mathbf{0} &\implies \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}\begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} \\ &\implies \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} \\ &\implies \mathbf{x}_{2} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \end{align*} $$

If we look again at the equations used to find the eigenvector $\mathbf{x}_{1}$ and the generalized eigenvector $\mathbf{x}_{2}$, we see that $\mathbf{x}_{2}$ is a vector which, after applying $(A - \lambda I)$, becomes $\mathbf{x}_{1}$.

$$ \begin{align*} && (A - \lambda I) \mathbf{x}_{1} &= \mathbf{0} \\ && (A - \lambda I)^{2} \mathbf{x}_{2} = (A - \lambda I) \left[ (A - \lambda I) \mathbf{x}_{2} \right] &= \mathbf{0} \\ \implies && (A - \lambda I) \mathbf{x}_{2} &= \mathbf{x}_{1} \end{align*} $$

Thus the generalized eigenvectors corresponding to the eigenvalue $\lambda = 1$ number (including the ordinary eigenvectors) a total of $2$. It is not a coincidence that this number equals the algebraic multiplicity of $\lambda = 1$. The geometric multiplicity of an eigenvalue is always less than or equal to its algebraic multiplicity; the geometric multiplicity equals the number of linearly independent ordinary eigenvectors corresponding to the eigenvalue, while the algebraic multiplicity equals the number of linearly independent generalized eigenvectors corresponding to the eigenvalue.

Generalized eigenspace

The eigenspace $E_{\lambda}$ for an eigenvalue $\lambda$ is the space spanned by the eigenvectors corresponding to $\lambda$. By extension, the generalized eigenspace $W_{\lambda}$ is defined as the space spanned by the generalized eigenvectors corresponding to $\lambda$ as follows. For the matrix $A \in M_{n \times n}(\mathbb{C})$ and its eigenvalue $\lambda$,

$$ W_{\lambda} = \left\{ \mathbf{v} \in \mathbb{C}^{n} : (A - \lambda I)^{k} \mathbf{v} = \mathbf{0} \text{ for some } k \in \mathbb{N} \right\} $$