📂Matrix Algebra

Eigenvalues and Eigenvectors

Definition¹

Given a matrix $n\times n$ $A$, for a non-zero column vector $\mathbf{0}$ $n\times 1$ and a constant $\mathbf{x}$, the following equation is referred to as the eigenvalue equation or the eigenvalue problem.

$$\begin{equation} A \mathbf{x} = \lambda \mathbf{x} \end{equation}$$

For a given $A$, a $\mathbf{x}$ that satisfies the eigenvalue equation above is called the eigenvalue of $A$, and $n\times 1$ is called the eigenvector corresponding to the eigenvalue $\mathbf{x}$ of $A$.

Explanation

The definition above applies not only when $\lambda \in \mathbb{R}$ and $\mathbf{x} \in \mathbb{R}^{n}$, but also when $\lambda \in \mathbb{C}$ and $\mathbf{x} \in \mathbb{C}^{n}$. The condition “non-zero” is important because, as the following equation shows, if $\mathbf{x} = \mathbf{0}$, the equation always holds.

$$A \mathbf{0} = \mathbf{0} = \lambda \mathbf{0}$$

Geometric Motivation

If the direction of a vector $\mathbf{x}$ remains the same after being transformed by the matrix $A$, this implies there exists some real number $\lambda$ such that

$$A \mathbf{x} = \lambda \mathbf{x}$$

While the matrix $A$ itself does not possess any notion of direction, if it has eigenvectors, it can be said to indicate a specific direction. Thus, such vectors $\mathbf{x}$ are named eigenvectors. Consider a matrix such as $2\times 2$.

$$A = \begin{bmatrix} 6 & 2 \\ 2 & 3 \end{bmatrix}$$

Then, vector $\begin{bmatrix} 2 \\ 1 \end{bmatrix}$, when transformed by $2\times 2$, becomes $\begin{bmatrix} 14 \\ 7 \end{bmatrix}$ and maintains the same direction. If we multiply vector $\begin{bmatrix} 2 \\ 1 \end{bmatrix}$ by $\lambda = 7$, the length of the vector also matches, satisfying the eigenvalue equation

$$\begin{align*} A \mathbf{x} &= \lambda \mathbf{x} \\ \begin{bmatrix} 6 & 2 \\ 2 & 3 \end{bmatrix} \begin{bmatrix} 2 \\ 1 \end{bmatrix} &= 7 \begin{bmatrix} 2 \\ 1 \end{bmatrix} \end{align*}$$

Hence, $\lambda=7$ is referred to as the eigenvalue. Upon closer inspection, although many eigenvectors can be found by scaling $\begin{bmatrix} 2 \\ 1 \end{bmatrix}$, the eigenvalue remains unchanged. Therefore, $\begin{bmatrix} 2 \\ 1 \end{bmatrix}$ is described as the eigenvector of $A$ corresponding to the eigenvalue $7$.

By extending this geometric discussion to a general context, eigenvalues algebraically satisfy the equation $A \mathbf{x} = \lambda \mathbf{x}$, and the eigenvectors are the non-trivial solutions to the equation for the given $\mathbf{x}$.

Solving the Eigenvalue Equation

To find the eigenvalues, one starts with the eigenvalue equation. Simplifying the equation $(1)$ yields:

$$\begin{align*} && A \mathbf{x} &= \lambda \mathbf{x} \\ \implies && A \mathbf{x} - \lambda \mathbf{x} &= \mathbf{0} \\ \implies && A \mathbf{x} - \lambda I \mathbf{x} &= \mathbf{0} \\ \implies && \left( A - \lambda I \right) \mathbf{x} &= \mathbf{0} \end{align*}$$

Here, the eigenvector must satisfy condition $\mathbf{x} \ne \mathbf{0}$. The linear system above will have a non-zero solution if and only if the inverse of $\left( A - \lambda I \right)$ does not exist, which is equivalent to the following equation.

$$\det (A -\lambda I) = 0$$

Therefore, $\mathbf{x}$ satisfying the above equation becomes the eigenvalue of $A$. This equation is called the characteristic equation of $A$. For a $n\times n$ matrix, $\det (A -\lambda I)$ becomes a polynomial of degree $n$, known as the characteristic polynomial.

Note that the eigenvalues of $A+B$ may differ from the sum of the eigenvalues of $A$ and $B$, and similarly, the eigenvalues of $AB$ may not match the product of the eigenvalues of $A$ and $B$. Furthermore, as it can be seen from solving the equation, there is no guarantee that eigenvalues will always be real numbers.

Examples

Finding Eigenvalues

As an example, consider again $A = \begin{bmatrix} 6 & 2 \\ 2 & 3 \end{bmatrix}$. Since $A-\lambda I = \begin{bmatrix} 6 - \lambda & 2 \\ 2 & 3 - \lambda \end{bmatrix}$, solving the characteristic equation of $A$ yields:

$$\begin{align*} && \det (A - \lambda I) &= 0 \\ \implies && (6 - \lambda)(3 - \lambda) - 4 &= 0 \\ \implies && \lambda^2 - 9 \lambda + 18 - 4 &= 0 \\ \implies && (\lambda - 2)(\lambda - 7) &= 0 \end{align*}$$

Therefore, the eigenvalues of $A$ are $\lambda = 2$ and $\lambda = 7$. By substituting $2$ and $7$ for $\mathbf{x}$, one can find the corresponding eigenvectors for each eigenvalue. Here, only the case for $\lambda = 7$ is presented.

Finding the Eigenvector Corresponding to $\lambda = 7$

Substituting $\lambda = 7$ into $(1)$ and simplifying yields:

$$\begin{align*} && \begin{bmatrix} 6 & 2 \\ 2 & 3 \end{bmatrix} \begin{bmatrix} x_{1} \\ x_{2} \end{bmatrix} &= 7\begin{bmatrix} x_{1} \\ x_{2} \end{bmatrix} \\ \implies && \begin{bmatrix} 6x_{1} + 2x_{2} \\ 2x_{1} + 3x_{2} \end{bmatrix} &= \begin{bmatrix} 7x_{1} \\ 7x_{2} \end{bmatrix} \\ \implies && \begin{bmatrix} -x_{1} + 2x_{2} \\ 2x_{1} - 4x_{2} \end{bmatrix} &= \begin{bmatrix} 0 \\ 0 \end{bmatrix} \end{align*}$$

Solving this results in:

$$\left\{ \begin{align*} -x_{1} + 2x_{2} &= 0 \\ 2x_{1} - 4x_{2} &= 0 \end{align*} \right.$$

$$\implies x_{1} = 2x_{2}$$

Thus, for all $0$ not equal to zero, the vector $\begin{bmatrix} 2x_{2} \\ x_{2} \end{bmatrix}$ is the eigenvector corresponding to $\lambda = 7$. Typically, the simplest form or a unit vector with magnitude $1$ is chosen. Substituting $x_{2} = 1$ gives the following eigenvector:

$$A = \begin{bmatrix} 2 \\ 1 \end{bmatrix}$$

Properties

• For a positive integer $k$, if $\mathbf{x}$ is an eigenvalue of matrix $A$ and $n\times 1$ is the corresponding eigenvector to $\mathbf{x}$, then $\lambda ^{k}$ is an eigenvalue of $A^{k}$, and $n\times 1$ is the corresponding eigenvector to $\lambda ^{k}$.