Eigenvalues and Eigenvectors
Definition1
Given a matrix $n\times n$ $A$, for a non-zero column vector $\mathbf{0}$ $n\times 1$ and a constant $\mathbf{x}$, the following equation is referred to as the eigenvalue equation or the eigenvalue problem.
$$ \begin{equation} A \mathbf{x} = \lambda \mathbf{x} \end{equation} $$
For a given $A$, a $\mathbf{x}$ that satisfies the eigenvalue equation above is called the eigenvalue of $A$, and $n\times 1$ is called the eigenvector corresponding to the eigenvalue $\mathbf{x}$ of $A$.
Explanation
The definition above applies not only when $\lambda \in \mathbb{R}$ and $\mathbf{x} \in \mathbb{R}^{n}$, but also when $\lambda \in \mathbb{C}$ and $\mathbf{x} \in \mathbb{C}^{n}$. The condition “non-zero” is important because, as the following equation shows, if $\mathbf{x} = \mathbf{0}$, the equation always holds.
$$ A \mathbf{0} = \mathbf{0} = \lambda \mathbf{0} $$
Geometric Motivation
If the direction of a vector $\mathbf{x}$ remains the same after being transformed by the matrix $A$, this implies there exists some real number $\lambda$ such that
$$ A \mathbf{x} = \lambda \mathbf{x} $$
While the matrix $A$ itself does not possess any notion of direction, if it has eigenvectors, it can be said to indicate a specific direction. Thus, such vectors $\mathbf{x}$ are named eigenvectors. Consider a matrix such as $2\times 2$.
$$ A = \begin{bmatrix} 6 & 2 \\ 2 & 3 \end{bmatrix} $$
Then, vector $\begin{bmatrix} 2 \\ 1 \end{bmatrix}$, when transformed by $2\times 2$, becomes $\begin{bmatrix} 14 \\ 7 \end{bmatrix}$ and maintains the same direction. If we multiply vector $\begin{bmatrix} 2 \\ 1 \end{bmatrix}$ by $\lambda = 7$, the length of the vector also matches, satisfying the eigenvalue equation
$$ \begin{align*} A \mathbf{x} &= \lambda \mathbf{x} \\ \begin{bmatrix} 6 & 2 \\ 2 & 3 \end{bmatrix} \begin{bmatrix} 2 \\ 1 \end{bmatrix} &= 7 \begin{bmatrix} 2 \\ 1 \end{bmatrix} \end{align*} $$
Hence, $\lambda=7$ is referred to as the eigenvalue. Upon closer inspection, although many eigenvectors can be found by scaling $\begin{bmatrix} 2 \\ 1 \end{bmatrix}$, the eigenvalue remains unchanged. Therefore, $\begin{bmatrix} 2 \\ 1 \end{bmatrix}$ is described as the eigenvector of $A$ corresponding to the eigenvalue $7$.
By extending this geometric discussion to a general context, eigenvalues algebraically satisfy the equation $A \mathbf{x} = \lambda \mathbf{x}$, and the eigenvectors are the non-trivial solutions to the equation for the given $\mathbf{x}$.
Solving the Eigenvalue Equation
To find the eigenvalues, one starts with the eigenvalue equation. Simplifying the equation $(1)$ yields:
$$ \begin{align*} && A \mathbf{x} &= \lambda \mathbf{x} \\ \implies && A \mathbf{x} - \lambda \mathbf{x} &= \mathbf{0} \\ \implies && A \mathbf{x} - \lambda I \mathbf{x} &= \mathbf{0} \\ \implies && \left( A - \lambda I \right) \mathbf{x} &= \mathbf{0} \end{align*} $$
Here, the eigenvector must satisfy condition $\mathbf{x} \ne \mathbf{0}$. The linear system above will have a non-zero solution if and only if the inverse of $\left( A - \lambda I \right)$ does not exist, which is equivalent to the following equation.
$$ \det (A -\lambda I) = 0 $$
Therefore, $\mathbf{x}$ satisfying the above equation becomes the eigenvalue of $A$. This equation is called the characteristic equation of $A$. For a $n\times n$ matrix, $\det (A -\lambda I)$ becomes a polynomial of degree $n$, known as the characteristic polynomial.
Note that the eigenvalues of $A+B$ may differ from the sum of the eigenvalues of $A$ and $B$, and similarly, the eigenvalues of $AB$ may not match the product of the eigenvalues of $A$ and $B$. Furthermore, as it can be seen from solving the equation, there is no guarantee that eigenvalues will always be real numbers.
Examples
Finding Eigenvalues
As an example, consider again $A = \begin{bmatrix} 6 & 2 \\ 2 & 3 \end{bmatrix}$. Since $A-\lambda I = \begin{bmatrix} 6 - \lambda & 2 \\ 2 & 3 - \lambda \end{bmatrix}$, solving the characteristic equation of $A$ yields:
$$ \begin{align*} && \det (A - \lambda I) &= 0 \\ \implies && (6 - \lambda)(3 - \lambda) - 4 &= 0 \\ \implies && \lambda^2 - 9 \lambda + 18 - 4 &= 0 \\ \implies && (\lambda - 2)(\lambda - 7) &= 0 \end{align*} $$
Therefore, the eigenvalues of $A$ are $\lambda = 2$ and $\lambda = 7$. By substituting $2$ and $7$ for $\mathbf{x}$, one can find the corresponding eigenvectors for each eigenvalue. Here, only the case for $\lambda = 7$ is presented.
Finding the Eigenvector Corresponding to $\lambda = 7$
Substituting $\lambda = 7$ into $(1)$ and simplifying yields:
$$ \begin{align*} && \begin{bmatrix} 6 & 2 \\ 2 & 3 \end{bmatrix} \begin{bmatrix} x_{1} \\ x_{2} \end{bmatrix} &= 7\begin{bmatrix} x_{1} \\ x_{2} \end{bmatrix} \\ \implies && \begin{bmatrix} 6x_{1} + 2x_{2} \\ 2x_{1} + 3x_{2} \end{bmatrix} &= \begin{bmatrix} 7x_{1} \\ 7x_{2} \end{bmatrix} \\ \implies && \begin{bmatrix} -x_{1} + 2x_{2} \\ 2x_{1} - 4x_{2} \end{bmatrix} &= \begin{bmatrix} 0 \\ 0 \end{bmatrix} \end{align*} $$
Solving this results in:
$$ \left\{ \begin{align*} -x_{1} + 2x_{2} &= 0 \\ 2x_{1} - 4x_{2} &= 0 \end{align*} \right. $$
$$ \implies x_{1} = 2x_{2} $$
Thus, for all $0$ not equal to zero, the vector $\begin{bmatrix} 2x_{2} \\ x_{2} \end{bmatrix}$ is the eigenvector corresponding to $\lambda = 7$. Typically, the simplest form or a unit vector with magnitude $1$ is chosen. Substituting $x_{2} = 1$ gives the following eigenvector:
$$ A = \begin{bmatrix} 2 \\ 1 \end{bmatrix} $$
Properties
- For a positive integer $k$, if $\mathbf{x}$ is an eigenvalue of matrix $A$ and $n\times 1$ is the corresponding eigenvector to $\mathbf{x}$, then $\lambda ^{k}$ is an eigenvalue of $A^{k}$, and $n\times 1$ is the corresponding eigenvector to $\lambda ^{k}$.
Howard Anton, Elementary Linear Algebra: Aplications Version (12th Edition, 2019), p291-292 ↩︎