📂Matrix Algebra

The Principal Axes Theorem

Definition¹

For $n$-dimensional vectors $\mathbf{x}, \mathbf{y} \in \mathbb{R}^{n}$, an invertible matrix $P$ satisfying the following, or the equation itself, is called a change of variable.

$$ \mathbf{x} = P \mathbf{y} $$

If $P$ is an orthogonal matrix, it is called an orthogonal change of variable.

Explanation

A change of variable is also called a change of coordinate. It is used, for example, to simplify expressions in the process of handling the quadratic form $\mathbf{x}^{\mathsf{T}} A \mathbf{x}$.

Theorem

Principal axes theorem:

If a real matrix $A \in M_{n \times n}(\mathbb{R})$ is symmetric, then there exists an orthogonal change of variable $P$ that can transform the quadratic form $\mathbf{x}^{\mathsf{T}} A \mathbf{x}$ into $\mathbf{y}^{\mathsf{T}} D \mathbf{y}$ having no cross terms. Such a $P$ orthogonally diagonalizes $A$, and the value of the quadratic form is as follows.

$$ \mathbf{x}^{\mathsf{T}} A \mathbf{x} = \mathbf{y}^{\mathsf{T}} D \mathbf{y} = \lambda_{1}y_{1}^{2} + \cdots + \lambda_{n}y_{n}^{2} $$

Here, $\lambda_{i}$ is the $i$-th diagonal entry of $D$ and the eigenvalue of $A$ corresponding to the $i$-th column vector of $P$.

Proof

If $A$ is a symmetric matrix, it is equivalent to being orthogonally diagonalizable. That is, for a diagonal matrix $D$, there exists an orthogonal matrix $P$ satisfying the following.

$$ A = P^{\mathsf{T}} D P $$

Let us choose such a $P$ as the orthogonal change of variable $\mathbf{x} = P \mathbf{y}$. Then we obtain the following.

$$ \mathbf{x}^{\mathsf{T}} A \mathbf{x} = (P \mathbf{y})^{\mathsf{T}} A (P \mathbf{y}) = \mathbf{y}^{\mathsf{T}} (P^{\mathsf{T}} A P) \mathbf{y} = \mathbf{y}^{\mathsf{T}} D \mathbf{y} $$

This value is as follows.

$$ \mathbf{y}^{\mathsf{T}} D \mathbf{y} = \begin{bmatrix} y_{1} & y_{2} & \cdots & y_{n} \end{bmatrix} \begin{bmatrix} \lambda_{1} & 0 & \cdots & 0 \\ 0 & \lambda_{2} & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda_{n} \end{bmatrix} \begin{bmatrix} y_{1} \\ y_{2} \\ \vdots \\ y_{n} \end{bmatrix} = \lambda_{1}y_{1}^{2} + \cdots + \lambda_{n}y_{n}^{2} $$

Moreover, these $\lambda_{i}$ are the eigenvalues of $A$ corresponding to the $i$-th column vectors of $P$.

■