📂Matrix Algebra

Generalization of the Ellipse: Ellipsoid

Definition

For a linear transformation $A \in \mathbb{R}^{m \times m}$, the image $AN$ of a $m$-dimensional unit sphere $N := \left\{ \mathbf{x} \in \mathbb{R}^{m} : \left\| \mathbf{x} \right\|_{2} = 1 \right\}$ is called an ellipsoid. The eigenvalues $\sigma_{1}^{2} > \cdots \ge \sigma_{m}^{2} \ge 0$ of $A$ and the corresponding unit eigenvectors $u_{1} , \cdots , u_{m}$ are referred to as the axes of the ellipsoid for $\sigma_{i} u_{i}$.

Explanation

A $m$-dimensional unit sphere consists of points that are centered at $\mathbb{0} \in \mathbb{R}^{m}$ with radius $1$. When $m=2$, it becomes the well-known unit circle.

An ellipsoid is also called an ellipsoidal body or hyperellipse. Rather than saying the terms ellipsoidal surface or ellipsoidal sphere are incorrect, it’s more insightful to grasp the definition based on the context being read. In some contexts, an ellipsoid refers to a fully solid object, whereas in others it may refer only to the outer shell.

Geometry

If one is sufficiently familiar with linear transformations, it can be easily understood why this is referred to as an extension of an ellipse into higher dimensions. Intuitively, one can imagine flattening a unit sphere along one axis by applying $A = \begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix}$. This happens as solutions to the equation of a circle $N : x^{2} + y^{2} = 1$ are transformed through $A$ into solutions for $\displaystyle AN : {{ x^{2} } \over { 2 }} + y^{2} = 1$. Since the eigenvalues of $A$ are $\sqrt{2}^{2} , \sqrt{1}^{2}$, it is evident that the axes of ellipsoid $AN$ are naturally $\sqrt{2}(1,0)$ and $\sqrt{1}(0,1)$.

Linear Algebra

The reason for explicitly referring to eigenvalues as $\sigma_{i}^{2}$ when discussing ellipsoids is due to their close relationship with singular value decomposition (SVD). Singular value decomposition is a method that finds some $\sigma_{i}>0$, $v_{i} \in \mathbb{R}^{n}$, and $u_{i} \in \mathbb{R}^{m}$ satisfying

$$A v_{i} = \sigma_{i} u_{i}$$

for $A \in \mathbb{R}^{m \times n}$. As proved in the existence of singular value decomposition, $\sigma_{i}^{2}$ are the eigenvalues for $A^{T}A$, and the unit eigenvectors $u_{1} , \cdots , u_{m}$ are mutually independent. From this perspective, referring to $\sigma_{i} u_{i}$ as axes is a natural definition.

Generalization

As can be understood from the linear algebraic explanation, the concept of ellipsoids can also be generalized for $A \in \mathbb{R}^{m \times n}$. However, from the reader’s perspective, understanding the relationship between singular values and eigenvalues might be challenging, and the geometric meaning becomes significantly weakened. Therefore, an introduction to the definition concerning $A \in \mathbb{R}^{m \times m}$ was necessary. If one successfully grasps this abstract definition, they could accept a more general definition of ellipsoids concerning the rank $r = \dim C (A)$ of $A$ set to $\sigma_{r+1} = \cdots = \sigma_{m} = 0$. However, in this broader context, $\sigma_{i}^{2}$ can no longer be referred to as the eigenvalues of $A$, and talking about singular value decomposition, one would only have “some positive number $\sigma_{i}>0$” to mention.