📂Matrix Algebra

Definite matrix

Definition¹

Positive Definite Matrix

A quadratic form $\mathbf{x}^{\ast} A \mathbf{x}$ is

• called a positive definite matrix or quadratic form if it satisfies $\mathbf{x}^{\ast} A \mathbf{x} > 0$ for all $\mathbf{x} \ne \mathbf{0}$.

• called a negative definite matrix or quadratic form if it satisfies $\mathbf{x}^{\ast} A \mathbf{x} < 0$ for all $\mathbf{x} \ne \mathbf{0}$.

• called indefinite if it sometimes satisfies $\mathbf{x}$ for the same quadratic form or matrix $A$.

For real matrices, one can think of replacing the part $\mathbf{x}^{\ast} A \mathbf{x}$ with $\mathbf{x}^{T} A \mathbf{x}$ in the definition.

Positive Semidefinite Matrix

A quadratic form $\mathbf{x}^{\ast} A \mathbf{x}$ is

• called a positive semidefinite matrix or quadratic form if it satisfies $\mathbf{x}^{\ast} A \mathbf{x} \ge 0$ for all $\mathbf{x} \ne \mathbf{0}$.

• called a negative semidefinite matrix or quadratic form if it satisfies $\mathbf{x}^{\ast} A \mathbf{x} \le 0$ for all $\mathbf{x} \ne \mathbf{0}$.

Explanation

Although these definitions are clean, a lot is omitted, making it hard to follow mentally. Let’s try to grasp the concepts gradually while looking at the formulas and explanations. Consider the case where the constants of the quadratic form are complex numbers, meaning $A$ is a Hermitian matrix. As shown in $A \mathbf{x} = \lambda \mathbf{x}$, $\lambda$ becomes the eigenvalue of $A$. Multiplying the left side of the equation by the conjugate transpose $\mathbf{x}^{\ast}$ results in:

$$ \mathbf{x}^{\ast} A \mathbf{x} = \lambda \mathbf{x}^{\ast} \mathbf{x} = \lambda \mathbf{x} \cdot \mathbf{x} = \lambda | \mathbf{x} |^{2} $$

Since $\mathbf{x} \ne \mathbf{0}$, this implies $|\mathbf{x}| ^2 > 0$, and since the eigenvalues of a Hermitian matrix are real, $\lambda |\mathbf{x}| ^2$ is also real. Thus, $\mathbf{x}^{\ast} A \mathbf{x}$ is real, and whether it is positive or negative can be determined. Although it may have been difficult to understand when written as a product of matrices and vectors, expressing it as $\lambda |\mathbf{x}| ^2$ makes it easier to comprehend.

Considering the sign of $\lambda |\mathbf{x}|^{2}$, since it is always $|\mathbf{x}|^{2} >0$, one only needs to consider the sign of $\lambda$. Ultimately, stating that for any non-zero vector the condition $\mathbf{x}^{\ast} A \mathbf{x} > 0$ is met means that all eigenvalues of $A$ are positive. Conversely, a negative definite matrix implies that all its eigenvalues are negative. Now, one can think of definiteness as defining the concept of negative/positive to matrices that originally do not have this concept. This is encompassed in Theorem 1.

Additionally, according to the equivalence condition of invertible matrices, both positive and negative definite matrices do not have $0$ as an eigenvalue, making them invertible matrices. (Theorem 2)

Applications

• In numerical linear algebra, there is particular interest in positive definiteness. Considering it as a condition, starting with a Hermitian matrix which, having all positive eigenvalues, is a very strong condition.
• In dynamics, the properties of negative definite matrices are used to study the stability of equilibrium points in the system.
• In statistics, it is fundamental that covariance matrices are positive semidefinite matrices, making them extremely important.

Theorem 1

For a quadratic form $\mathbf{x}^{\ast} A\mathbf{x}$,

• The necessary and sufficient condition for $\mathbf{x}^{\ast} A\mathbf{x}$ to be positive definite is that all eigenvalues of $A$ are positive.

• The necessary and sufficient condition for $\mathbf{x}^{\ast} A\mathbf{x}$ to be negative definite is that all eigenvalues of $A$ are negative.

• The necessary and sufficient condition for $\mathbf{x}^{\ast} A\mathbf{x}$ to be indefinite is that $A$ has at least one negative and at least one positive eigenvalue.

Theorem 2

Positive definite and negative definite matrices are always invertible.

Theorem 3

For a symmetric matrix $A$,

• If $A$ is positive definite, then $\mathbf{x}^{T}A\mathbf{x}=1$ is an equation of an ellipse.

• If $A$ is negative definite, then $\mathbf{x}^{T}A\mathbf{x}=1$ does not have a graph.

• If $A$ is indefinite, then $\mathbf{x}^{T}A\mathbf{x}=1$ is an equation of a hyperbola.