Matrix Algebra

This section focuses on the content related to matrices within linear algebra, including systems of linear equations, diagonalization of matrices, matrix decomposition, eigenvalue problems, matrix transformations, and more. Information on general vector spaces and linear transformations can be found in the Linear Algebra category. Even if the content is the same, the articles in the Linear Algebra category may be more abstract or complex.

Basic Matrix Algebra

• Operations and Notation Table for Vectors and Matrices

Types of Matrices

• Definition of a Vector
• Definition of a Matrix
• Matrix Operations: Scalar Multiplication, Addition, Multiplication
  • Inner Product of Matrices $\mathbf{u}^{T} \mathbf{v}$
  • Direct Sum $B_{1} \oplus B_{2} := \diag \begin{bmatrix} B_{1} \ B_{2} \end{bmatrix}$
  • Kronecker Product $A \otimes B$
  • Hadamard Product $A \odot B$
• Identity Matrix $I, E$
• Zero Matrix $O$
  • Nilpotent Matrix $A^{k} = O$
• $1$ Matrix
• Square Matrix
• Triangular Matrix
  • Square Nilpotent Triangular Matrix
  • 🔒 (23/11/30) Determinant of a Triangular Matrix
• Diagonal Matrix $\diag$
  • Matrix Multiplication by Diagonal Matrix for Row-wise or Column-wise Scalar Multiplication
  • Trace of a Matrix $\tr A$
• Block Matrix
  • Proof of Strassen’s Algorithm
• 🔒 (24/01/25) Sparse Matrix
  • Permutation Matrix $P$

Properties of Matrices

• Inverse Matrix $A^{-1}$, Invertible Matrix
  • Conditions Equivalent to a Matrix Being Invertible
  • Matrix Similarity
  • Moore-Penrose Pseudoinverse $A^{\dagger}$
• Transpose Matrix $A^{T}$
  • Symmetric Matrix $A^{T} = A$, Skew-Symmetric Matrix $A^{T} = -A$
• Conjugate Transpose Matrix $A^{\ast}$
  • Hermitian Matrix $A^{\ast} = A$
  • Eigenvalues of a Hermitian Matrix Are Always Real
  • Eigenvectors of a Hermitian Matrix Corresponding to Different Eigenvalues Are Orthogonal
• Orthogonal Matrix $A^{T} = A^{-1}$
  • Properties of Orthogonal Matrices
  • Conditions Equivalent to Being an Orthogonal Matrix
  • Unitary Matrix $A^{\ast} = A^{-1}$
• Projection Matrix $P^{2} = P$
  • Orthogonal Projection
• Positive Definite Matrix $\mathbf{x}^{\ast} A \mathbf{x} \ne 0$
  • 🔒 (24/06/23) A Positive Definite Matrix Is a Hermitian Matrix
  • 🔒 (24/07/01) Inverse and Square Root of a Positive Definite Matrix
  • 🔒 (24/07/25) Proof of the Extended Cauchy-Schwarz Inequality for Positive Definite Matrices
• 🔒 (24/01/29) Square Root of a Matrix $\sqrt{A}$
• 🔒 (24/04/30) Hankel Matrix $H$

Systems of Linear Equations

• Systems of Linear Equations
• Overdetermined and Underdetermined Systems
• Augmented Matrix and Elementary Row Operations
• Gauss-Jordan Elimination
  • 🔒 (24/02/02) Algorithm for Finding the Inverse Matrix Using Gaussian Elimination
• Homogeneous Systems of Linear Equations
• Elementary Matrices
• Inverse Matrix and Systems of Linear Equations

Determinants

• Determinant $\det$
  • Properties of Determinants
• 🔒 (23/11/26) Cofactor and Classical Adjoint Matrix $\text{adj} A = C^{T}$
  • Laplace Expansion
• Proof of Cramer’s Rule
• Derivation of the Determinant of a Vandermonde Matrix
• Derivation of the Determinant of a Tridiagonal Matrix
• 🔒 (23/11/18) Derivation of Sherman-Morrison Formula $\left( A + \mathbf{u} \mathbf{v}^{T} \right)^{-1}$
  • 🔒 (23/11/18) Proof of the Matrix Determinant Lemma

Eigenvalue Problems

• Eigenvalues and Eigenvectors
  • Similar Matrices Have the Same Eigenvalues
• Algebraic and Geometric Multiplicities of Eigenvalues
  • The Algebraic Multiplicity of an Eigenvalue Is Greater Than or Equal to Its Geometric Multiplicity
• [

Spectrum](../../posts/3614)

• Spectral Radius $\rho (A)$
  • Perron-Frobenius Theorem
• Generalization of an Ellipse: Ellipsoid

Linear Transformations

• Matrix Transformation
• Rotation Transformation
  • 🔒 (24/04/22) 3D Rotation Transformation Matrix: Roll, Pitch, Yaw

Matrix Spaces

• Row Space, Column Space, Null Space
  • Bases for Row Space, Column Space, and Null Space
  • Fundamental Spaces of a Matrix
• Rank, Nullity
  • Understanding Rank and Nullity Through Systems of Equations
  • Properties of Full Rank Matrices = Necessary and Sufficient Condition for the Existence of $(X^T X)^{-1}$

Numerical Matrix Algebra

Matrix Decomposition

• Eigenvalue Diagonalization of an Invertible Matrix $A = Q^{\ast} \Lambda Q$
  • Eigenvalue Diagonalization of a Hermitian Matrix: Proof of Spectral Theorem
  • 🔒 (24/06/27) Spectral Decomposition $A = \sum \lambda e e^{T}$
• Cholesky Decomposition of a Positive Definite Matrix
  • Proof of Uniqueness of Cholesky Decomposition
• LU Decomposition of an Invertible Matrix $A = LU$
  • PLU Decomposition $A = PLU$
• LDU Decomposition of a Symmetric Matrix $A = LDL^{T}$
• Schur Decomposition of a Square Matrix $A = QTQ^{\ast}$
• Singular Value Decomposition (SVD) of a Matrix
  • Existence of Full Singular Value Decomposition
• QR Decomposition of a Matrix

Least Squares Method

• Frobenius Norm $|\cdot|_{F}$
• Least Squares Method
  • Least Squares Method Through Singular Value Decomposition
  • Least Squares Method Through Cholesky Decomposition
  • Least Squares Method Through QR Decomposition

References

• Stephen H. Friedberg, Linear Algebra (4th Edition, 2002)
• Kim Sang-dong. (2012). Numerical Matrix Analysis
• Howard Anton, Elementary Linear Algebra: Applications Version (12th Edition, 2019)