Matrix Algebra

This section focuses on matrices within linear algebra, covering systems of equations, matrix diagonalization, matrix decomposition, eigenvalue problems, and matrix transformations. Topics on general vector spaces and linear transformations can be found in the Linear Algebra category. Even if the content overlaps, articles in the Linear Algebra category may be more abstract or challenging. Numerical linear algebra from the Tistory branch has been integrated into this category.

Basic Matrix Algebra

• Table of Vector and Matrix Operations/Notation

Forms of Matrices

• Definition of Vectors
• Definition of Matrices
• Matrix Operations: Scalar Multiplication, Addition, Multiplication
  • Inner Product of Matrices $\mathbf{u}^{T} \mathbf{v}$
  • Direct Sum $B_{1} \oplus B_{2} := \diag \begin{bmatrix} B_{1} \ B_{2} \end{bmatrix}$
  • Kronecker Product $A \otimes B$
  • Hadamard Product $A \odot B$
  • Formula for Easily Computing the Sum of Elements in the Product of Two Square Matrices
• Identity Matrix $I, E$
• Zero Matrix $O$
  • Nilpotent Matrix $A^{k} = O$
• $1$ Matrix
• Square Matrix
• Triangular Matrix
  • Strictly Triangular Square Matrices are Nilpotent
  • Determinant of a Triangular Matrix
• Diagonal Matrix $\diag$
  • Row-wise and Column-wise Scalar Multiplication via Diagonal Matrix Multiplication
  • Trace $\tr A$
• Block Matrix $\begin{bmatrix} A & B \ C & D \end{bmatrix}$
  • Proof of Strassen’s Algorithm
• Sparse Matrix
  • Permutation Matrix $P$
• Hankel Matrix $H$

Properties of Matrices

• Inverse Matrix $A^{-1}$, Invertible Matrix
  • Equivalent Conditions for a Matrix to be Invertible
  • Similarity of Matrices
  • Pseudo-Inverse $A^{\dagger}$
  • Form of the Inverse Matrix of a 2x2 Square Matrix and the Inverse Function of a Fractional Function
• Transpose Matrix $A^{T}$
  • Symmetric Matrix $A^{T} = A$, Skew-Symmetric Matrix $A^{T} = -A$
• Conjugate Transpose Matrix $A^{\ast}$
  • Hermitian Matrix $A^{\ast} = A$
  • Normal Matrix $A A^{\ast} = A^{\ast} A$
  • Eigenvalues of Hermitian Matrices are Always Real
  • Eigenvectors Corresponding to Distinct Eigenvalues of a Hermitian Matrix are Orthogonal
  • Convex Cone of Hermitian Matrices and Positive Semi-Definite Matrices $\mathbb{H}{n}$, $\mathbb{P}{n}$
  • Löwner Order of Hermitian Matrices $A \ge B$
• Orthogonal Matrix $A^{T} = A^{-1}$
  • Properties of Orthogonal Matrices
  • Equivalent Conditions for a Matrix to be Orthogonal
  • Unitary Matrix $A^{\ast} = A^{-1}$
• Projection Matrix $P^{2} = P$
  • Orthogonal Projection
• Definite Matrix $\mathbf{x}^{\ast} A \mathbf{x} \ne 0$
  • Definite Matrices are Hermitian
  • Inverse and Square Root of Positive Definite Matrices
  • Proof of the Extended Cauchy-Schwarz Inequality for Positive Definite Matrices
• Exponential and Logarithm of Matrices $e^{A}$, $\log A$
• Real Powers of Positive Definite Matrices $A^{t}$
  • Square Root Matrix $A^{1/2} = \sqrt{A}$

Systems of Equations

• System of Linear Equations
• Overdetermined and Underdetermined Systems
• Augmented Matrix and Elementary Row Operations
• Gauss-Jordan Elimination Method
  • Algorithm to Find the Inverse Matrix Using Gaussian Elimination
• Homogeneous Systems of Linear Equations
• Elementary Matrices
• Inverse Matrices and Systems of Linear Equations

Determinants

• Determinant $\det$
  • Properties of Determinants
• Cofactor and Classical Adjoint Matrix $\text{adj} A = C^{T}$
  • Laplace Expansion
• Proof of Cramer’s Rule
• Derivation of the Determinant of a Vandermonde Matrix
• Derivation of the Determinant of a Tridiagonal Matrix
• Derivation of the Sherman-Morrison Formula $\left( A + \mathbf{u} \mathbf{v}^{T} \right)^{-1}$
  • Proof of the Matrix Determinant Lemma

Eigenvalue Problems

• Eigenvalues and Eigenvectors
  • Similar Matrices Have the Same Eigenvalues
• Algebraic and Geometric Multiplicities of Eigenvalues
  • The Algebraic Multiplicity of an Eigenvalue is Greater Than or Equal to Its Geometric Multiplicity
• Spectrum
  • Spectral Radius $\rho (A)$
    • Perron-Frobenius Theorem
• Generalization of Ellipses: Ellipsoids

Linear Transformations

• Matrix Transformations
• Rotational Transformations
  • Formula for Powers of a Rotation Matrix
  • 3D Rotation Matrices: Roll, Pitch, Yaw
• 🔒(25/02/02) Affine Transformation

Matrix Spaces

• Row Space, Column Space, Null Space
  • Bases for Row Space, Column Space, Null Space
  • Fundamental Spaces of a Matrix
• Rank and Nullity
  • Understanding Rank and Nullity Through Systems of Equations
  • Properties of Full Rank Matrices = Necessary and Sufficient Condition for the Invertibility of $X^T X$

Numerical Matrix Algebra

Matrix Decomposition

• Eigenvalue Diagonalization of Invertible Matrices $A = Q^{\ast} \Lambda Q$
  • Eigenvalue Diagonalization of Hermitian Matrices: Proof of the Spectral Theorem
  • Spectral Decomposition $A = \sum \lambda e e^{T}$
• Cholesky Decomposition of Positive Definite Matrices
  • Proof of Uniqueness of Cholesky Decomposition
• LU Decomposition of Invertible Matrices $A = LU$
  • PLU Decomposition $A = PLU$
• LDU Decomposition of Symmetric Matrices $A = LDL^{T}$
• Schur Decomposition of Square Matrices $A = QTQ^{\ast}$
• Singular Value Decomposition (SVD) of Matrices
  • Proof of Existence of Full Singular Value Decomposition
• QR Decomposition of Matrices

Least Squares Method

• Frobenius Norm $\left| \cdot \right|_{F}$
• Least Squares Method
  • Least Squares Method Using Singular Value Decomposition
  • Least Squares Method Using Cholesky Decomposition
  • Least Squares Method Using QR Decomposition

주요 참고문헌

• Stephen H. Friedberg, Linear Algebra (4th Edition, 2002)
• 김상동. (2012). 수치행렬해석
• Howard Anton, Elementary Linear Algebra: Aplications Version (12th Edition, 2019)