Optimization
In mathematics, finding the maximum and minimum values of a given function $f: X \to \mathbb{R}$ is referred to as optimization. When the objective function is directly related to real-world problems, it receives significant attention from the perspective of applied mathematics, and the importance of optimization theory in solving these problems is unquestionable.
Linear Programming
Standard Form
Simplex Method
Duality
Practice
- Solving Linear Programming Problems with Excel
- Solving Linear Programming Problems with Julia
JuMP.jl
- Solving Linear Programming Problems with Python
scipy
- Solving Linear Programming Problems with MATLAB
Optimization Toolbox
- Solving Linear Programming Problems with R
lpSolve
Nonlinear Programming
Gradient Descent
Proximal Algorithms
- Proximal Operator $\operatorname{prox}_{\lambda f} (\mathbf{x})$
- Proximal Minimization Algorithm $\mathbf{x}^{(k+1)} = \operatorname{prox}_{\lambda f}(\mathbf{x}^{(k)})$
- Proximal Gradient Method $\mathbf{x}^{(k+1)} = \operatorname{prox}_{\lambda g}(\mathbf{x}^{(k)} - \lambda \nabla f(\mathbf{x}^{(k)}))$
- PALM (Proximal Alternating Linearized Minimization)
Evolutionary Programming
Heuristics
Particle Swarm
References
- Luenberger. (2021). Linear and Nonlinear Programming (5th Edition)
- Matousek. (2007). Understanding and Using Linear Programming
- Vanderbei. (2020). Linear Programming(5th Edition)
All posts
- Gradient Descent in Mathematics
- Optimization Techniques in Mathematics
- Stochastic Gradient Descent
- Optimal Value: Maximum and Minimum
- Optimal Solution: Maximum and Minimum Factors
- Definition of Linear Programming Problem
- Linear Programming Problem in Equation Form
- Linear Programming Problem Basis Solution
- Proof of the Uniqueness of Base Solubility
- Proof of the Existence of an Optimal Solution in the Equation Form of Linear Programming Problems
- If an Optimal Solution Exists in Linear Programming Problems, One of Them is a Basic Feasible Solution
- Linear Programming: Dictionaries and Tableau
- Linear Programming: The Simplex Method
- Initialization and Auxiliary Problem in Simplex Method
- Infinity of the Objective Function in Linear Programming
- Simplex Method Cycling
- Simplex Method's Bland's Rule
- Linear Programming: Proof of the Fundamental Theorem
- Linear Programming Duality
- Proof of Weak Duality Theorem in Linear Programming
- Proof of Strong Duality Theorem in Linear Programming
- Solving Linear Programming Problems with Excel
- Solving Linear Programming Problems with Julia
- Solving Linear Programming Problems with Python
- Solving Linear Programming Problems with MATLAB
- Solving Linear Programming Problems with R
- Optimization Theory: Method of Lagrange Multipliers
- Secant Method: Newton's Method
- Second Order Necessary/Sufficient Conditions for the Extreme Values of Multivariable Functions
- First-Order Necessary Conditions for Extrema of Multivariable Functions
- Proximal Operator
- Proximal Minimization Algorithm
- Alternating Optimization
- Subgradient
- Subgradient Method
- Proximal Gradient Method
- Proximal Alternating Linearized Minimization Algorithm (PALM)