Course No.：S070100XJ012
Course Category：Basic course of subject
Period/Credits：40/2
Prerequisites：numerical analysis, linear algebra
Aims & Requirements：
This is one of the speciality foundation courses for master degree candidates majoring in computational mathematics and applied mathematics. This course explains basic theories and methods of optimization, discusses optimality conditions and computational methods of unconstrained and constrained optimization, and the convergence properties of different kinds of algorithms. In addition, this course introduces some special optimization problems and methods.
Participants of this course are hoped to have a general understanding of theories and methods of optimizations, and a preliminary command of the applications and basic techniques of the main optimization algorithms.
Primary Coverage：
Chapter 1 Introduction
Optimality conditions of optimization problems; structure of optimization methods
Chapter 2 one dimensional optimization and line search
Newton method；golden section method；inexact one-dimensional search
Chapter 3 gradient methods and conjugate gradient methods
Steepest descent method; conjugate direction method; conjugate gradient method
Chapter 4 Newton’s method and quasi-Newton method
Steepest descent method and Newton’s method; derivaiton of quasi-Newton methods; properties of quasi-Newton methods, special quasi-Newton methods.
Chapter 5 Non-linear least square problems
Gauss-Newton method; Levenberg-Marquardt method and trust region methods; quasi-Newton methods.
Chapter 6 quadratic programming
Duality properties; active set method; duality algorithm; interior-point algorithm.
Chapter 7 penalty function methods
Theories of penalty function; methods of penalty function.
Chapter 8 method of feasible directions
Method of feasible points and generalized elimination method; gradient projection method.
Chapter 9 sequential quadratic programming (SQP)
Lagrange–Newton method；Wilson-Han-Powell method；super-linear convergence of SQP；Marotos effect；special SQP type methods
Textbook：
Yuan, Yaxiang, Nonlinear optimization algorithms. Science Press. 2007.
References：
1. Yuan Yaxiang & Sun Wenyu, Optimization theories and methods. Science Press. Beijing.1997.
2. R..Fletcher，Practical Methods of Optimization, Second Edition，John Wiley and Sons，Chichester，1987。
3. J. Nocedal and S. Wright, Numerical Optimization, Springer, 1999。
Author：Yuan Yaxiang, Dai Yuhong（Academy of Mathematics and Systems Science）
Date：June, 2009.
