cse543t: algorithms for nonlinear optimization yixin chen department of computer science &...
TRANSCRIPT
![Page 1: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/1.jpg)
CSE543T: Algorithms for Nonlinear Optimization
Yixin ChenDepartment of Computer Science & Engineering
Washington University in St Louis
Spring, 2011
![Page 2: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/2.jpg)
Nonlinear Programming (NLP) problem
Minimize f(x)
(optional) Subject to h(x) =0
g(x) <= 0
Characteristics:– variable space X: continuous, discrete, mixed, curves
• Decision variables, control variables, system inputs
– function properties• Closed form, evaluation process, stochastic, …
![Page 3: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/3.jpg)
Why non-linear?
• Deal or no deal? You can choose– Take $2M and go home; Or,– Deal: could get $0.1 or $5M– What will you choose?
![Page 4: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/4.jpg)
Why study nonlinear optimization?
• Optimization is everywhere– Constraints are everywhere
– The world is nonlinear (curvature and local optima)
• Plenty of applications– Investment, networking design, machine learning (NN,
HMM, CRF), data mining, sensors, structural design, bioinformatics, medical treatment planning, games
• Nonlinear optimization is difficult– Curse of dimensionality, among many other reasons
![Page 5: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/5.jpg)
Example: Linear Regression
![Page 6: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/6.jpg)
Logistic Regression
• Y=0 or 1
• Where nonlinear functions clearly make more sense
• P(Y=1) = 1/(1+exp(-wTx))
• Find w to maximize
∏i=1..n P(Yi=1)Yi (1-P(Yi=1))(1-Yi)
![Page 7: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/7.jpg)
![Page 8: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/8.jpg)
SVM
• Minimize || w||
• subject to: yi(w xi - b) ≥ 1
![Page 9: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/9.jpg)
Sensor network optimization
• Variables– Number of sensors– Locations of sensors
• Functions– Minimize the maximum false alarming rate– Subject to minimum detection probability > 95%
– Highly nonlinear functions• Based on a Gaussian model and voting procedure
• Computed by a Monte-Carlo simulation
![Page 10: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/10.jpg)
Sodoku
![Page 11: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/11.jpg)
Goals of the course
• Introduce the theory and algorithms for nonlinear optimization– One step further to the back of the blackbox– Able to hack solvers – But not too much gory details of mathematics
• Hands-on experiences with optimization packages– Know how to choose solvers – Understanding of the characteristics of problems/solvers
• Mathematical modeling – Modeling languages– CSE applications
• Help solve problems in your own research
![Page 12: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/12.jpg)
Overview of the course
• Unconstrained optimization (6 lectures)
• Convex constrained optimization (3 lectures)
• General constrained optimization (12-14 lectures)– Penalty methods – Lagrange methods– Dual methods
![Page 13: CSE543T: Algorithms for Nonlinear Optimization Yixin Chen Department of Computer Science & Engineering Washington University in St Louis Spring, 2011](https://reader035.vdocuments.us/reader035/viewer/2022062518/56649ea45503460f94ba891e/html5/thumbnails/13.jpg)
Assignments and grading
• Three or four homework (40%)
• One project (30%) – In-class presentations – Reports
• One final exam (30%)