1 smu course emis 8381 nonlinear programming january 14, 2012 hossam zaki
TRANSCRIPT
![Page 1: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/1.jpg)
1
SMU Course EMIS 8381
Nonlinear Programming
January 14, 2012
Hossam Zaki
![Page 2: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/2.jpg)
2
Outline
• Introductions– 5 minutes each
• Course Scope – What is an optimization
problem?
– Examplea
– Classification of Optimization Problems
• Course Syllabus– Description & Pre
requisites– Text & References– Calendar &Grading
• On Line Resources• Course Topics
• Course Context– DSS Example & Paradigm
– Career Roles & Questions
![Page 3: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/3.jpg)
3
Scope
![Page 4: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/4.jpg)
4
What is an Optimization Problem?
• Optimization problems involve the selection of values of a number of interrelated variables in such a way to optimize on one or more selection criteria designed to measure the quality of each selection
![Page 5: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/5.jpg)
5
Example: Price Optimization
• A distributor sells 100K products to 20K customers needs to determine every month the set of prices that will maximize margin while satisfying product and customer pricing rules such as – GbB (Good, better, Best), – PGS (Platinum, Gold, Silver)
• Sources of nonlinearity– Margin = (price – cost) * quantity– Quantity = a – b * price
![Page 6: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/6.jpg)
6
High Level Price OptimizationVolume
Price
V*
Market Response Model
Price Optimizer
Price
Margin $
Pm*
MG*
PriceIncreaseZone
PriceIncreaseZone
PriceDecreaseZone
PriceDecreaseZone
![Page 7: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/7.jpg)
7
Example: Portfolio Optimization
• Markowitz Model– Given stocks returns and covariances, find a
minimum variance portfolio that will generate a pre specified average return. That is, determine how much to invest in each stock to minimize risk at a certain level of return
– Sources of nonlinearity• Risk = w(i) * w(j) * cov(i,j)
![Page 8: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/8.jpg)
8
Example: Regression
• Given N(1000) observations, find the values of the (linear or nonlinear) model parameters that will minimize the distance between model and observed
• Sources of nonlinearity– Minimum [ Y (mode) – Y (observed)]2 – Y (model ) = a0 + sum ( aj * xj)
![Page 9: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/9.jpg)
9
Optimization Problem Statement
• Select the values of a vector x in Rn
• From a set of feasible vectors X in Rn
– that satisfy a group of algebraic constraints
• In such a way that will optimize the value of a real-valued function f(x)
![Page 10: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/10.jpg)
10
Terminology
• Decision Variables = x
• Objective Function(s) = f(x)
• Feasible Region = X– Constraints defining X
• Inequality constraints : g(x)<=0• Equality constraints : h(x)=0
![Page 11: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/11.jpg)
11
Optimization Problems Three Classifiers
Constraints Objective
Decision Variables
OptimizationProblems
![Page 12: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/12.jpg)
12
DV Attributes
Sequential
InfiniteDims
FiniteDims
Discrete
Continuous
DV
![Page 13: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/13.jpg)
13
Objective Attributes
Non Linear
Linear
NonDifferentiable
Differentiable
Closed form
From Simulation
Many
One
Objective
![Page 14: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/14.jpg)
14
Constraints Attributes
Stochastic
Deterministic
Nonlinear
Block Diagonal
Network
GUB
Simple Bounds
Linear
UnConstrained
Constraints
![Page 15: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/15.jpg)
1515
Optimization Problems
Constraints• Unconstrained• Constrained• Linear• Non linear • Simple bounds• GUB• Network• Block diagonal
Decision Variables (DV)• Discrete• Continuous• Finite • Infinite
Objective• One• Many• Differentiable• Non Differentiable• Closed Form• From Simulation• Linear• Non linear• Deterministic• Stochastic
Unconstrained
Continuous DV
Linear Constraints
One ObjDifferentiable Obj
© 2006, 2007 Zilliant, Inc. -- CONFIDENTIAL
Finite DV
Deterministic Obj
![Page 16: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/16.jpg)
16
Course Scope
• Decision Variable– Continuous– Finite dimensions
• Objective– Single– Closed form or From Simulation– Linear or Nonlinear– Deterministic
• Constraints– Deterministic
![Page 17: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/17.jpg)
17
Exercise # 1
1. Linear Programming
2. Goal Programming
3. Network Programming
4. Integer Programming
5. Non differentiable Optimization
6. Global Optimization
7. Dynamic Programming
8. Stochastic Programming
9. Quadratic Programming
10.Fractional Programming
11.Geometric Programming
12.Multi Objective Optimization
• Identify the attributes that define the following optimization problems and provide one simple example. Display the results in table format
![Page 18: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/18.jpg)
18
Course Syllabus
![Page 19: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/19.jpg)
19
Course Description
• This course discusses, presents and explains the most important methods and results used to model and solve nonlinear optimization problems.
![Page 20: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/20.jpg)
20
Prerequisites
• Advanced calculus (partial derivatives)
• Linear algebra (vectors and matrices).
• Knowledge on linear programming is helpful but not required (we will cover what we need).
![Page 21: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/21.jpg)
21
Text & References
• Text Book– "Nonlinear Programming: Theory and Algorithms" by, M. Bazaraa,
H. Sherali and C Shetty, 3rd Edition, John Wiley, ISBN
• References– “Handbooks in OR & MS, Volume 1, Optimization”, Nemhauser et al
editors, North Holand, Chapters I and III, 1989
– Dennis & Schnabel, “Numerical Methods for Unconstrained Optimization”, Prentice-Hall, 1983
– Gill, Murray and Wright, “Practical Optimization”, Academic Press, 1981
– D. Luenberger, “Linear and Nonlinear Programming”, 2nd Edition, Addison Wesely, 1984
– D. Bertsekas, “Nonlinear Programming”, 2nd Edition, Athena
Scientific, 1999
![Page 22: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/22.jpg)
22
Class Calendar
• Class (Section001) – Saturday 9 AM-11:50 PM
• 10 Minute break every 50 minutes
– Meets in 205 Junkins
• 14 Class Periods– First Class Period: January 14
• No Class: 3/10 (SB) & 4/7 (GF is on 4/6)
– Last Class Period: April 28
![Page 23: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/23.jpg)
23
Grading
• In-class Exams (70%)– Mid Term on 3/2 = (30%)– Final on 4/28 = (40%)
• Term Paper & Presentation (20%)– A nonlinear optimization topic, e.g. algorithm,
application and /or software demo not covered in class
– Due on 3/31
• Homework (10%)
![Page 24: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/24.jpg)
24
On Line Resources
![Page 25: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/25.jpg)
25
Mathematical Programming Glossary
http://glossary.computing.society.informs.org/index.php?page=N.html • General Information - A list of dictionaries, suggested methods of
citation, and contribution instructions. • The Nature of Mathematical Programming - See this for basic terms
and a standard form of a mathematical program that is used throughout this glossary.
• Notation - Read this to clarify notation. • Supplements - A list of supplements that are cited by entries. • Myths and Counter Examples - Some common and uncommon
misconceptions. • Tours - Collections of Glossary entries for a particular subject. • Biographies - Some notes on famous mathematicians. • Please remember this is a glossary, not a survey, so no attempt is
made to cite credits.
![Page 26: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/26.jpg)
26
1998 Nonlinear Programming Software Survey
• http://www.lionhrtpub.com/orms/surveys/nlp/nlp.html
• The information in this survey was provided by the vendors in response to a questionnaire developed by Stephen Nash. The survey should not be considered as comprehensive, but rather as a representation of available NLP packages. The listings are limited to products that fit the parameters of the survey as outlined in the accompanying article.
![Page 27: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/27.jpg)
27
Nonlinear ProgrammingFrequently Asked Questions
http://www-unix.mcs.anl.gov/otc/Guide/faq/nonlinear-programming-faq.html
• Q1. "What is Nonlinear Programming?" • Q2. "What software is there for nonlinear
optimization?" • Q4. "What references are there in this
field?" • Q5. "What's available online in this field?"
![Page 28: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/28.jpg)
28
NEOS Guide
http://www-fp.mcs.anl.gov/OTC/Guide/ • The Optimization Tree. Our thumbnail sketch of
optimization (also known as numerical optimization or mathematical programming) and its various sub disciplines.
• The Optimization Software Guide. Information on software packages from the book by Moré and Wright, updated for the NEOS Guide.
• Frequently Asked Questions on Linear and Nonlinear Programming. These are the FAQs initiated by John Gregory, now maintained by Bob Fourer as part of the NEOS Guide.
![Page 29: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/29.jpg)
29
MIT Nonlinear Course
• Prof. Dimitri Bertsekas http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-252JNon-linear-ProgrammingSpring2003/CourseHome/
![Page 30: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/30.jpg)
30
UCLA Nonlinear EE Course
• Prof. Lieven Vandenberghe
http://www.ee.ucla.edu/ee236b/
![Page 31: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/31.jpg)
31
Course Topics
![Page 32: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/32.jpg)
32
Course Topics
• Chapter 1– Intro
• Appendix A– Math Review
• Chapter 2– Convex Sets
• Chapter 3– Convex Functions
• Chapter 4– Optimality Conditions
• Chapter 6– Lagragian Duality
• Chapter 8– Unconstrained Opt
• Chapter 9 – Penalty and Barrier
• Chapter 10– Methods of Feasible
Directions
![Page 33: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/33.jpg)
33
Exercise # 2
Plot and solve graphically:
Maximize x1
Subject to
h1(x1,x2) = x12 – x2 + a = 0
h2(x1,x2) = – x1 + x22 + a = 0
For the following values of a:a = 1, 0.25, 0 and -1
![Page 34: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/34.jpg)
34
Course Context
![Page 35: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/35.jpg)
35
DSS Example
Simulation Model
StochasticShow-up rates
StochasticDaily
Demand
Stochastic Success Rates
Daily Resources
FeedbackCapacity
Configuration
OperationsCharacteristics
Optimization ModelBusiness Rules
Alternative Resources
Existing Capacity
Aggregate DemandProfile
Cost of newResources
Timing Constraints
What if
![Page 36: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/36.jpg)
36
Decision Support System Development Methodology
DV = Decision VariableDB = Database
9. Solve Real Problems
2. Develop DSS High Level Design
3. Formulate Mathematical Model
DV, Objective, Constraints, Goals
5. Estimate and/or Forecast needed data, e.g. demand
6. Develop or Select Solver, prepare input
files, connect to DB
8. Refine Formulation: Aggregate, Decompose,
Transform
7. Prepare & Solve Small Sample Problems
10. Prepare & analyze output.Validate results
12. Estimate Lift (Improvement)
ITERATE
11. Perform What-if
1. Understand Business Context and Formulate Problem Statement in English
4. Receive, Clean and Synthesize Business Data.
Create DB
![Page 37: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/37.jpg)
37
How will the course help u with career questions?
P
SVP
VP
Director
Senior Manager
Manager
Senior Scientist
Scientist, OR Specialist, Consultant
Which problems to invest $ in?
Reformulate?Which solver to use?
How to formulate?
Which parameters?
What is the impact of a data change?
What is the expected lift?
How many person-months?
How to prioritize tasks?
How to gain tech diff over competitors?
How to validate results?
Should we productize?
Develop in house or buy? How many scientists do we need?
Which algorithm?
Service or license?
![Page 38: 1 SMU Course EMIS 8381 Nonlinear Programming January 14, 2012 Hossam Zaki](https://reader030.vdocuments.us/reader030/viewer/2022033105/56649edd5503460f94bedc3c/html5/thumbnails/38.jpg)
38
Questions?