Download - 2. Hearn - Optimization
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 1/39
Optimization and
Discrete Mathematics14 March 2011
Don Hearn
Program Manager
AFOSR/RSL
Air Force Research Laboratory
AFOSR
Distribution A: Approved for public release; distribution is unlimited. 88ABW-2011-0779
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 3/39
3
Outline
• Portfolio Overview
• Optimization Research
• Program Goals and Strategies
•
AF Applications Identified; Relation to Subareas• Program Trend
• Transitions
• Scientific Opportunities and Transformation
Opportunities; Relation of Other Agency Funding• Examples from the portfolio
• Summary
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 4/39
4
Optimization Research
• Starts with an important hard problem
• Optimization models are proposed
• Theoretical properties – existence anduniqueness of solutions, computationalcomplexity, approximations, duality, …
•
Solution methods – convergence theory, erroranalysis, bounds, …
• If successful, generalize!
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 5/39
5
Maximize (or Minimize) f(x)
Subject tog(x) ≤ 0
h(x) = 0
where x is an n-dimensional vector and g, h are vector functions.Data can be deterministic or stochastic and time-dependent.
The simplest (in terms of functional form) is the Linear Program:
Minimize 3x + 5y + z + 6u + ……..
Subject to3 x + y + 2z + …….. <= 152
2x + 7y + 4z +u ….. >= 130
12x + 33y + 2u ….. = 217
etc
Optimization Models
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 6/39
6
Textbook Nonlinear Example
• Local optima are a problem:
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 7/39
7
Program Goals & Strategies
• Goals –
• Cutting edge optimization research• Of future use to the Air Force
• Strategies –
• Stress rigorous models and algorithms , especially for
environments with rapidly evolving and uncertain data• Engage more recognized optimization researchers,
including more young PIs• Foster collaborations –
• Optimizers, engineers and scientists•
Between PIs and AF labs• With other NX programs• With NSF, other agencies
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 8/39
8
Future AF Applications Identified
• Engineering design, including peta-scale methods
• Risk Management, Experimental Design, T&E
• Real-time logistics
•Optimal learning/machine learning
• Meta-material design
• Satellite and target tracking
• Embedded optimization for control
• Biological and social network analysis
• …….. (to be continued)
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 9/39
9
Scientific Challenges andPotential Impacts
Some Scientific Challenges• Non-differentiable Optimization • Nonlinear Optimization for Integer and Combinatorial
Models • Simulation Optimization • Optimization for analysis of Biological and Social
Networks • Logistics • Risk and Uncertainty in Engineering
Potential Transformational Impacts• Meta-material design • Engineering optimization • AI, Machine Learning, Reinforcement Learning • Air Force Logistics
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 10/39
10
Program Trends
Analysis based optimization – Emphasis on theoretical resultsto exploit problem structure and establish convergenceproperties supporting the development of algorithms
Search based optimization – New methods to address very largecontinuous or discrete problems, but with an emphasis on the
mathematical underpinnings, provable bounds, etc
Dynamic, stochastic and simulation optimization– Newalgorithms that address data dynamics and uncertainty andinclude optimization of simulation parameters
Combinatorial optimization – New algorithms that addressfundamental problems in networks and graphs such asidentifying substructures
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 11/39
11
Recent Transition: Model Adaptive Searchfor detoxification process design
Dennis (Rice), Audet (École Polytechnique de Montréal), Abramson (Boeing)
•
The MADS algorithm,based on (Clarke)generalized derivatives,reduced the solutiontimes by 37% on this
problem in whichexpensive simulationswere need to evaluatethe objective function.
• There are a number ofother transitions for airfoil design, surgerydesign, etc.
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 12/39
12
New Meta Algorithms for EngineeringDesign Using Surrogate Functions
Dennis (Rice), Audet (École Polytechnique de Montréal), Abramson (Boeing)
14
13
12
11
10
9
8
0.25 Overlap to Diameter r
atio
21 ’ Cabin
Length
Gross Weight (lbs)
0.10 0.15 0.20 0.25 0.30
Rotor Overlap to Diameter Ratio
36.5
39.8
43.1
46.4 49.6 56.1 52.9 59.4
92000
90750
89500
88250
87710
Cabin Length (ft)
InfeasibleRegion
0.25 Overlap to Diameter r
atio
21 ’ Cabin
Length
0.25 Overlap to Diameter r
atio
21 ’ Cabin
Length
Gross Weight (lbs)
0.10 0.15 0.20 0.25 0.30
Rotor Overlap to Diameter Ratio
Rotor Disk Loading,
psf
36.5
39.8
43.1
46.4 49.6 56.1 52.9 59.4
92000
90750
89500
88250
87710
Cabin Length (ft)
InfeasibleRegion
Helicopter rotor tilt design at BoeingOriginal airfoil
Optimized airfoil, 89% noise reduction
Optimization of angle and graft size forbypass surgery
Current design Proposed y-graft design
Snow water equivalent estimation
Highly fragmenteddomainFeasible domainrepresented by thedark pixels.
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 14/39
14
Other OrganizationsThat Fund Related Work
• NSF Several projects jointly funded in the 4
subareas• ONR
•
2 projects (3 PIs) jointly funded –applications are in military logisticsand emergency planning
• DOE
• No joint grants. One PI has additionalfunding in energy management
• ARO • No joint project funding. One PI is on
Army MURI
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 15/39
15
International Collaborations
• Ecole Polytechnique de Montreal • Claude Audet (with US researchers at
Rice, Boeing, AFIT)
•
University of Waterloo • S. Vavasis & H. Wolkowicz
• SOARD
• Claudia Sagastizabal, RJ Brazil (with R.Mifflin, Wash. St.)
(plus many unsponsored collaborations)
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 16/39
16
Examples from the Portfolio
Di t S h
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 17/39
17
Discrete SearchTheory and Methods
– Jacobson, Howe and Whitley
• Study of landscapes in combinatorial problems
• Applications in radar scheduling, routing, general NP Hard problems such as TSP, Air Fleet assignment,homeland security, …
Ongoing research: Search Algorithms
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 18/39
18
Approach/Technical Challenges• Disconnect between theory and practice in search.• Current theory needs to be extended to account for
complex structures of real world applications.• Uncertainty and dynamism are not well captured by
current theory.
Accomplishments
Algorithms and complexity proof for a scheduling Proofs about plateaus in Elementary Landscapes Predictive models of plateau size for some classes of
landscapes based on percolation theory Recent results on next slide…
Ongoing research: Search Algorithmsfor Scheduling Under Uncertainty
Colorado State University, Howe & Whitley
orbital path
track
signal
L d A l i d Al ith D l t f
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 19/39
19
Landscape Analysis and Algorithm Development forPlateau Plagued Search Spaces
Colorado State University, Howe & Whitley
Traveling Salesman Problem(e.g., supply chain, logistics,circuit designs, vehicle routing)
• Generalized PartitionCrossover (GPX): technique forcombining two locally optimalsolutions to construct better
locally optimal solution in O(n) time
• Results: Analytical andempirical analysis shows GPXcan dominate state of the artmethods
Implications: Can dramatically change how TSP instances are
solved
O ti l D i A t All ti P bl
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 20/39
20
20
Optimal Dynamic Asset Allocation Problems:Addressing the Impact of Information Uncertainty & Quality
University of Illinois, Sheldon Jacobson
Payoff to Stakeholders:• Theory: New fundamental understanding of how data deficienciesand uncertainties impact sequential stochastic assignment problemsolutions.•Long-term payoff: Ability to solve problems that more closely mimic real-world systems driven by real-time data.
Goal: Create a general theoretical framework for addressing optimal dynamic asset
allocation problems in the presence of information uncertainty and limited information quality.
Technical Approaches:
Applications
Assignment Problems Screening and Protection Scheduling Problems
• Sequential Stochastic Assignment• Surrogate Model Formulation and Analysis• Dynamic Programming Algorithm Design
A l i B d O ti i ti
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 21/39
21
Analysis-Based OptimizationTheory and Methods
– Mifflin and Sagastizabal
• New theory leading to rapid convergence in non- differentiable optimization
– Freund and Peraire
• SDP for meta-material design
–
Srikant• Wireless Network Operation
I t t B kth h S li
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 22/39
22
Important Breakthrough: Superlinearconvergence for nonsmooth problems
Bob Mifflin (Washington State) andCladia Sagastizabal (CEPEL, Rio de Janeiro)
jointly supported with NSF and SOARD
Band gap optimization for wave
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 23/39
23
Band-gap optimization for wavepropagation in periodic media
MIT, Freund, Parrilo & Peraire
Design of materials that give
very precise control of light orsound waves
New in this work is the use ofSemi-Definite Optimization forband structure calculation and
optimization in phononiccrystals
Applications includedesign of elasticwaveguides, beamsplitters, acoustic lasers,perfect acoustic mirrors,vibration protectiondevices
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 24/39
24
Optimization of photonic band gapsMIT, Freund & Peraire
Goal: Design of photonic crystals withmultiple and combined band gaps
Approach: Solution Strategies
• Pixel-based topology optimization
• Large-scale non-convex eigenvalueoptimization problem
• Relaxation and linearization
Methods: Numerical optimization tools
• Semi-definite programming
• Subspace approximation method
•
Discrete system via Finite Elementdiscretization and Adaptive meshrefinement
Results: Square and triangular lattices
• Very large absolute band gaps
• Multiple complete band gaps
• Much larger than existing results.
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 25/39
25
Distributed SchedulingUIUC, R. Srikant
• Wireless Networks:
Links may not be able totransmit simultaneouslydue to interference.
• Collision: Twointerfering users cannot
transmit at the sametime
• Medium Access Control(MAC) Protocol:Determines which links
should be scheduled totransmit at each timeinstant.
• (Relates to RSL Complex Networks portfolio)
Main Result: The first, fully distributedalgorithm which maximizes networkthroughput and explicitly takes
interference into account.Theoretical Contribution: An optimalityproof that blends results from convexoptimization, stochastic networks andmixing times of Markov chains.
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 26/39
26
Dynamic and Stochastic Methods
• Stochastic Optimization –
– Marcus and Fu
• Simulation-based optimization
• Dynamic Optimization
– Powell, Sen, Magnanti and Levi
• Fleet scheduling, Refueling, ADP, Learning methods, Logistics
Ongoing (high-risk) research: Simulation-Based
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 27/39
27
Ongoing (high-risk) research: Simulation-BasedMethodologies for Estimation and Control
UMD (with NSF), S. Marcus & M. Fu
Convergence of Distribution toOptimum in Simulation-Based
Optimization
ACCOMPLISHMENTS
• Simulation-based optimizationalgorithms with probable convergence &efficient performance
• Evolutionary and sampling-basedmethods for Markov Decision processes
The Knowledge Gradient for Optimal Learning
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 28/39
28
Technical challenge
• Optimal information collection policies arecomputationally intractable.
• Efficient algorithm for capturing correlated beliefs,making it possible to solve problems withthousands of choices on small budgets.
Impact
• Used to find new drugs, improve manufacturingprocesses, and optimize policies for stochastic
simulators.
Challenge
•
Finding the best material, technology, operatingpolicy or design when field or laboratoryexperiments are expensive.
Breakthrough
• The knowledge gradient for correlated beliefs(KGCB) is a powerful method for guiding theefficient collection of information. Proving to be
robust across broad problem classes.
The Knowledge Gradient for Optimal LearningIn Expensive Information Collection Problems
Princeton University, Warren B. Powell
A knowledge gradient surface
An Optimization Framework for
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 29/39
29
An Optimization Framework forAir Force Logistics of the Future
MIT, Magnanti & Levi
•
Air Force Logistics –– 20 new initiatives to
revolutionize supply chainsand logistics
• DOD logistics + inventory costs
–$130 + $80 billion
• Magnanti and Levi (MIT) - fast real-time algorithms for
– Depot to base supply and jointreplenishment
• Future war space example -sensor asset allocation
• Workshop in 2010 onidentification of specific AFmodels for the future
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 30/39
30
Combinatorial Optimization
Butenko (Texas A&M), Vavasis (U. Waterloo),Prokopyev (U. Pittsburgh), Krokhmal (U. Iowa)
Underlying structures are often graphs; analysis is
increasingly continuous non-linear optimization
C h i b i h / t k
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 31/39
31
Cohesive subgroups in graphs/networksTexas A&M, S. Butenko
• Data set generated by a certain complex system –
Represent using graphs or networks:
Nodes = agents, Links = interactions
• Interested in detecting cohesive (tightly knit) groupsof nodes representing clusters of the system'scomponents
• (The cohesive subgroup notion was first introduced insocial network analysis; this relates to the RSLCollective Behavior and Socio-cultural Modelingportfolio.)
• Earliest mathematical models of cohesive subgroupswere based on the graph-theoretic concept of a clique
• Other subgroups include k-clubs
A li ti
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 32/39
32
Applications
• Acquaintance networks - criminal network analysis
• Wire transfer database networks - detecting moneylaundering
• Call networks - organized crime detection
• Protein interaction networks - predicting proteincomplexes
• Gene co-expression networks - detecting networkmotifs
• Internet graphs - information search and retrieval
A l f l t k
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 33/39
33
An example of a complex network
Protein-protein interaction map of H. Pylori
Maximum 2- and 3- Club
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 34/39
34
Maximum 2- and 3- ClubSubstructures
H. Pylori
Saccharomyces cerevisiae - Yeast
Finding hidden structures with convex optimization
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 35/39
35
Finding hidden structures with convex optimizationU. Waterloo, S. Vavasis & H. Wolkowicz
The following apparently random graph contains a subset of 5
nodes all mutually interconnected:
Recent data mining discoveries fromWaterloo:•Finding hidden cliques and bi-cliques inlarge graphs (see example)•Finding features in images from an imagedatabase•Clustering data in the presence ofsignificant background noise using convexoptimization
5-clique found via convex
optimization
Polyomino Tiling
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 36/39
36
Polyomino TilingU. Pittsburgh, O. Prokopyev
• Application is time delay control of phased arraysystems
• Jointly supported with Dr. Nachman of
AFOSR/RSE, motivated by work at the SensorsDirectorate:
• Irregular Polyomino-Shaped Subarrays for Space-Based Active Arrays (R.J. Mailloux, S.G. Santarelli, T.M. Roberts,
D. Luu)
Moti ation and Initial Approach
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 37/39
37
Motivation and Initial Approach
•Introducing time delay into phased array systemscauses significant quantization side lobes whichseverely degrade obtained pattern.
• Mailloux et al. [2006] have shown that an array of
`irregularly` packed polyomino-shaped subarrays haspeak side lobes suppressed more than 14db whencompared to an array with rectangular subarrays.
• We aim to model and solve the problem of designing
and arranging polyomino shaped subarrays usingoptimization techniques. First attempt maximizesinformation entropy.
16x16 Board Tiled with Tetrominoes
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 38/39
38
Maximize entropy. Minimize entropy.(Observe how centers of gravity
are aligned: order = less entropy)
16x16 Board Tiled with Tetrominoes
Summary
8/7/2019 2. Hearn - Optimization
http://slidepdf.com/reader/full/2-hearn-optimization 39/39
39
Summary
• Goals –
• Cutting edge optimization research• Of future use to the Air Force
• The primary strategy is to identify future AF applicationsthat will require important improvements in optimization
theory and new algorithms.
Questions?