global optimization: software and applications - sintef

1

Global Optimization Models, Algorithms, Software, and Applications

János D. PintérPCS Inc., Canada, and

Dept. of Ind. Eng., Bilkent University, Ankara, TurkeyPresented at the eVITA Winter School 2009, Geilo, Norway

2

Presentation Topics• The Relevance of Nonlinear and Global Optimization• General (Continuous) GO Model, and Some Examples • Review of Exact and Heuristic GO Algorithms• GO Software Implementations• Illustrative Applications and Case Studies• References• Software Demonstrations (as time allows, or after talk)

AcknowledgementsTo all developer partners, clients, and many other interested colleagues for cooperation and feedback

J.D. Pintér, Global Optimization eVITA Winter School 2009, Norway

3

• Decision making under resource constraints is a key paradigm in strategic planning, design and operations by government and private organizations • Examples: environmental management; healthcare; industrial design and production; inventory planning; scheduling, transportation and distribution, and many others• Quantitative decision support systems (DSS) tools −specifically, optimization models and solvers − can effectively assist decision makers and analysts in finding better solutions

Decisions, Models and Optimization


4

A KISS* Model Classification Convex Deterministic ModelsLinear Programming, Convex Nonlinear Programming (including special cases)

Non-Convex Deterministic ModelsContinuous Global Optimization, Combinatorial Optimization, Mixed Integer/Continuous Optimization (including special cases)

Stochastic ModelsGeneric Stochastic Optimization model; special cases that lead to LP, CP, and general NLP equivalents; and to “black box” models

Formally, both the convex and stochastic model-classes can be considered as subsets of the non-convex model class

Combinatorial models can also be formulated as continuous GO models; however, added specifications and insight are helpful

* Keep it Simple, Stupid… J.D. Pintér, Global Optimization eVITA Winter School 2009, Norway

5

Nonlinear Systems Modeling & Optimization • As the previous slide already indicates, nonlinear systems are arguably more the norm than the exception…• Why? Because nonlinearity is found literally everywhere: in processes leading to natural objects, formations, organisms, and in their interactions• This fact is reflected by descriptive models in applied mathematics, physics, chemistry, biology, engineering, econometrics and finances, and in the social sciences• Some of the most frequently used elementary nonlinear function forms: polynomials, power functions, exponential, logarithm, and trigonometric functions


6

Nonlinear Systems Modeling & Optimization(continued)

• Composite and more complicated nonlinear functions: special functions, integral equations, linear system of ordinary differential equations, partial differential equations, and so on

• Statistical models: probability distributions, stochastic processes

• “Black box” deterministic or stochastic simulation models, closed (e.g. confidential) models, models with computationally expensive functions,…

• Need for suitable descriptive system models, used in combination with control (optimization) methods


7

Examples of Basic Nonlinear Functions

A large variety of such functions exists: many of these are used to describe objects, and processesof practical relevance


8

Nonlinearity in Nature[A small collection of great photos from the Web]

Nature is clearly the most successful of all artists.Alvar Aalto, Finnish architect and designer (1898-1976)

9

Nonlinearity in Nature

Twisted Vines − Nature's Art, Malaysia, 2006© Thomas Allen, www.abstractechoes.com


10

Nonlinear Universe: Further ExamplesCredits: Scientific Computing & Instrumentation, 2004


11

Nonlinearity in Man-Made SystemsExample: Audio Speaker Design

Credits: “How Stuff Works” Website, 2005


12

Nonlinearity in Man-Made SystemsExample: Automotive Engine Design

Credits: “How Stuff Works” Website & Daimler-Chrysler, 2005

2003 Jeep® Grand Cherokee


13

Discovery SpaceshipA Man-Made System with Many Nonlinear Components

Credits: Robert Sullivan, New York Times, 2006


14J.D. Pintér, Global Optimization eVITA Winter School 2009, Norway

15

Notice that many of these application areas need NLP/GO (software)

NAG Survey on Technical Computing Needs


16

The Global Optimization Challenge:Theoretical Motivation

“The great watershed in optimization isn't between linearity and nonlinearity, but between convexity and nonconvexity.”

R. Tyrell Rockafellar

Lagrange multipliers and optimality,SIAM Review 35 (1993) 2, 183-238.


17

The Relevance of Global Optimization:Practical Motivation

“Theorists interested in optimization have been too willing to accept the legacy of the great eighteenth and nineteenth century mathematicians who painted a clean world of [linear, or convex] quadratic objective functions, ideal constraints and ever present derivatives.

The real world of search is fraught with discontinuities, and vast multi-modal, noisy search spaces...”

D. E. Goldberg (A well-known genetic algorithms pioneer)


18

The Relevance of Global Optimization• Optimization is often based on highly nonlinear descriptive models • Several important and general model-classes:

Provably non-convex models“Black box” systems design and operationsDecision-making under uncertaintyDynamic optimization models

• Nonlinear models frequently possess multiple optima: hence, finding their “very best” solution requires a suitable global scope search approach• The objective of global optimization is to find the absolutely best solution, in the possible presence of a multitude of local sub-optima


19

min f(x) f: Rn → R1

g(x) ≤ 0 g: Rn → Rm

l ≤ x ≤ u l, x, u, (l < u) are real n-vectors

Key (“minimalist”) analytical assumptions: • l, u are finite vectors; l ≤ x ≤ u is interpreted component-wise• the feasible set D={xl ≤ x ≤ xu: g(x) ≤ 0} is non-empty • f and the components of g are continuous functions

Continuous Global Optimization Model


20

• The structural assumptions stated on the previous slide are sufficient to guarantee the existence of the global solution set X*; for any x* in X*, define z* = f(x*)• They also support the application of theoretically exact, globally convergent search methods • In practice, we wish to find numerical estimates of x* or X*, and z*, using efficient global scope search methods• The CGO prototype model covers many special cases• Several examples follow on the next slides that hint at the potential difficulty of GO models

Continuous Global Optimization Model


21

A Mildly Non-Convex Function (a Classic NLP Test)

This function is projected into a 2-dimensional subspace, the other two coordinates are set at their optimal solution value; see figure

Notice that a local search method may end up in either one of two different “valleys”, depending on its starting point

[ ] )1)(1(8.19)1()1(1.10

)1()(90)1()(100)(min

422

42

2

23

234

21

212

−−+−+−+

−+−+−+−=

xxxx

xxxxxxxfx

Wood’s 4-variable polynomial test function


22

A Concave Minimization Problem

min f(x) x∈D; f = − x12 − 0.5⋅x1 − x2

2 − 0.3⋅x2 is concave; D=[−1,1]2

f attains its minimum at (1,1), and all vertices of D are local minima

-1

-0.5

0

0.5

1 -1

-0.5

0

0.5

1

-2

-1

0

-1

-0.5

0

0.5


23

GO Models Can Pose Difficult Challenges For Local Scope Search...

Example: minimize sin(x2+x)+cos(3x) for −5 ≤ x ≤ 2Local search can fail (local information is not sufficient)


24

GO Models Can Be Even More Difficult(In Principle, Arbitrarily Difficult...)

Obviously, a local view of such a function is not sufficient: instead, global scope search is needed

A GO model-instanceCited from the Handbook of Global Optimization, Vol. 2, Ch. 15


25

GO models can be extremely difficult to solve, even in (very) low-dimensions, if the search effort is limited… as in prefixed (default) GO solver settings

f(x)=x·sin(πx) 0≤x≤1000

A “Monster” GO Test Example


26

Another Inherent Issue in GO: “Curse of Dimensionality”

-10

-5

0

5

10-10

-5

0

5

10

-20

0

20

-10

-5

0

5

-10 -5 5 10

-15

-10

-5

5

10

Shubert’s one-dimensional box-constrained optimization model, and its simplest two-dimensional extensionComputational complexity increases exponentially, when the model size (n, m) grows

minâ

Σk=1,…,5 k sin(k+(k+1)x)

–10≤x≤10.

Σk=1,…,5 k sin(k+(k+1)x) + Σk=1,…,5 k sin(k+(k+1)y)

–10≤x≤10, –10≤y≤10..


27

Further Examples: Increasingly More Difficult (Parameterized) Test Functions

Example:x2+y2+c⋅sin2(x2+x+y2-y)x=-8..1, y=-3..10; c=1, 10, 100

Note: easy to modify, in order to generaterandomized solution points of model instances


28

A General Class of GO Models Formulated with DC Functions

f = f1−f2 if f1 and f2 are convex, then f is a DC function (the difference of convex functions)

A similar component-wise structure is postulated for gDC structure supports the general B&B algorithmic framework (to be discussed later on)However, a general DC structure is difficult to exploit (in terms of implementable algorithms), except for the case of general quadratic optimization under linear and quadratic constraints


29

Lipschitz(-Continuous) GO ModelsFunction f is Lipschitz-continuous on D, if there exists a suitable Lipschitz-constant L=L(D,f)≥0 such that|f(x1)-f(x2)| ≤ L||x1-x2 || holds for all pairs x1,x2∈D

Similar conditions can be postulated for all functions in g

The Lipschitz model-structure allows to generate lower bound estimates of the optimum value, based on an arbitrarily given finite sample set (next slide)

Based upon (mere) continuity and Lipschitz properties, a broad class of globally convergent algorithms can be axiomatically defined, designed and implemented, to solve general GO models − including all examples listed above(except “black boxes”, at least in theory)


30

Example: Feasible set in R3

defined by the constraints

x⋅y⋅z ≤ 1 x2+2y2+z2+x⋅z ≤ 2 3x2+2y2–(1–z)2 ≤ 0

Convex programming methods(direction and line searches) may fail on difficult non-convexfeasible sets

Tricky Feasible Sets Can Be Another Major Source of Difficulty…


31

The Formal Equivalence of Combinatorial and Corresponding Continuous GO Models 1

Each finitely bounded integer variable can be represented by a suitable set of binary variablesExample 1: all integer 0 ≤ x ≤ 15 values can be exactly represented by 4 binary variables, since 24 -1=15 For instance, 13 = 1*23+1*22+0*21+1*20 = 11012

Example 2: all integer 0≤x≤106 values can be described by at most 20 binary variables, since 220>106

Therefore it suffices to use binary variables instead of a given set of finitely bounded integer variables


32

The Formal Equivalence of Combinatorial and Corresponding Continuous GO Models 2Next, each binary variable can be represented jointly by its continuous extension, and a single non-convex constraintExample: x∈{0,1} (i.e., x is binary) is equivalent to the pair of relations 0 ≤ x ≤ 1 and x(1-x) ≤ 0; other formulations also existTherefore formally it suffices to use continuous variablesinstead of binary ones, and hence also instead of finitely bounded integers; consequently, the same applies also to the most general optimization problems defined with mixed integer-continuous variables and continuous functionsThis simple technical note shows a close connection between combinatorial and global optimization, both in terms of theiroverall complexity, and also regarding the classes of suitable solution strategies for such models


33

The Mixed Integer Global Optimization Challenge

Each binary variable selection (combination) induces a CGO sub-model

The overall numerical complexity is characterized by the combined complexity of combinatorial optimization and continuous global optimization… Hence, it is massively exponential as the model size characterized by n=nB+nC and m grows


34

Global Optimization Models: SummaryKey model types reviewed:• Concave Optimization (Minimization Over a Convex Set)• DC Optimization• Lipschitz Optimization• Continuous Optimization

These general model-classes cover all GO models of relevance, including also further specific cases

The following chain of set inclusions is valid:

{Concave GO}⊂{DC GO}⊂{Lipschitz GO}⊂{Continuous GO}

Recall also that CGO models cover mixed integer-continuous models


35

Global Optimization: Historical PerspectiveAn approximate timeline

Theory beginnings: 1950’s, foundations: 1970’s↓↑

Methods beginnings: 1960’s, key results: 1980’s↓↑

Software beginnings: 1980’s, professional: ~2000+↓↑

Applications GO needed for a long time, but only recently tackled by suitable GO tools

Ideally, all key components of knowledge are (should be) developed in close interaction


36

• A significant repertoire of GO methods, including both exact and heuristic approaches, have been suggested since ~1950; systematic studies conducted since ~1970• These solution methods differ with respect to the key analytical conditions of their applicability; and their proof of global convergence properties ─ or lack of it…• A brief review of some of the GO approaches is provided in the following slides

Global Optimization Strategies


37

• Related general in-depth references are offered by the Handbook of Global Optimization, Vol. 1 (Horst and Pardalos, Eds., 1995) and Vol. 2 (Pardalos and Romeijn, Eds., 2002); and in other volumes of the topical Kluwer(now Springer) book series; see also Neumaier’sreviews (2001, 2004)• For simplicity, we shall consider here the box-constrained GO modelmin f(x) l ≤ x ≤ u• Note that the presence of constraints could cause considerable grief to many of the GO approaches discussed below…

Global Optimization Strategies


38

GO Solution Approaches: Exact MethodsAdaptive Stochastic Search Methods

These are procedures based (at least partially) on randomized sampling in the feasible set D

Search strategy (parameter) adjustments, sample clustering, deterministic solution refinement options, statistical stopping rules can be added as enhancements to the basic (pure random) sampling scheme

Applicable to both discrete and continuous global optimization problems under general conditions

See e.g., Zhigljavsky (1991), Boender and Romeijn (1995), Pintér (1996), Zabinsky (2003)


39

Bayesian Search Algorithms

These methods are based on some a priori postulated stochastic model of the objective function f

The subsequent adaptive estimation of the problem-instance characteristics is based upon the search (sample points and function values) results, towards building a posterior function (problem) model

Typically, “myopic” (one-step optimal) approximate decisions govern the search procedure, since only these can be implemented

Applicable to continuous GO models, w/o added Lipschitz or other structural assumptions

Consult, e.g., Mockus, Eddy, Mockus, Mockus and Reklaitis (1996), Sergeyev and Strongin (2000)


40

Branch and Bound Algorithms

Adaptive partition, sampling, and bounding procedures (within subsets of the feasible set D) can be applied to continuous GO models, analogously to the well-known integer linear programming methodology

This general approach subsumes many specific cases, and allows for significant flexibility in implementations

Applicable to diverse structured GOPs such as concave minimization, DC programming, and Lipschitz problems

Consult, e.g., Neumaier (1990), Hansen (1992), Ratschek and Rokne (1995), Horst and Tuy (1996), Kearfott (1996), Pintér (1996), Tawarmalani and Sahinidis (2002)


41

Enumeration Strategies

These are based upon a complete (streamlined) enumeration of all possible global or local solutions

Applicable to combinatorial optimization problems, and to certain structured continuous GOPs such as e.g.,concave minimization models

Consult, e.g., Horst and Tuy (1996)


42

Homotopy and Trajectory Methods

These strategies have the ambitious objective of visiting all stationary points of the objective function f, within the set D; then checking for minima, maxima, saddle points

This search effort then leads to the list of all - global as well as local - optima (the latter being a subset of the stationary points)

In principle, applicable to smooth GO problems, but the numerical demands can be very substantial

Consult, for instance, Diener (1995) and Forster (1995)


43

Integral Methods

These methods are aimed at the determination of the essential supremum of the objective function f over D, by numerically approximating the level sets of f

Consult, e.g., Zheng and Zhuang (1995), or Hichert, Hoffmann and Phú (1997)


44

Naïve Approaches

These include both passive (simultaneous) grid search and passive (pure) random search

Note that these basic (and similar) methods are obviously convergent under mild analytical assumptions, they are truly “hopeless” in solving higher-dimensional problems(already for n = 3 or more)

For more details, see for instance Zhigljavsky (1991) or Pintér (1996), with further references therein


45

Relaxation (Outer Approximation) Strategies

In this general approach, the GOP is replaced by a sequence of relaxed sub-problems that are easier to solve

Successive refinement of sub-problems to approximate the initial problem is applied: cutting planes and more general cuts, diverse minorant function constructions, and other customizations are possible

Applicable to diverse structured GO models such as concave minimization, or DC programming

See, e.g., Horst and Tuy (1996), or Benson (1995)


46

GO Solution Approaches: Heuristic Methods

These often offer a “plausible” approach to handle difficult models, but without any theoretical justification (global convergence guarantee)

Approximate Convex Underestimation

This strategy attempts to estimate the (possible large-scale, overall) convexity characteristics of the objective function based on directed sampling in D

Applicable to smooth GO problems

See Dill, Phillips and Rosen (1997), and some related studies in classical (local) optimization studies


47

Continuation Methods

These approaches first transform the objective function into some more smooth, simpler function with fewer local minimizers, and then use a local minimization procedure to (hopefully) trace all minimizers back to the original function



48

Genetic Algorithms, Evolution Strategies

These “adaptive population” based heuristic approaches mimic biological and social evolution models (including e.g. ant colonies, memetic and other algorithmic approaches)

Various deterministic and stochastic algorithms can be constructed, based on diverse “evolutionary” rules

These strategies are applicable to both discrete and continuous GO problems under mild structural requirements; typically, customization is needed

Consult, e.g., Michalewicz (1996), Osman and Kelly (1996), Glover and Laguna (1997), or Voss, Martello, Osman and Roucairol (1999)


49

A General Framework for Population-Based Strategies

Initial population of sample points

Iteration cycle steps:• Competitive selection, drop the poorest solutions • The remaining pool of points with higher fitness value can be recombined with other solutions, by swapping components with another• The active points can also be mutated by making some (stochastic) change to a current point• Recombination and mutation moves are applied sequentially, in each major iteration cycle• Check algorithm stopping criteria: stop, or return to execute next major iteration cycle


50

Sequential Improvement of Local Optima

These approaches ─ including tunneling, deflation, and filled function methods ─ operate on adaptively definedauxiliary functions, to assist the search for improving optima


Consult, for instance, Levy and Gomez (1985), and their many followers (Ge Renpu and others)


51

Simple Globalized Extensions of Local Search Methods

These “pragmatic” strategies are often based on a rather quick global search (e.g. a limited passive grid or random search) phase, followed by local scope search

Applicable to smooth GO problems: differentiability is typically postulated (only), to guarantee the convergence of the local search component

However, global convergence is guaranteed only by the global scope search phase (which could be inefficient in a rudimentary implementation)

Consult, for instance, Zhigljavsky (1991) or Pintér (1996)


52

Simulated Annealing

SA is based upon the physical analogy of cooling crystal structures that spontaneously arrive at a stabilized configuration, characterized by (globally or locally) minimal potential energy

Applicable to both discrete and continuous GOPs under mild structural requirements

See, for instance, Osman and Kelly (1996), or Glover and Laguna (1997)


53

Tabu Search

The essential idea of this meta-heuristics is to forbid search moves towards points already visited in the (usually discrete) search space, within the next few steps, as governed by the algorithm

Tabu search has been mainly used so far to solve combinatorial optimization problems, but it can also be extended to handle continuous GOPs

Consult, e.g., Osman and Kelly (1996), Glover and Laguna (1997), or Voss, Martello, Osman and Roucairol (1999)


54

GO Solution Approaches: Concluding NotesObserve that overlaps may (in fact, do) exist among the algorithm categories listed above Both exact and heuristic methods could suffer from drawbacks: “overly sophisticated for practice” vs. “simplistic” approaches and their implementationsSearch strategy combinations are often both desirable and possible: this, however, leads to non-trivial issues in algorithm design


55

Global OptimizationSoftware Development“Those who say it cannot be done should not

interrupt those who are busy doing it.”Chinese proverb

"It does not matter whether a cat is black or white, as long as it catches mice."Deng Xiaoping

“I don't want it perfect, I want it Tuesday.”J.P. Morgan


56

GO Software Development Environments• General purpose, “low level” programming languages: C, Fortran, Pascal, … and their modern extensions

• Business analysis and modeling: Excel and its various extensions and add-ons (Excel PSP, @RISK,…)

• Specialized algebraic modeling languages with a focus on optimization: AIMMS, AMPL, GAMS, LINGO, LPL, MPL,…

• Integrated scientific and technical computing systems: Maple, Mathematica, MATLAB,…

• Relative pros and cons: instead of a “dogmatic” approach, one should choose the most appropriate platform considering user needs and requirements


57

GO Software: State-of-Art in a Nutshell 1

• Websites (e.g., by Fourer, Mittelmann and Spellucci, Neumaier, NEOS, and others) list discuss research and commercial codes: examples of the latter listed below

• Excel Premium Solver Platform: Evolutionary, Interval, MS-GRG, MS-KNITRO, MS-SQP, OptQuest solver engines

• Modeling languages and related solver options AIMMS: BARON, LGOAMPL: LGOGAMS: BARON, DICOPT, LGO, OQNLPLINGO: built-in global solver by the developers; also in

What’sBest! for spreadsheetsMPL: LGO


58

GO Software: State-of-Art in a Nutshell 2

• Integrated scientific-technical computing environments Maple: Global Optimization Toolbox (LGO for Maple)Mathematica: Global Optimization (package), MathOptimizer, MathOptimizer Professional (LGO for Mathematica), NMinimizeMatlab: GADS Toolbox TOMLAB solvers for MATLAB: CGO, LGO, OQNLP

Detailed information and references: • Developer websites• Handbook of GO, Vol. 2, Chapter 15• Neumaier’s GO website


59

• LGO is introduced here as an example of GO software• LGO offers a suite of global and local nonlinear

optimization algorithms, in an integrated framework• Globally search methods (solver options):

continuous branch-and-bound adaptive random search (single-start)adaptive random search (multi-start)exact penalty function applied in global search phase

• Local optimization follows from the best global search based point(s), or from a user-supplied initial point, by the generalized reduced gradient method

LGO (Lipschitz Global Optimizer) Solver Suite: Summary of Key Features


60

• LGO can analyze and solve complex nonlinear models, under minimal analytical assumptions

• Computable values of continuous or Lipschitz model functions are needed only, without higher order information

• Hence, LGO can be applied also to completely “black box” system models, defined by continuous functions

• Tractable model sizes depend only on hardware and time… however, the inherent massive complexity of GO problems remains a challenge (for all GO software products)

LGO: Summary of Key Features (continued)


61

• LGO reviews in ORMS Today, Optimization Methods and Software; various other LGO implementations reviewed in ORMS Today, Scientific Computing, Scientific Computing World, IEEE Control Systems Magazine, Int. J. of Modeling, Identification and Control, and in AlgOR

• LGO is currently available to use with C/C++/C# and Fortran compilers; with links to AIMMS, AMPL, GAMS, Excel and MPL; and with links to Maple, Mathematica, and Matlab

• MPL/LGO demo accompanies Hillier & Lieberman OR textbook (from 8th edition, 2005)

• LGO demos for C/C#/Excel/Fortran,… are available upon request

LGO: Summary of Key Features (continued)


62

LGO Solver Suite: Technical Background Notes• LGO offers a suite of global and local nonlinear optimization algorithms, in an integrated framework• This approach is dictated by the demands of (many, although not all) GO software users who need to solve their optimization problems relatively quickly• The global search components are (theoretically) globally convergent, either deterministically, orstochastically (with probability 1) • The local search component aims at finding KKT pointsthat satisfy the necessary local optimality conditions• This flexible combination of strategies leads to globaland local search based numerical solutions


63

Lipschitzian minorant construction, based on a given sample point sequence and the Lipschitz-constant (overestimate); the basis for a B&B algorithm

. Lipschitz lower bound

Lipschitz Function and its Minorant: An Example

|_____|___|____|_____|_____|_____|____|_____|____|_____|__|____|____|_____|_____|

Search interval: the function values at the sample points | are shown above by dots


64Partition sets, sample points, and selected subset

Example 2: Adaptive Partition and Sampling in R2


65

Credits: intpakX v1 - User's Guide, by Markus Grimmer, University of Wuppertal, Germany © 1999-2005 Scientific Computing/Software Engineering Research Group

Example 3: Interval Arithmetic Package (in Maple)


66

• Exact deterministic methods have the advantage of guaranteed quality of the solution found. Examples: branch-and-bound strategies, including interval methods• However, the computational demand of such methods in the worst case is exponential in n and m. Essentially, no method is better for the worst possible function(s) f than passive grid search… • In practice, to verify optimality and to sufficiently reduce the gap between the incumbent solution and the guaranteed lower bound can be very demanding: this may not be acceptable in certain (as a matter of fact, in numerous) practical applications

Deterministic vs. Stochastic GO Methods


67

God is subtle but he is not malicious…Albert Einstein

• Models to solve typically come from a “random source” (typically with unknown statistical features)• This is a key motivation to look for alternative solution approaches, including stochastic search algorithms • We will highlight the basics, and then some more advanced uses of stochastic search strategies

Deterministic vs. Stochastic GO Methods


68

The Power of Stochastic Search: An Illustrative Example

A complicated multi-modal function, and 1,000 random sample pointsPure (passive) random search will find points in an arbitrarily small neighborhood of the global solution x*, if the sampling effort tends to infinityAdaptive search strategies and statistical modeling tools become essential in higher dimensions, to improve search efficiency

-2

0

2-4

-2

0

2

0

20

40

-2

0

2

-3 -2 -1 0 1 2 3

-4

-3

-2

-1

0

1

2

-3 -2 -1 0 1 2 3

-4

-3

-2

-1

0

1

2


69

• Global convergence of pure random search (w.p. 1) over D (assuming that f is continuous, and D has a suitable, but still very general topological structure)

• Global convergence of adaptive random search• Global convergence of stochastically combined (sub)algorithms, assuming the “sufficiently frequent” usage of a globally convergent algorithm component

Stochastic Search Methods: Some Key Theoretical Results


70

• The following set of slides serves to illustrate various features of (LGO) software implementations • Some of the key features apply also to other GO software implementations, mutatis mutandis• The examples also hint at the capabilites and the limitations of GO software (as of today)

GO Software Implementations: Illustrative Examples


71

A Simple-to-Use LGO Demo (C, C#)

72

LGO Demo: Example poly+trigRefer to previous slide where this model is solved

Model formulation and bounds given in *.mod and *.bds text files

Example 1

Model: cited from poly+trig.mod

0.1*x[0]*x[0] + Math.Sin(x[0]) * Math.Sin(100*x[0]) objective fct

Bounds: cited from poly+trig.bds

0 lower bound3 nominal value5 upper bound


73

LGO Demo: Example poly+trig

Global solution argument found ~1.30376; optimum value ~-0.79458

Notice the many suboptimal solutions, including severalthat are close to the globally optimal solution value


74

LGO Demo: Example 2

75

LGO Demo: Example 2Model: log+trig.mod

Math.Log(1+x[1]*x[1]) + 100*Math.Pow(Math.Sin(x[0]*x[1]),2) objectiveMath.Pow(x[1],3) + 1 - Math.Pow(x[0]*x[0]-1,2) constraint10 equalityx[1]*x[0] + x[0] – 1 constraint2-1 inequality

Bounds: log+trig.bds

-2 lower bound0 nominal value3 upper bound-5 lower bound0 nominal value13 upper bound


76

LGO Demo: Example 2

The unique global solution is x[0]=x[1]=0, f*=0

Two views of the objective function in log+trig.mod (from previous slide)


77

LGO Integrated Development Environment

LGO IDE works with C and Fortran compilers

78

LGO Link to Excel

Model Formulation Example:Circuit Design Problem

Note: This development work is independent of the Excel Solverdevelopment by Frontline Systems

79

LGO Link to Excel

80

Model Development and Solution by AIMMS/LGO


81

AIMMS/LGO Solver Link Options

82

An AMPL ModelSolved by LGO

83

GAMS Preprocessing

LGO Solver ResultSummary

84

An MPL/LGO Model and its Solution

85

GO in Integrated Scientific and Technical Computing Systems• Maple, Mathematica, Matlab (and some others that are more specific to certain engineering or scientific fields)• Model prototyping and development: simple and advanced calculations, programming, documentation, visualization,… supported in “live” interactive documents• Data I/O and management features• Links to external software products• Portability across hardware and OS platforms• “One-stop” tools for interdisciplinary development• ISTCs are particularly suitable for developing complex, advanced nonlinear models; obvious GO relevance• Several articles discuss our implementations (refs later)

86

MathOptimizer Model

Note that dense nonlinear models (including many GO models) are similarly formulated across platforms: relatively easy model conversions, converters available in several cases (example: GAMS CONVERT utility)

87

An example from the MathOptimizer User Guide:Surface and contour plot of a randomly generated test function

Note: MO is a native Mathematica solver product, as opposed to the LGO implementations reviewed here

Advanced Visualization Tools in ISTCs


88

User Guide can be invoked from Mathematica’s online Help menu

The same applies to MathOptimizer

This featuresupports efficientprototyping and modular development

90

Model setup, solution and visualization in Matlab

TOMLAB /LGO solver

91

Model formulation, numerical solution, and visualization,using the GO Toolbox for Maple

92

Branin’s Test Problem with Multiple (3) Global Solutions

The GOT can also be used to find a sequence of global solutions


93

Maple GO Toolbox: Optimization Assistant

See e.g. Optimization Methods and Software (2006)J.D. Pintér, Global Optimization

eVITA Winter School 2009, Norway

94

Maple GO Toolbox: Optimization Plotter


95

Illustrative Case Studies A Concise Summary

• Many of the actual client case studies reviewed here are based on multi-disciplinary research, in addition to the global optimization component• All detailed case studies can (could) be presented in full detail, each in a separate lecture… instead, we shall briefly review a selection of these • References, demo software examples, publications, and additional details are all available upon request


96

Illustrative Case Studies reviewed in this talk (as time allows)

• An illustrative “black box” client model • Trefethen’s HDHD Challenge, Problem 4• Systems of nonlinear equations• Optimization problems featuring numerical procedures• Nonlinear model fitting examples• Experimental design• Non-uniform circle packings, and other packings• Computational chemistry: potential energy models• Portfolio selection, with a non-convex purchase cost• Solving differential equations by the shooting method• Data classification and visualization• Circuit design model • Rocket trajectory optimization


97

Illustrative Case Studies reviewed in this talk (as time allows)

• Industrial design model examples• Collision (trajectory) analysis• Design optimization in robotics • Laser design • Cancer therapy planning• Sonar equipment design• Oil field production optimization• Automotive suspension system designand other areas

• In addition, many standard NLP/GO and other test problems have been used to evaluate solver performance across the various modeling environments reviewed here• Experiments conducted by developer partners and clients, in addition to the author’s own work


98

“Black Box” Model Received from Client: “Can your software handle this problem?…”

99

-10

-5

0

5

10 -10

-5

0

5

10

0

20

40

-10

-5

0

5

10

-4-2

02

4-4

-2

0

2

4

0

5

10

15

-4-2

02

4

Trefethen’s HDHD Challenge, Problem 4 (SIAM News, 2002)


Looks easy from far away, and very difficult when more details are seen

100

HDHD Challenge, Problem 4 continued

• This model has been used as a test for LGO, as well as with MathOptimizer, MathOptimizer Pro, TOMLAB /LGO, and the Maple GOT• The solution found by all listed implementations is identical to more than 10 decimals to the announced solution; the latter was originally based on an enormous grid sampling effort combined with local search

x*~ (-0.024627…, 0.211789…)f*~ -3.30687…

• Close-up picture near to global solution: still looks quite difficult...


101

Example solved by Maple GO Toolboxx-y+sin(2*x)-cos(y)=04*x-exp(-y)+5*sin(6*x-y)+3*cos(3*y)=0

Solving Systems of Nonlinear Equations

Error function plot

A numerical solution found is x ~ 0.0147589760525313926, y ~ − 0.712474169476650099l2-norm error ~ 1.22136735435643598 <10-16

Note: there could be other solutions; systematic search is possible

Equivalent GO model formulation assuming that solution exists; else minimal norm solution sought

F(x)=0 ↔ min ||F(x)||

Optimization Methods and Software (2006)J.D. Pintér, Global Optimization


102

Nonlinear Equations

Observe the equations (lines in x-y subspace projection), and the solution point found asindicated by the green dot

103

Optimization with Arbitrary Computable Model Functions

A unique - and practically important - feature

104

Optimization of A Parametric Integral Expression> objf := a->2*evalf(Int(0.01*x+sin(5*x)*cos(3*x)-sin(3*x)*cos(2*x)+sin(x)*cos(7*x), x=-1.5*a..2*a));> bounds := -3..10;> GlobalSolve(objf, bounds)

Result found by the GOT: a*~ 4.92574563473295957 f*~-1.86463327469610008; here f*=objf(a*)Total number of function evaluations: 1351Runtime in external LGO solver: 14 seconds (integration takes time for each input value a)

> plot(objf, bounds);

The plot shows the high multi-modality of the parametric integral value (as a function of a)Notice the location of the global solution (a*,f*)


(a*,f*)

objf

a

105

Nonlinear Model Calibration in Presence of Noise

An example model (in Mathematica notation), inspired by a client’s (medication dosage effect) study:

a+Sin[b*(Pi*t)+c]+Cos[d*(3Pi*t)+e]+Sin[f*(5Pi*t)+g]+ξ

The parameters a,b,c,d,e,f,g are randomly generated from interval [0,1]; ξ is a stochastic noise term from U[-0.1,0.1]

Subsequently, the optimal parameterization is recovered numerically by MathOptimizer: this gives superior results, in comparison with Mathematica’s corresponding built-in local solver functionality (NonlinearFit)

Optimization Methods and Software (2003)J.D. Pintér, Global Optimization


106

1 2 3 4 5 6

-2

-1

1

2

3

a+Sin[b*(Pi*t)+c]+Cos[d*(3Pi*t)+e]+Sin[f*(5Pi*t)+g] +ξ

Calibration of Nonlinear Model in Presence of Noise (cont.)

Global search based fit, obtained by MathOptimizerJ.D. Pintér, Global Optimization


107

Arrhenius Probe Model CalibrationCredits: Grigoris Pantoleontos et al., Chemical Engineering Department, Aristotle University of Thessaloniki, Greece

ln(y) = A-Ea /RT Arrhenius formulaDescribes temperature dependence of reaction rate coefficient y

A multi-component version of the formula:

Here Ri[j] is calculated from another (rather complicated) expression

The study by GP et al. is aimed at the determination of the parameters ci, Ai and Ei i=1,2,3 by comparing the computed model output values to the experimental ones

The figure shows the initially given data points (red circles), the component curves (green, blue, yellow), and the resulting curve (bold blue); a fairly good fitThe solution of this computationally intensive example (9 variables to calibrate, very large search region, hundreds of data points, difficult model functions to compute) took about an hour on a desktop PC (in 2007); GP used the Maple GOT

[ ]( )( ) [ ]( )jR1jRTemp/EexpAcy iiiii −⋅−=


109

Data Classification (Clustering) by Global OptimizationDetails: Global Optimization in Action, Ch. 4.5

Classification objective: Find the “most homogeneous” or “most discriminative” grouping of a given set of entities (see black dots shown in rhs figure)

This can be done numerically, by globally optimizing the position of cluster centres (medoids, see red dots) For any given (candidate) medoid configuration, one can use e.g. the “nearest neighbor” rule to associate the points xj with the cluster centres ck

Example:400 3-dimensional points classified into 4 clusters


110

Data Classification (Clustering) by Global Optimization

Key advantage of the GO model formulation compared to the usual combinatorial optimization based approach: model dimension is ndim*nclusters=12... vs. nentities=400... The example is solved in seconds, the approach also scales up well

For a given (prefixed) number of clusters, one can use the following model to identify the cluster medoids {ck}:

min ΣkΣj||ck−xj|| s.t. clk≤ck≤cuk

For any given set of {ck} and each xj, the index k=k(xj) is chosen followingthe “nearest neighbor” rule

In general, this is a GO problem

Model description and detailed discussion with numerical examples in Global Optimization in Action, Ch. 4.2

111

Maxi-Min and Related Point ArrangementsIn a large variety of applications, one is interested in the “best possible covering” arrangement of points in a set • numerical approximation methods• design of experiments for expensive “black box” models• potential energy models (in physics, chemistry, biology)• crystallography, viral morphology, and other areas

For illustration, consider a maxi-min model instancemax { min ||xi - xk || } xl ≤ xi ≤ xu xi ∈Rd i=1,…,m{ xi } 1≤i<k≤m

Additional restrictions, alternative feasible sets, and other quality criteria can also be considered

Permutations avoided by lexicographic point arrangements In general, difficult non-convex models arise


112

LGO IDE: model visualization (m=13, d=2)

MaxiMin Point Arrangement Problem


113

Example: 40 circles; optimized radius of circles r~0.0787391…Solution time using MOP: less than 5 minutes (3 GHz PC)No postulated structural info is exploited: MOP used “blindly”

Packing Uniform Size Circles in the Unit Square


114

In such problems, we study the packing of different size circles in an embedding circle. Since this model formulation typically has infinitely many solutions per se, we will additionally try to bring the circles as close together as possible. The primary objective (obj1) is to find the circumscribed circle with the smallest radius; the secondary objective (obj2) brings the circles close together by minimizing the average distance among all circle centers.A scaled linear combination of these two objectives is used. Note that alternative formulations are also possible, and that rotational symmetries of solutions can also be avoided (by added constraints), thereby making the solution of a specific model formulation essentially unique. Applications: wires packed together in a cable, dashboard design…

Mathematica in Education and Research (2005), The Mathematica Journal (2006) Co-author: Frank J. Kampas

Non-Uniform Size Circle Packings in a Circle

115

Non-Uniform Size Circle Packings


116

General Circle Packings in Minimal Volume (Length) Container

Example: given 30 circles with radii below ; given height of container; find minimal container width

rlist = {1.275, 1.67, 2.05, 1.739, 1.399, 1.18, 0.564, 1.374, 1.237, 0.845, 1.484, 0.868, 0.807, 1.551, 1.274, 0.855, 1.493, 1.281, 1.491, 0.747, 1.085, 1.044, 0.955, 1.404, 1.292, 0.853, 0.76, 0.527, 0.592, 0.887}Best known radius is 17.291; MOP default option based radius 18.915 in ~ 20 secs; relative quality ~ 91% Further structure based refinements are possible and recommended

Pintér and Kampas (Mathematica in Edu. and Res., 2005), Castillo, Kampas, and Pintér (EJOR, 2008), Kampas and Pintér (WTC presentations, 2006, 2007; downloadable notebooks)


117

Sphere Packings in Optimized Sphere

Example: 15 spheres with radii ri=i-1/3 solved numerically by MOPRadius of embedding sphere: ~1.96308, 1.5 sec runtime on a 2007 PC,vs. ~10 min when using the built-in Mathematica function NMinimize

More details: Kampas and JDP, WTC talks + notebooks

Given a collection of spheres, findthe minimal size sphere that includesall of these, in a non-overlapping arrangement


118

Industrial Packings and Polygon Cutting Stock Plans

Credits: Ignacio Castillo, University of Alberta, Edmonton, Canada (now at WLU)

119

Credits: G. Fasano, MIP-based heuristics for non-standard 3D-packing problems with additional constraints; technical report and papers by GF

Packing Objects with Orientation and Other Characteristics(Constraints Such as Mass Balance): An Example


120

Potential Energy ModelsPoint arrangements on the surface of unit spherexi=(xi1,xi2,xi3) ||xi||=1x(m)={x1,…,xm} m-tuple (point configuration)djk=d(xj,xk) 1≤j<k≤m Euclidean distance

Model versions considered:max ∑1≤j<k≤m log(djk) Fekete (elliptic or log-potential)min ∑1≤j<k≤m 1/djk (djk>0) Coulomb-Feketemax ∑1≤j<k ≤m djk

a Power sum, 0<a<2max {min 1≤j<k≤m djk} Tammes (hard sphere)In all cases, the objective function is multi-extremal; Combined GO + expert knowledge based solutionapproaches Applications: math, physics, chemistry, biology,...

121

Elliptic Fekete model (m=25 points)


122

Coulomb-Fekete model (m=25 points)


123

Powersum model (m=25 points)


124

Tammes model (m=25 points)


125

Log-potential (Fekete Potential) ModelIn this example charged particles (points that are repelling each other) are confined to stay on the surface of the unit sphere

Our objective is to find their optimized configuration, using the Fekete potential model

5-particle example: as expected, the arrangement is symmetrical with respect to the configuration elements

Finding such an arrangement is not trivial for arbitrary m-point configurations: GO techniquescan be used

Mathematica model implementationBy Frank J. Kampas


126

Log-potential Model: 13 Points

Solution found by MathOptimizer expressed in spherical coordinates

127

Electrons in a Sphere

In this example charged particles that are repelling each otherare confined to stay within the unit sphere Objective: find their optimized configuration6-electron example: again (as expected), the arrangement found issymmetrical; all optimized points are on the surface of the sphereMathematica model implementationBy Frank J. Kampas J.D. Pintér, Global Optimization


128

Summary of Numerical StudiesPutative global optima collected (to use in comparisons) from the Web site of AT&T Bell Laboratories: these resultshave been derived by extensive numerical experimentsOur illustrative results (LGO 2000) successfully approximate the corresponding best known results to more than 99.99 precision for the log-potential, Coulomb, and power sum models; LGO solution time was ~ 10-15 sec (P4 1.6 GHz PC)Hard sphere model solution quality was only ~90% of best:Results significantly improved since that time; more recent published results using LGO and MathOptimizer ProModel variants and illustrative results appeared in Annals of Operations Research (2001); J. of Computational and Applied Mathematics (2001)Co-authors of JCAM paper: Walter Stortelder and Jacques de SwartSubsequent work and reports/papers/talks with Frank J. Kampas


129

Lennard-Jones Potential Energy Model

A pair of neutral atoms or molecules is subject to two distinct forces: an attractive force at long ranges (van der Waals force) and a repulsive force at short ranges (Pauli repulsion). The Lennard-Jones potential is a simple mathematical model that approximates these twoforces. The L-J potential is of the form

Credits: Wikipedia

Here ∈ is the depth of the ‘potential well’; σ is the distance at which the inter-particle potential is zero; and r is the distance between the particles. The aggregated (pairwise) interactions model of a group of particles leads to difficult global optimization problems: these models are used both in GO tests and in (physics, chemistry, biology) research

130

Lennard-Jones clusters for some challenging casesCredits: Jon Doye, University of Cambridge

Lennard-Jones Model and Optimized Configurations


131

Lennard-Jones clusters for further interesting casesCredits: Jon Doye, University of Cambridge



132

Lennard-Jones clusters for higher-dimensional casesCredits: Carlos Barron (University of Houston)

The optimal geometry of Lennard-Jones clusters: Computer Physics Comm. (1999), 148-309.Authors: Romero, D., Barron, C., and Gomez, S.

The missing atoms are denoted by red dots



133

The Morse potential energy function (model) is of the form

Morse Potential Energy ModelCredits: Wikipedia

Here r is the distance between the atoms, re is the equilibrium bond distance, De is the well depth (defined relative to the dissociated atoms), and a controls the 'width' of the potential.

Again, the minimal energy model of a group of particles leads to difficult global optimization problems that are used both in GO tests and in (physics, chemistry, biology) research


134

Globally optimized Morse clusters, n=24Credits: Locatelli and Schoen, 2003

New putative optimal configuration

Previous putative optimal configuration


135





136




137

Credits: László Füsti-Molnár and Kenneth M. MerzDepartment of Chemistry, Quantum Theory Project, University of Florida

Molecular Alignment and Docking Using ab initio Quality Scoring

Docking potential drug candidates into active sites of enzymes in receptor based drug design, or aligning molecules into abstract external fields or to other molecules in ligand based drug design,represents one of the biggest challenges in contemporary drug R&D.

An accurate and efficient molecular alignment technique is presented by the authors named below. It is based on first principle electronic structure calculations. This new scheme maximizes quantum similarity matrixes in the relative orientation of the molecules.

The authors have been using LGO to find the optimal alignment; as a result, they have found noticeably improved alignments.


138



A local optimum in the alignment of Methylacrylate to Crotonic acid

(The color of Crotonic acid is set to white)


139



The globally optimized alignment of Methylacrylate to Crotonic acid

(The color of Crotonic acid is set to brown)


140

Further Application Perspectives in Chemistry and Biology – The Real Deal…

Example: The molecular surface of crambinCredits: Molecular Surface Graphics, http://www.netsci.org/Science/Compchem/feature14.html

141

Atomic Structures of MacromoleculesCredits: Marin van Heel, Imperial College, London, UK

These (and great many other similar, biologically relevant) structures result from a natural tendency towards self-organizing

Notice the close conceptual relation among maximin point arrangements, circle packings and the various models of atomic and molecular structures

These arrangements are all aimed at finding globally minimal energy configurations of their objects, according to a context-specific criterion function

Therefore GO technology can be brought to a wide range of object configuration problems − as always, in combination with domain-specific expertise


142

Financial Modeling and Optimization

Example:Model development, solution, and visualization in Maple

Castillo-Lee-Pintér, Integrated Software Tools for the OR/MS Classroom, AlgOR (2008)

143

Portfolio Optimization with Transaction CostsObjective: minimize portfolio variance; Q cov. matrix xTQxConstraints: expected return (ER) xTr≥ER

asset allocation (of capital C) ∑ixi+ ∑it(xi)≤C

Note: other considerations will make model more complex… GO can be applied to many such (more realistic) models

Credits: Jason Schattman, Maplesoft


144

Portfolio Optimization with Transaction Costs

Castillo-Lee-Pintér, Integrated Software Tools for the OR/MS Classroom, AlgOR (2008)

The figure shows the location of the optimal budget allocation point (in green) on the boundary of the feasible region The surfaces representing the active budget constraint (blue) and the expected return constraint (grey) are also shown − recall KKT theory

(continued)


145

Supply Chain Management: A Reliability Optimization Example

Cited from Hum and Parlar (2006); numerical example in the e-bookGlobal Optimization with Maple (2006)


146

Solving a System of ODEs by the “Shooting Method”The SM consists of adjusting the initial conditions of the solution until the boundary conditions are met. Unless the initial conditions are very close to the correct value, singularities are frequently encountered.

Therefore one can use a finite difference approach and solve the resulting system of equations with MathOptimizer Professional. Then, based on the initial condition values found, one can find a more precise solution by the SM.

Note: the model shown is received from an MOP user

Further technical details in

MOP User Guide

147

Circuit Design

NLEQ SystemSolved by Excel/LGO

148

A Time-Discretized Control ModelSolved by Excel/LGO

149

A Time-Discretized Control Modelcontinued; the full formulation is displayed above

Credits: Frontline Systems


150

Industrial Design ProblemsAn illustrative application:

Design of an “optimized” parfume bottle, using the Maple GOT

Objective: minimize package volume

Constraints: Bottle volume ≥ required Width of the base ≥ required Aesthetic proportions

Example by MaplesoftJ.D. Pintér, Global Optimization


154

Collision Analysis for Moving Solid Bodies

Given a number of solid bodies, each with corresponding geometry, initial position, and analytical trajectory description: our task is to decide whether they will collide or not

Obviously, this problem-type is of interest in various practical applications: e.g. in robot motion analysis, production (job shop) floor planning, and other areas

One can approach this problem by finding the time moment when the smallest distance between all pairs of the moving bodies is minimal: this is a (generally speaking, far from trivial) global optimization problem


155

Collision analysis for moving solid bodies: An example

Details, including code implementation: The Mathematica Journal(2006) Co-author: Frank J. Kampas


156

Credits: Yisheng Guan and Hong Zhang, University of Alberta, Edmonton, Canada

Kinetic Grasp Feasibility Analysis in Robotics Design


157

Laser DesignOptimization and Engineering (2003); with G. Isenor & M. Cada

Basic Concepts

The laser is a device that produces a beam of light that is coherent. The beam is produced by a process known as stimulated emission.

The word laser is an acronym for the phrase “Light Amplification by Stimulated Emission of Radiation”.

The idea of stimulated emission was proposed by Albert Einstein in 1916. It took another four decades to build the first lasers as a scientific research tool; soon they found numerous significant applications.


158

Index-coupled distributed feedback laserJ.D. Pintér, Global Optimization


159

Various laser design issues can be analyzed using GO

Example:

min f(x) field flatness function (key qualitymeasure)

g(x) ≤ ε boundary condition (error limit)

xl ≤ x ≤ xu explicit, finite parameter bounds

x = (KL1, KL2, KL3, λ, Co) laser design parameters

Essential difficulty: f and g are complicated “black box” functions. The LGO IDE software has been used to analyze and solve this model (in several variants).

A very significant improvement (over 90% reduction) of the field flatness function has been attained.


160

Radiotherapy PlanningSignificance of the problem: world-wide interest and R&D activities devoted to cancer therapy by irradiationSpecific area of our research: intensity modulated radiation therapy planning, delivered by multi-leaf collimators, to cure individual patients Objective: determine the operations (movements) of the leafs in an MLC equipment, to optimally approximate the prescribed dose intensity distribution in 3 dimensions, thereby• providing prescribed radiation intensity to a target area or volume (the body parts affected by cancer)• avoiding unwanted radiation as much as possible (especially of organs at risk, as well as other body parts)For details, cf. Tervo et al., Annals of Operations Research, 2003


161

Dose Delivery and Effect ModelingSophisticated, computationally intensive mathematical models of dose delivery by MLC equipment have been developed in several versions by researchers at the University of Kuopio, Finland. The key novel feature of this approach is to optimize dose distribution directly via adjusting MLC parameters. These optimization models are all characterized by • tens or hundreds of variables (leaf positions and their coordinated movements, to describe MLC operations), • a large number of relatively simple constraints (feasible leaf positions), • a few significant complex “black box” constraints; complex objective function (target dose, and limits on unwanted dose in OAR and body tissue).


162

Leaf positions

Radiation intensity

A resulting radiation profile(based on all leaf positionsthat determine total exposure)

Superposition of overall irradiation effect, as a function of leaf positions and radiation intensity

Joint operation of leafs in MLC equipment (simplified scheme)


163

A Numerical Example

Illustrative model (2D phantom) used in optimized radiation dose distribution test calculations: overall irradiation area, hypothetical target area, and an organ at risk are shown


164

Dose distribution found by local optimization of nominal solutionJ.D. Pintér, Global Optimization


165

Globally optimized dose distributionJ.D. Pintér, Global Optimization


166

Modeling and Optimization of TransducersMathOptimizer User Guide, joint presentations with C.J. Purcell

• Traditional engineering design often based on experimental studies: change key parameters and then trace their effect (e.g. by physical experiments and their graphical summaries) – as a rule, expensive and time consuming…• Parametric studies are ideal tasks for computers: numerical models can (partially) replace experiments• Parametric models can be directly optimized• In our study, a combination of detailed system modeling and optimization has been applied; this has resulted in improved (in some cases “surprising” and entirely new) designs


167

Engineering Design Optimization by Trial and Error

Expensive and time-consuming...J.D. Pintér, Global Optimization


168

ModelMaker

• Mathematica package for developing advanced finite element models (FEM)

• Numeric and symbolic parameterized models can be developed

• Models and results presented in interactive Mathematica document (notebook) format

• Built-in, extensible documentation • Supports other FEM packages (such as

Mavart, Mavart3D, MavartMag, Atila,…)• Developed since 1994 by C. Purcell, DRDC


169

Example: Folded Shell Projector

FSP is a sonar projector (or in-air loudspeaker) with overall cylinder shape with corrugations on the sides


170

Experimental DesignThree FSPs with varying transformer ratio (a key design parameter): optimization needed...

Curves generated experimentally


171

R0

C0

Cb

Cg

Ld Rd

Cp

Lp

Rp

Cs Ls Rs

Lr RrV

TWT

α

TWT

β

TWT

γ

The optimization problem consists of finding circuit design parameters such that the sonar projector gives a broad efficiency vs. frequency. This model has been solved using MathOptimizer. The results have been applied to the actual design of sonar equipment, leading to improved designs.

Sonar Transducer Design: Numerical Model

This electric circuit simulates a piezoelectric sonar projector


172

Mathematica code of a numerical model (only a portion is shown here),subsequently solved by MathOptimizer

173

Example: Optimized FFR Transducer Design

• FFR is a free flooding ring projector and refers to a high power, unlimited depth sonar projector, in the shape of a ring • Used by Canada and the UK in sonar research over the last 1t years• TVR is the transmitting voltage response of a sonar projector and gives the response of the device in units of microPascals/Volt (mP/V) measured at 1 meter from the device center, and converted into decibels (by taking 20*log10 of the resulting mP/V value)• A higher TVR means more output sound per unit of input voltage, and thus it is a key design objective to provide a uniformly high TVR


174

Example: Multi-Mode Pipe Projector (MMPP)• Low frequency, depth insensitive sonar projector• Prototype MMPP demonstrated reasonable

bandwidth from 2.5 to 6 kHz, but TVR (sonar response) was too low


175

• Goal of optimization is to improve TVR and to increase bandwidth (3 to 9 kHz)• First optimization done using NMinimize (a Mathematica fct)• Second optimization done using MathOptimizer• Optimization was run on wave-guide wall thickness, end-cap thickness and wave-guide wall height

MMPP Modeling


176

MMPP Optimization Results


177

MMPP Optimization: Summary • Optimization provided solutions enhancing the

MMPP design in a very short time period• Optimized MMPP is a broadband, lightweight,

depth-insensitive design which can be employed for numerous applications such as – communications– active sonar– oil well borehole shakeras well as some others– patents submitted / used for actual designs

Details for a special case described in MathOptimizer User Guide,and in Wolfram Research Developer Conference Proceedings Co-author: C.J. Purcell


178

Credits: T. Mason, P. Zwietering, C. Emelle, et al. Shell R&D, Rijswijk, NLEURO 2006 talk, JIMO 2007 paper by Mason, Emelle, Van Berkel, Bagirov, Kampas, and Pintér


Oil Field Production Analysis and Optimization

179

Oil Field Production Analysis and Optimization: The Global Optimization Advantage

Improved gas lift (production) found by using HFTP/LGO at Shell IEP J.D. Pintér, Global Optimization


180

Case study developmentusing Maple


181

Suspension System Tuning

Model calibration problemsolved by using the Global OptimizationToolbox for Maple


182

Double Wishbone Suspension and Steering System

ObjectiveGiven a desired (target) behaviour of a double wishbonesuspension system, in terms of the displacement curves for bumps on the road, determine its so-called hard point settings

DynaFlex Pro by MotionPro, Inc. is used to model the system

The resulting inverse problem is then solved by the GOT

Design by Global Optimization


183

Designation of the Hard Points

The designer can specify or optimize the Cartesian coordinates of the hard points that define the double wishbone suspension: the label associated with each hard point is indicated in the lhs figure, see A to M The Cartesian coordinates relative to the chassis-fixed XYZ frame are shown in the rhs figure: the hard point coordinates are expressed in millimetersUsing global optimization, superior new designs have been found; the GOT is now used also be several leading automotive companies as an R&D toolCredits: Maplesoft and MotionPro, Inc., Waterloo, ON

184

Global Optimization Software Users: Summary• Universities

• Research organizations

• Advanced industries, R&D departments

• Consulting organizations

• Scientists, engineers, econometrists and financial modelers

• GO software is used worldwide (software by PCS and partners is used at several hundred organizations)


185

Authors/Editors Application Areas, with Information on Software (in works denoted by +S)

Grossmann, 1996 Chemical Engineering Design + SPardalos, Shalloway & Xue, 1996 Computational Chemistry and BiologyPintér, 1996 Environmental Modeling/Mgmt, and others + SCorliss and Kearfott, 1999 Rigorous Optimization in Industry + SFloudas et al., 1999 Handbook of Test Problems Papalambros and Wilde (2000) Engineering Design Edgar, Himmelblau & Lasdon, 2001 Chemical Engineering Design/Operations+ SGao, Ogden & Stavroulakis, 2001 Physics (Mechanics) Pardalos and Resende, 2002 Topical chapter by Floudas (Chem. Engrg)Schittkowski, 2002 Model Fitting (Calibration) + STawarmalani and Sahinidis (2002) Chemical Engineering Design/Operations+ SDiwekar (2003) Environmental Modeling/Mgmt + S

Global Optimization Applications and Perspectives: Illustrative References


186

Authors/Editors Application Areas, with Details on Software (in works denoted by +S)

Locatelli, Schoen et al. 2000+ Computational Chemistry and Biology + SStojanovic, 2003 Financial Modeling + SZabinsky, 2003 Engineering Design + SNeumaier, 2004 See topical review sections + SBartholomew-Biggs, 2005 Financial Modeling and OptimizationLiberti & Maculan, 2005 Chapters on Software Implementations + SNowak, 2005 MINLP Software Devpt & Tests + SPintér, 2006 Global Optimization with Maple + SPintér, 2006 GO: Sci & Engrg Case Studies + SPintér, 200… Applied NLO in Modeling Environments + SKampas & Pintér, 200… Modeling & Opt. Using Mathematica + S

Further information is welcome

Note: Keep an eye also on other literature not written by GO researchers ─ numerous examples discussed by professionals who need GO…

Global Optimization Applications and Perspectives: Illustrative References


190This e-book includes hands-on demos of the LGO IDE

192J.D. Pintér, Global Optimization eVITA Winter School 2009, Norway

193

Frank J. Kampas and János D. PintérELSEVIER SCIENCE (forthcoming)

Optimization with Mathematica Scientific, Engineering, and Economic Applications


194

Conclusions 1

• Global optimization is a subject of growing importance: it is relevant in many areas in the sciences, engineering, and economics

• Development and application of sophisticated, complex numerical models frequently requires the use of global scope optimization methodology

• Professionally developed and supported GO solver options are available for a range of platforms; further development in progress


195

Several Key Application Areas• Advanced engineering • Chemical and process industries• Defense, security• Econometrics and finance• Math/physics/chemistry/biology • Medical and pharmaceutical R&D

Conclusions 2


196

Some Key Challenges and Future Work• Integrate exact and heuristic methods• Handle problems with (very) costly functions• Handle problems w/o any exploitable structure• Stochastic optimization: simulation and optimization• Dynamic models: ODE/PDE solvers and optimization

Conclusions 3


197

• Customized model, algorithm, software, DSS development and related consulting services • Workshops and tutorials • Demonstration software, reports, and articles available• New test examples and practical challenges are welcome

Conclusions 4

Interest in R&D and Business Cooperation


198

Thanks for your attention!

Further information: www.pinterconsulting.comComments and questions: [email protected]


global optimization: software and applications - sintef

Documents