grid-enabled adaptive surrogate modeling for computer ...adaptive data sampling adaptive model...
TRANSCRIPT
Department of Information Technology (INTEC)
INTEC Broadband Communication Networks research group (IBCN)
Grid-Enabled
Adaptive Surrogate Modelingfor Computer-Based Design
SUMO Lab
INTEC Broadband Communication Networks
Research Group (IBCN)
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 2
Outline
� Who are we ?
� Introduction
� Surrogate modeling
� SUMO Toolbox
� Examples
� Conclusions
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 3
Outline
� Who are we ?
� who are we
� what do we do
� Introduction
� Surrogate modeling
� SUMO Toolbox
� Examples
� Conclusions
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 4
Ghent University
Faculty of Engineering
Department of Information Technology (INTEC)
INTEC Broadband and Communications Networks (IBCN)
Who are we ?
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 5
... ... ...
Who are we ?
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN
� IBCN members� 10 professors
� ~12 postdocs
� ~85 research members
� SUMO Lab – (surrogate modeling) � PhD students
� Ivo Couckuyt
� Dirk Gorissen
� Francesco Ferranti
� Adam Narbudowicz
associated members (UA)
� Karel Crombecq
� Wouter Hendrickx
� Professor
� Tom Dhaene
� Eric Laermans
� Postdoc
� Dirk Deschrijver
associated members (UA)
� Luciano De Tommasi
Who are we ?
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 7
Research topics
� Surrogate models of complex systems
� replacement metamodels
� Surrogate based optimization
� EGO based approaches
� Machine learning and Experimental design
� High Performance Computing (HPC)
� (Parametric) Macromodels of electronic
systems
� (Parametric) Model Order Reduction
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 8
Research topics
� Surrogate models of complex systems
� replacement metamodels
� Surrogate based optimization
� EGO based approaches
� Machine learning and Experimental design
� High Performance Computing (HPC)
� (Parametric) Macromodels of electronic
systems
� (Parametric) Model Order Reduction
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 9
What do we do
Distributed
Computing
Artificial
Intelligence
Modeling &
Simulation
Distributed AI
(Multi Agent Systems, ...)
Neural Networks, SVM,
Machine Learning, ...
Parallel and Distributed
Simulation (PADS)
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 10
What do we do
Distributed
Computing
Artificial
Intelligence
Modeling &
Simulation
Adaptive Surrogate Modelingefficient and accurate characterization, modeling and simulation of
complex systems in science and engineering
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 11
Outline
� Who are we ?
� Introduction
� Surrogate model ?
� What are we looking for ?
� Existing approaches and techniques
� Surrogate modeling
� SUMO Toolbox
� Examples
� Conclusions
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 12
A New Science Paradigm
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 13
� thousand years ago : experimental science
� description of natural phenomena
A New Science Paradigm
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 14
� thousand years ago : experimental science
� description of natural phenomena
� last few hundred years : theoretical science
� Newton’s laws, Maxwell’s equations … �a.
a�
2
=4πGρ
3− Κ
c2
a2
A New Science Paradigm
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 15
� thousand years ago : experimental science
� description of natural phenomena
� last few hundred years : theoretical science
� Newton’s laws, Maxwell’s equations …
� last few decades : computational science
� simulation of complex phenomena
�a.
a�
2
=4πGρ
3− Κ
c2
a2
A New Science Paradigm
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 16
� thousand years ago : experimental science
� description of natural phenomena
� last few hundred years : theoretical science
� Newton’s laws, Maxwell’s equations …
� last few decades : computational science
� simulation of complex phenomena
� today : e-Science or data-centric science
� massive computing
� large data exploration and mining
� unify : theory, experiment, and simulation (With thanks to Jim Gray)
A New Science Paradigm
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 17
Surrogate ModelsModel Order Reduction,
Data Mining
ComputationalModeling
Real-worldData
Interpretation,Insight,Design
DistributedData & Computing
A New Science Paradigm
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 18
Surrogate ModelsModel Order Reduction,
Data Mining
ComputationalModeling
Real-worldData
Interpretation,Insight,Design
DistributedData & Computing
A New Science Paradigm
data
knowledge
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN
Problem domain
User
EngineerDesignerScientist
…
Application
ElectronicsAerodynamics
BiologyMetallurgy
…
Computer Simulation
FEM
EMsimulation
...CFD
optimization visualization design spaceexploration
prototyping ...
Costly!!SurrogateModeling
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 20
Scope
� Surrogate modeling is not the only way
� Faster hardware, different approach, different solver, simplify problem (MOR, …)
� Right tool for the right job
� Focus is on global surrogate modling
� Though Surrogate Based Optimization (SBO) is not
ignored either
� Focus is on input-output problems
� Static, not dynamic
� No time series prediction
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 21
Surrogate model ?
� examples: mechanical, electrical, optical, electronic, chemical …
systems
input output
� system modeling
� real world� I/O system
� stimulus / response
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 22
Surrogate model ?
� model = abstraction of a real system
� simulation = virtual experiment
input output
input output
� system modeling
� real world� I/O system
� stimulus / response
� simulation model� approximation
� discretization
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 23
Surrogate model ?
� system modeling
� real world
� I/O system
� stimulus / response
� simulation model
� approximation
� discretization
� surrogate model
� metamodel, RSM, emulator
� scalable analytical model
� “model of model”
input output
input output
input output
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 24
� simulation model : widely used in engineering design
� each new sample in the input design space, requires new computer simulation
� accurate, high fidelity numerical model
Surrogate model ?
input output
input2
input1
output
input2
input1
output
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 25
Surrogate model ?
� however, simulation models...
� ...complex
� ...time consuming to run
� ...optimization is expensive
� ...not always available
� ...highly specialized
� scalability?
� model chaining?
� integration with other tools?
� hardware / software requirements?
� licensing?
� ...
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 26
� surrogate model
� analytical surrogate model
� one-time upfront time investment
� harness the power of the grid for simulation execution
� adaptive sampling
� covers complete design space
� design optimization, “what-if” analysis,
� sensitivity analysis
input2
input1
output
input2
input1
output
out = f(in)
input output
Surrogate model ?
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 27
What are we looking for?
speed
accuracy
� accuracy / speed trade-off
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 28
What are we looking for?
simulator
speed
accuracy
� simulators� domain-specific
� high-accuracy
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 29
What are we looking for?
DoE / RSMmodel
speed
accuracy
� models� 2nd order polynomial
Response Surface Models (RSM)
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 30
What are we looking for?
speed
accuracy
surrogate model
� best of both worlds� combining accuracy & generality of simulators,
with the speed & flexibility of models
simulator
DoE / RSMmodel
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 31
Surrogate model ?
� advantages
� instant evaluation
� compact formulation (few 100 parameters)
� applications
� prototyping
� design space exploration
� design optimization
� sensitivity analysis
� what-if analysis
� ...
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN
Applications
input output
out = f(in)
geologymath
automotive
fluid dynamicselectronics telecom
multimedia
artchemistry
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN
��
Surrogate models
� challenges – research topics
� experimental design ?
� sample selection ?
� model selection, model tuning ?
� type (e.g. ANN, SVM, RBF, …)
� complexity, hyperparameters (e.g. #layers, #neurons of ANN)
� parameters (e.g. weights of ANN)
� black box – grey box – white box ?
model assessment & selection crucial
only as good as data & designer
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 34
What are we looking for?
� Grid-enabled adaptive algorithm for automatic surrogate model construction
� fully automated
� minimize prior, problem specific knowledge
� trade-off
� minimal number of samples
� computationally expensive
� support for distributed computing
� pre-defined accuracy
� pluggable / extensible
� no one-size-fits-all
� integrate easily into the design process
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN
� traditional approaches
� discrete model library
� database
� look-up tables, combined with local curve fitting
� hand-made analytical models
� ...
Existing approaches and techniques
Domain Expert
surrogate model
6 months
(With thanks to Luca Daniel (MIT))
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN
� common drawbacks
� oversampling / undersampling
� waste of resources / important details missed
� overmodeling / undermodeling
� accuracy unknown
� prior knowledge required
� problem specific
� “not invented here” syndrome
highly skilled modeler
several months of work
Existing approaches and techniques
��
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 37
Outline
� Who are we ?
� Introduction
� Surrogate modeling
� adaptive modeling
� adaptive sampling
� distributed computing
� adaptive surrogate modeling
� SUMO Toolbox
� Examples
� Conclusions
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 38
3 key technologies (+ 1 in development)
� adaptive data sampling
� adaptive model building
� distributed computing
� optimization
�
input2
input1
output
input2
input1
output
out = f(in)
input output
Surrogate modeling
scalable surrogate model, valid over design space
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 39
� traditional approach� uniform sampling
� oversampling
� undersampling
p3
p1
p2
p3
p1
p2
� adaptive sampling� “optimal” sample
distribution
� reflective exploration
Adaptive Sampling
active learning, sequential
design, optimal experimental design (OED)
� traditional approach
� local approximation
� overmodeling
� undermodeling
� adaptive modeling
� global approximation
� optimal model
complexity
Adaptive modeling
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 41
Surrogates – distributed computing
� traditional approach� sequential computing
� distributed computing� cluster
� grid
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 42
Surrogates – optimization
� traditional approach� classic optimization
� not well suited for computational expense simulations
� Optimization
� surrogate-assisted
optimization
� global surrogate model
� intermediate surrogate models & zoom-in
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 43
� flow chart
simulate sample
assess model
design space
build model
surrogate modeloptimal design
add
extra
sample
tune
model
complexity
adaptiveadaptive
samplingsamplingadaptiveadaptive
modelingmodeling
Surrogate modeling – flow chart
distributed distributed
computingcomputing
adaptive sampling
simulate samplesimulate sample
experimental
design (DoE)
sequential
sampling
model selection,
tuning, fitting
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 44
Outline
� Who are we ?
� Introduction
� Surrogate modeling
� SUMO Toolbox
� control flow & design
� automatic model type selection
� integrating gridcomputing
� Examples
� Conclusions
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 45
distributed distributed
computingcomputingadaptiveadaptive
modelingmodeling
adaptiveadaptive
samplingsampling
SUMO toolbox – control flow
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 46
SUMO Toolbox – Pluggability
� levels of pluggability� supports multiple model types
� Polynomial/Rational functions
� Artificial Neural Networks
� RBF models
� Support Vector Machines
� (Blind) Kriging models
� Splines
� modeling algorithm (NSGA-II, pattern search, GA, PSO, ...)
� model selection (crossvalidation, hold-out, R², AIC, ...)
� initial experimental design (factorial, LHS, custom, ...)
� sequential design (error-based, density-based, hybrid, ...)
� sample evaluation (local, distributed)
� multiple EGO criteria (GEI, EI, custom, ...)
adaptivemodeling
distributed computing
adaptivesampling
optimization
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 47
SUMO toolbox – pluggability
adaptiveadaptive
modelingmodelingadaptiveadaptive
samplingsampling
distributed distributed
computingcomputing
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 48
SUMO Toolbox – General remarks
� SUMO Toolbox
� modular design to allow 3rd party extensions
� problem specificity can be controlled
� powerful XML based configuration framework
� modeling primitives can be combined in many ways
� sensible defaults but many 'expert' options available
– user remains in control
� extensive logging and profiling framework
� intermediate models (and plots) stored for further reference
� understand behavior
� GUI Tool for easy visualization and data exploration
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 49
SUMO Toolbox – GUI
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 50
Model type selection
� however, which plugins to use?
� most important within adaptive modeling
� many surrogate model types available:
� Rational functions, RBF models, Kriging, MLP,
RBFNN, SVM, LS-SVM, regresison trees, splines, ...
� which type to use?
� problem & data dependent
� little theory available
� e.g., rational functions and EM data
� usually pragmatic
� impossible to solve in general
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 51
Model type selection
� each model is characterized by parameter set θ
� how to select θi?
� by hand?
� rule of thumb?
� optimization algorithm?
� BFGS, GA, pattern search, simulated annealing, PSO, ...
� optimization landscape is dynamic!
� cfr. adaptive sampling
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 52
Model type selection
� SUMO Toolbox makes it trivial to run and compare different methods
� however, an idea...
� Tackle the model type selection and model
parameter optimization problem in one speciatedevolutionary algorithm
� let evolution decide
� survival of the fittest
� multiple final solutions possible
� hybrid solutions possible (cfr. ensembles)
� interesting population dynamics?
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 53
Model type selection
� island model (migration model)
� most natural
� ring topology with different migration directions
� NB: inter-model speciation, not intra-model
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 54
Model type selection
� heterogeneous recombination
� Rational model x SVM = ???
� use ensembles
� phenotypic (behavioral) recombination
� avoid when possible
� many ensemble methods
� use simple average
� others can easily be used instead
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 55
Model type selection
� 3D example (z=0)
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 56
Model type selection
� Model type selection
� start with 6 model types
� after multiple “generations”, favourable model(s) automatically selected
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 57
Model type selection
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 58
Model type selection
� promising results
� computation time ≤ pure sequential
� delivers more insight
� however
� model type selection is not solved absolutely
� theoretically impossible without assumptions
� GA meta parameters expected to be more robust
� sensitivity to migration/selection parameters?
� constraints on reproducibility?
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN
Model Assesment
� Ok, we have generated a model� Why should you trust it?
� Available assessment metrics depend on� the model type
� the problem requirements
� In general if only data is available� => data or response based generalization
estimaters (more than just accuracy!)
� Some model types support more� e.g., pole-zero rational models vs SVM
� Golden standard : the domain expert
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN
Model Assesment
� The SUMO-Toolbox can support� Whatever the model type supports
� Any metric that can be expressed as a function� metric(model) → Rn
� A metric can be..� Checked manually at the end
� Enforced during the model generation process� As a constraint
� As a penalty or score
� Note the ‘n’ in Rn
� Multiple metrics can be combined� Weighted sum
� Multi-objectively
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 61
Multi-objective modeling
� stop once a good model is found
� problem: What is a good model?
� traditionally: one target accuracy/measure
� e.g., average relative crossvalidation error of 5%
� specified upfront
simplistic and impractical!
� many desirable model properties
� smoothness, accuracy, generalization, complexity, ...
� may conflict
� one metric cannot capture all requirements
� upfront specification is hard (interpretability)
� 5% Problem
�
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 62
Multi-objective modeling
� model selection = inherently multi-objective
� different solutions
� scalarization, multi-level approach, multi-objective, hybrid, ...
� each has different merits
� multi-objective (MO) approach
� use standard MO algorithms for hyperparameteroptimization (NSGA-II, AMALGAM, ...)
� multi-output modeling
� enables automatic model type selection per output
� extension: dynamic number of objectives
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 63
Multi-objective modeling
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN
Integrating gridcomputing
� Simulations are expensive� Adaptive sampling
� 1-time up front investment
� Provide interface to the grid
� Goal� Transparent integration
� Avoid middleware lock-in
� Hide grid details
� Integration on 3 levels� Resource level
� Scheduling level
� Service level
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 65
Integrating gridcomputing
� resource level
� raw distribution
� un simulations in parallel
� SampleEvaluator abstraction
� cfr. flow chart
� clean object oriented interface
� translates modeler requests into middleware
specific jobs
� support multiple backends
� Sun Grid Engine
� LCG middleware
� APST
� ...
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 66
Integrating gridcomputing
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 67
Integrating gridcomputing
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 68
Integrating gridcomputing
� scheduling level
� data points have different priorities
� e.g., domain borders, optima, sparse regions, ...
� compute resources are heterogeneous
� resources are shared (dynamic!)
� integrate grid resource information and modeling
information into scheduling decisions
Application and resource aware scheduling
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 69
Integrating gridcomputing
� service level
� integration as part of a larger service oriented
architecture (SOA)
� easy access and integration into the design process
� web browser, Jini, SOAP, ...
� complicated workflows possible
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 70
Integrating gridcomputing
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 71
Outline
� Who are we ?
� Introduction
� Surrogate modeling
� SUMO Toolbox
� Examples
� Conclusions
Example - Step Discontinuity (SD)
� Step discontinuity in a rectangular waveguide
� Frequency : 7-13 GHz
� Step length [l] : 2-8 mm
� Gap height [h] : 0.5-5 mm
� Waveguide width [a] : 22.86 mm
� Waveguide height [b] :
10.16 mm
� Distributed backend:
� Remote 256 node SGE
cluster
SD : First output
� S11 scattering
parameter
� 7GHz (green)
� 10GHz (red)
� 13GHz (blue)
SD : Second output
� S12 scattering parameter� 7GHz (green)
� 10GHz (red)
� 13GHz (blue)
Simulator configuration<Simulator>
<Name>Step Discontinuity</Name>
<InputParameters>
<Parameter name="frequency" type="real" minimum="7" maximum="13"/>
<Parameter name="gapHeight" type="real"/>
<Parameter name="stepLength" type="real"/>
</InputParameters>
<OutputParameters>
<Parameter name="S11" type="complex"/>
<Parameter name="S12" type="complex"/>
</OutputParameters>
<Implementation>
<Executable platform="unix" arch="amd64">StepDiscontinuity</Executable>
<DataFiles>...</DataFiles>
</Implementation>
</Simulator>
Toolbox configuration<ToolboxConfiguration version="6.1">
<Plan>
...
<SampleSelector>gradient</SampleSelector>
<Measure type="CrossValidation" target=".0001" errorFcn=”absoluteRMS” use="on" />
...
<Run>
<Simulator>StepDiscontinuity.xml</Simulator>
<SampleEvaluator>sge</SampleEvaluator>
<Outputs>
<Output name="S11" complexHandling=”complex”>
<AdaptiveModelBuilder>poly</AdaptiveModelBuilder>
</Output>
<Output name="S12" complexHandling=”split”>
<AdaptiveModelBuilder>kriging</AdaptiveModelBuilder>
</Output>
<Output name="S11,S12" complexHandling=”modulus”>
<AdaptiveModelBuilder>anngenetic</AdaptiveModelBuilder>
</Output>
</Outputs>
…
Modeling progres - Movie
Results – Rational models
Results – RBF models
EM example : Filter
� Double-folded microstrip stub bandstopfilter (see Bandler 1994)
� Model scattering parameters
� Simulated with ADS Momentum
� Input: L1, L2, frequency
� Output: S11, S12
EM example : Filter
� Experimental setup
� Model with ANN
� topology selected automatically using a GA
� Minimize (LRM + in-sample error)
� Select samples using a combination of
� Error based sampling and gradient based sampling
– Select samples where the model is uncertain and the response is non-linear
� Important
� Momentum = a frequency domain solver
� Frequency is sampled automatically
� => only sample in L1-L2 space
EM example : Filter
EM example : Filter
� Sample distribution
EM example: 3D Iris
� iris in rectangular waveguide (From Lamecki 2005)
� Simulation of scattering parameters� Input : frequency, iris height, length, width,
� Output : S11, S12
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 85
� Ackley function
� Classic 2D test function from optimization
Mathematics
� Schwefel Function� Classic 2D test function from optimization
Synthetic example
Chemistry example
� methane – air combustion (From Ihme 2007)
� Simulation of temperature� Input : mixture fraction variable z,
reaction progress variable c
� Output : temperature
Aerodynamics example
� re-usable Langly Glide Back Booster (LGBB)
(From Gramancy 2004 / NASA)
� Simulation of lift� Input : mach number, angle of attack, slip slide
angle
� Output : lift
Aerodynamics
� Xfoil – subsonic airfoil development (Xfoil)
� Model drag on Wheel-Fairing airfoil
Geology example
� seamount (From Parker 1987)
� Elevation data from a submerged mountain� Input : latitude, longitude
� Output : depth
� motorcycle accident (From Silverman 1987)
� Simulate a motorcycle crash against a wall
� Input : time in milliseconds since impact.
� Output : the recorded head acceleration (in g)
Automotive example
� Video quality data (From Nick Vercammen, IBBT)
� How does streaming/encoding affect quality
� Input : encoding, transmission parameters
� Output : quality metric
Multimedia example
� Wireless sensor data (From Sensor Lab, IBBT)
� Model reception quality
� Input : sender/receiver coordinates
� Output : reception quality metric
Networking example
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 94
Art
� David data� (From the Digital Michalangelo project,
Stanford University)
� Soil and Water Assesment Tool� river basin model (Grote Nete, Belgium)
� Quantify the impact of land management practices in large, complex watersheds
� Runtime for one simulation: 4 to 10 minutes
� SWAT2005: http://www.brc.tamus.edu/swat/
Hydrology : SWAT example
� Approximating global behaviour� Models error (MSE) between SWAT and observations
� Constructed model with 1016 samples
� 5.6 samples in each dimension (4 dimensions)
� 3.1 percent error (generating validation set on-the-fly)
SWAT (cont'd)
Microwave filter
� Band-pass filter (2.32-2.56 Ghz)
� Software: MicroWave Studio (MWS)
� Computer Simulation Technology (CST)
� Inputs: 5 dimensions
� Spacing: S1, S2
� Offsets: Off1, Off2, Off3
� Output
� Max(S-parameter) between 2.32 and 2.56 Ghz
� Simulation time: 5 – 10 minutes
� Optimization algorithm: EGO + Kriging
Microwave filter (cont'd)
� Initial design of 83 samples
� Results
� 'Very good' minimum after ~150 samples
� Add 100 simulations more 'to be sure'
Microwave filter (cont'd)
� Model browser
� Sensitivity
� Robustness
� ...
Microwave filter (cont'd)
� Green curve
� Reference
� Red curve
� Optimum
found
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 101
Outline
� Who are we ?
� Introduction
� Surrogate modeling
� SUMO Toolbox
� Examples
� Conclusions
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN
Conclusions
� Compact surrogate models
� Global, local
� Fully automated
� Adaptive model selection
� Adaptive sample selection
� Distributed computing
� (optimization)
� Surrogate Modeling (SUMO) Toolbox
� Easy to setup and run different modeling experiments
� Natural platform for benchmarking different techniques
� Download from http://www.sumo.intec.ugent.be
Surrogate Modeling Lab – www.sumo.intec.ugent.be
Department of Information Technology (INTEC) - IBCN 103
� Questions ?
Questions