ihdels presentation
TRANSCRIPT
Iterative Hybridization of DE with LSfor the CEC'2015 Special Sessionon Large Scale Global Optimization
Daniel Molina1 Francisco Herrera2
1University of Cádiz 2University of Granada
27 May 2015, CEC'2015
http://sci2s.ugr.es
Outline
1 Developing ideas for the proposal
2 IHDELS
3 Results and comparisons
4 Conclusions
Outline
1 Developing ideas for the proposal
2 IHDELS
3 Results and comparisons
4 Conclusions
Problem to Optimize
Optimization problems
Global Optima f (x∗) ≤ f (x) ∀x ∈ Domain
Real-parameter Optimization Domain ⊆ <D ,x∗ = [x1, x2, · · · , xD ]
Large Scale Global Optimization How?
Large Scale
High dimensionality.
Global optimisation
Interested in one optimum.
Problem to Optimize
Optimization problems
Global Optima f (x∗) ≤ f (x) ∀x ∈ Domain
Real-parameter Optimization Domain ⊆ <D ,x∗ = [x1, x2, · · · , xD ]
Large Scale Global Optimization How?
Large Scale
High dimensionality.
Global optimisation
Interested in one optimum.
How is Large Scale Global Optimization?
Di�cult problem
Why?
High dimension ⇒ wide domain search.
How are MHs for high-dimensional problems?
MOS
DE-CC-G.
. . .
Idea: To design an MA
An EA for diversity.
An LS for exploitation.
How is Large Scale Global Optimization?
Di�cult problem
Why?
High dimension ⇒ wide domain search.
How are MHs for high-dimensional problems?
MOS
DE-CC-G.
. . .
Idea: To design an MA
An EA for diversity.
An LS for exploitation.
How is Large Scale Global Optimization?
Di�cult problem
Why?
High dimension ⇒ wide domain search.
How are MHs for high-dimensional problems?
MOS
DE-CC-G.
. . .
Idea: To design an MA
An EA for diversity.
An LS for exploitation.
How is Large Scale Global Optimization?
Di�cult problem
Why?
High dimension ⇒ wide domain search.
How are MHs for high-dimensional problems?
MOS
DE-CC-G.
. . .
Idea: To design an MA
An EA for diversity.
An LS for exploitation.
A lot of exploitation: LS
How is Large Scale Global Optimization?
Di�cult problem
Why?
High dimension ⇒ wide domain search.
How are MHs for high-dimensional problems?
MOS
DE-CC-G.
. . .
Idea: To design an MA
An EA for diversity.
An LS for exploitation.
A lot of exploitation: LS
Which EA uses?
Which EA is widely used in real optimisation?
Di�erential Evolution (DE).
1 Select randomsolutions and bestone: xr1, xr2, xg .
1 Create a di� vector:F · (xr1 − xr2).
2 Use the di� vectorto guide the search.
1 Can use also thebest point toimprove the search.
Which EA uses?
Which EA is widely used in real optimisation?
Di�erential Evolution (DE).
1 Select randomsolutions and bestone: xr1, xr2, xg .
1 Create a di� vector:F · (xr1 − xr2).
2 Use the di� vectorto guide the search.
1 Can use also thebest point toimprove the search.
Which EA uses?
Which EA is widely used in real optimisation?
Di�erential Evolution (DE).
1 Select randomsolutions and bestone: xr1, xr2, xg .
1 Create a di� vector:F · (xr1 − xr2).
2 Use the di� vectorto guide the search.
1 Can use also thebest point toimprove the search.
Which EA uses?
Which EA is widely used in real optimisation?
Di�erential Evolution (DE).
1 Select randomsolutions and bestone: xr1, xr2, xg .
1 Create a di� vector:F · (xr1 − xr2).
2 Use the di� vectorto guide the search.
1 Can use also thebest point toimprove the search.
Which EA uses?
Which EA is widely used in real optimisation?
Di�erential Evolution (DE).
1 Select randomsolutions and bestone: xr1, xr2, xg .
1 Create a di� vector:F · (xr1 − xr2).
2 Use the di� vectorto guide the search.
1 Can use also thebest point toimprove the search.
Which EA uses?
Which EA is widely used in real optimisation?
Di�erential Evolution (DE).
1 Select randomsolutions and bestone: xr1, xr2, xg .
1 Create a di� vector:F · (xr1 − xr2).
2 Use the di� vectorto guide the search.
1 Can use also thebest point toimprove the search.
Adequate for Large Scale Problems?
Advantages
Good in continuous optimization.
Simple implementation.
It change a ratio of dimension at each time (CR).
Very e�cient in the search.
Adequate for Large Scale Problems?
Advantages
Good in continuous optimization.
Simple implementation.
It change a ratio of dimension at each time (CR).
Very e�cient in the search.
Adequate for large scale problems
Problem with DE
Parameters
Crossover rate: CR.
Ratio of di�erence vector used: F.
Several di�erent mutation strategies.
Problems
Very sensitive to these parameters.
Solution: Adapt its parameters
DEs like SaDE use self-adapted parameters.
Problem with DE
Parameters
Crossover rate: CR.
Ratio of di�erence vector used: F.
Several di�erent mutation strategies.
Problems
Very sensitive to these parameters.
Solution: Adapt its parameters
DEs like SaDE use self-adapted parameters.
Problem with DE
Parameters
Crossover rate: CR.
Ratio of di�erence vector used: F.
Several di�erent mutation strategies.
Problems
Very sensitive to these parameters.
Solution: Adapt its parameters
DEs like SaDE use self-adapted parameters.
Idea: use SaDE as our EA
Choosing the Local Search
What LS we can use?
There are many LS methods for continuous optimization.
Really we have to decide?
Use a pool of LS methods.
Adapt the probability of each LS application in functionon results.
� Same idea used by SaDE for the mutation strategy.
Choosing the Local Search
What LS we can use?
There are many LS methods for continuous optimization.
Really we have to decide?
Use a pool of LS methods.
Adapt the probability of each LS application in functionon results.
� Same idea used by SaDE for the mutation strategy.
Choosing the Local Search
What LS we can use?
There are many LS methods for continuous optimization.
Really we have to decide?
Use a pool of LS methods.
Adapt the probability of each LS application in functionon results.
� Same idea used by SaDE for the mutation strategy.
Outline
1 Developing ideas for the proposal
2 IHDELS
3 Results and comparisons
4 Conclusions
IHDELS
Iterative Hybridation DE with LS: IHDELS.
Features
Run alternatively DE and a LS.
The population is maintained between iterations.
Apply the LS method to the best one.
� If it does not improve, to a random variable.
Selected LS
Several methods of LS, decide which one use by anadaptive probability.
After FreqLS evaluations, update the probability of eachLS in function on the accumulated improvement.
IHDELS
Iterative Hybridation DE with LS: IHDELS.
Features
Run alternatively DE and a LS.
The population is maintained between iterations.
Apply the LS method to the best one.
� If it does not improve, to a random variable.
Selected LS
Several methods of LS, decide which one use by anadaptive probability.
After FreqLS evaluations, update the probability of eachLS in function on the accumulated improvement.
IHDELS
Iterative Hybridation DE with LS: IHDELS.
Features
Run alternatively DE and a LS.
The population is maintained between iterations.
Apply the LS method to the best one.
� If it does not improve, to a random variable.
Selected LS
Several methods of LS, decide which one use by anadaptive probability.
After FreqLS evaluations, update the probability of eachLS in function on the accumulated improvement.
Hybridation model
Init the algorithm
1 Init population for EA.
2 Init Probability of each LS method (ProbLS).
3 Apply initially the LS method to the best solution.
Do the search1 Apply SaDE during FEDE evaluations.
2 Select the LS using ProbLS.
3 Apply the LS to one individual, during FELS evaluations.
� The best one if it is not a local optimum.� Randomly otherwise.
Adapt ProbLS
Considering the improve of previous FreqLS, update ProbLS.
Hybridation model
Init the algorithm
1 Init population for EA.
2 Init Probability of each LS method (ProbLS).
3 Apply initially the LS method to the best solution.
Do the search1 Apply SaDE during FEDE evaluations.
2 Select the LS using ProbLS.
3 Apply the LS to one individual, during FELS evaluations.
� The best one if it is not a local optimum.� Randomly otherwise.
Adapt ProbLS
Considering the improve of previous FreqLS, update ProbLS.
Hybridation model
Init the algorithm
1 Init population for EA.
2 Init Probability of each LS method (ProbLS).
3 Apply initially the LS method to the best solution.
Do the search1 Apply SaDE during FEDE evaluations.
2 Select the LS using ProbLS.
3 Apply the LS to one individual, during FELS evaluations.
� The best one if it is not a local optimum.� Randomly otherwise.
Adapt ProbLS
Considering the improve of previous FreqLS, update ProbLS.
SaDE
Initialization
A population of NP solutions are randomly selected from thedomain search.
Pool of Mutation Strategy
It applies each mutation strategy following a probability.
This probability is learned from its success rate (solutionsthat can survive to next generation).
The success rate is calculate within a certain number ofgenerations, called the Learning Period (LP).
SaDE
Mutation Strategies
DE/rand/1:vi = xr1 + F · (xr2 - xr3)DE/rand/2:vi = xr1 + F · (xr2 - xr3) + F · (xr4 - xr5)DE/rand-to-best/1:vi = xi +F · (xbest - xi) + F · (xr1-xr2) + F · (xr3 - xr4)DE/current-to-rand/1:vi = xi + k· (xr2 - xr3) F · (xr4 - xr5)
F ← Normal(0.5, 0.3)
SaDE
Crossover
ui =
{xi + vi if rand[0,1] < CR
xi otherwise
Adaptation CR parameter
CR ← Normal(CRmin, 0.1).
CRmin is self-adaptive calculated using the values fromlast successful solutions.
LS Pool
LS methods in the pool
MTS-LS1 Specially-designed for large scale problems.
L-BFGS-B Quasi-newton LS method (it estimate thegradient).
MTS-LS1
De�ne group of variables to improve C randomly sorted.
De�ne SR = 10% ratio of domain search.
Update current solution x during Istr evaluations, ∀i ∈ C .
1 xi' ← xi + SR.2 if �tness(x ′) ≤ �tness(x) x = x ′, and go Step 1.3 in other case x'i ← xi - 0.5 · SR.4 if �tness(x ′) ≤ �tness(x), and go Step 1.5 decreases SR (SR ← SR/2), if x was not improved.
LS Pool
LS methods in the pool
MTS-LS1 Specially-designed for large scale problems.
L-BFGS-B Quasi-newton LS method (it estimate thegradient).
MTS-LS1
De�ne group of variables to improve C randomly sorted.
De�ne SR = 10% ratio of domain search.
Update current solution x during Istr evaluations, ∀i ∈ C .
1 xi' ← xi + SR.2 if �tness(x ′) ≤ �tness(x) x = x ′, and go Step 1.3 in other case x'i ← xi - 0.5 · SR.4 if �tness(x ′) ≤ �tness(x), and go Step 1.5 decreases SR (SR ← SR/2), if x was not improved.
LS Pool
Why these two?
They are complementary.
MTS-LS1 explores dimension by dimension.
L-BFGS-B search in the gradient direction.
Estimating gradient?
The error increases with dimensionality.
It is not good by itself, but it is good in combination withMTS-LS1.
LS Pool
Why these two?
They are complementary.
MTS-LS1 explores dimension by dimension.
L-BFGS-B search in the gradient direction.
Estimating gradient?
The error increases with dimensionality.
It is not good by itself, but it is good in combination withMTS-LS1.
Calculating LS Probabilities
Initial values:
Each LS has the same probability
Prob[LS ] =1
|LS |∀LS in the Pool
After FreqLS evaluations
After FreqLS iterations, the probabilities for each LS(LSm) is updated following:
PLSM =ILSM∑
m∈LS ILSm
ILSM =
FreqLS∑i=1
Improvement obtained by LSM
Restart mechanism?
Restart mechanism
In optimization is good to restart when the population hasconverged.
IHDELS restart
It was designed a conservative restart mechanism.
Restart if during a complete iteration (DE+LS) the bestsolution does not improve at all.
The restart mechanism inits randomly the population.
� Except the current best solution.
Results obtained
In the experiments, only twice was applied.
Neither of them the results were improved.
Outline
1 Developing ideas for the proposal
2 IHDELS
3 Results and comparisons
4 Conclusions
Benchmark
About the benchmark
15 functions with dimension 1000D.
Di�erent group of variables:
� Fully Separable: F1-F3.� Partially Separable: F4-F11.� Overlapping functions: F12-F14.� Non-separable: F15.
Run during 3 · 106 evaluations.Measured the results with di�erent evaluation numbers:1.25 · 105, 6 · 105, 3 · 106.
IHDELS Parameters
IHDELS Parameter values
Parameter Description ValueDE popsize Population size 100FEDE Evaluations for each DE run 50000FELS Evaluations for each LS run 50000FreqLS Frequency of updating 10
the probabilities of each LSMTSSR Initial step size for MTS-LS1 10%
IHDELS results
Measure f1 f2 f3 f4 f5
Best 0.00e+00 1.27e+03 2.00e+01 1.03e+08 5.88e+06Median 4.80e-29 1.27e+03 2.00e+01 3.09e+08 9.68e+06Worst 3.94e-27 1.40e+03 2.03e+01 5.46e+08 1.32e+07Mean 4.34e-28 1.32e+03 2.01e+01 3.04e+08 9.59e+06
Measure f6 f7 f8 f9 f10
Best 1.00e+06 1.33e+04 5.36e+10 3.27e+08 9.05e+07Median 1.03e+06 3.18e+04 1.36e+12 7.12e+08 9.19e+07Worst 1.05e+06 8.03e+04 2.52e+12 8.44e+08 9.26e+07Mean 1.03e+06 3.46e+04 1.36e+12 6.74e+08 9.16e+07
Measure f11 f12 f13 f14 f15
Best 5.59e+06 2.63e-13 1.90e+06 9.41e+06 1.41e+06Median 9.87e+06 5.16e+02 4.02e+06 1.48e+07 3.13e+06Worst 2.23e+07 9.11e+02 5.15e+06 2.90e+07 4.58e+06Mean 1.07e+07 3.77e+02 3.80e+06 1.58e+07 2.81e+06
IHDELS results
Measure f1 f2 f3 f4 f5
Best 0.00e+00 1.27e+03 2.00e+01 1.03e+08 5.88e+06Median 4.80e-29 1.27e+03 2.00e+01 3.09e+08 9.68e+06Worst 3.94e-27 1.40e+03 2.03e+01 5.46e+08 1.32e+07Mean 4.34e-28 1.32e+03 2.01e+01 3.04e+08 9.59e+06
Measure f6 f7 f8 f9 f10
Best 1.00e+06 1.33e+04 5.36e+10 3.27e+08 9.05e+07Median 1.03e+06 3.18e+04 1.36e+12 7.12e+08 9.19e+07Worst 1.05e+06 8.03e+04 2.52e+12 8.44e+08 9.26e+07Mean 1.03e+06 3.46e+04 1.36e+12 6.74e+08 9.16e+07
Measure f11 f12 f13 f14 f15
Best 5.59e+06 2.63e-13 1.90e+06 9.41e+06 1.41e+06Median 9.87e+06 5.16e+02 4.02e+06 1.48e+07 3.13e+06Worst 2.23e+07 9.11e+02 5.15e+06 2.90e+07 4.58e+06Mean 1.07e+07 3.77e+02 3.80e+06 1.58e+07 2.81e+06
Comparing against DE-CC-CG for FEs=3.0e+06
Algorithm Measure f1 f2 f3 f4 f5
Best 1.75e-13 9.90e+02 2.63e-10 7.58e+09 7.28e+14Median 2.00e-13 1.03e+03 2.85e-10 2.12e+10 7.28e+14
DECC-CG Worst 2.45e-13 1.07e+03 3.16e-10 6.99e+10 7.28e+14Mean 2.03e-13 1.03e+03 2.87e-10 2.60e+10 7.28e+14
Best 0.00e+00 1.27e+03 2.00e+01 1.03e+08 5.88e+06Median 4.80e-29 1.27e+03 2.00e+01 3.09e+08 9.68e+06
IHDELS Worst 3.94e-27 1.40e+03 2.03e+01 5.46e+08 1.32e+07Mean 4.34e-28 1.32e+03 2.01e+01 3.04e+08 9.59e+06
Algorithm Measure f6 f7 f8 f9 f10
Best 6.96e-08 1.96e+08 1.43e+14 2.20e+08 9.29e+04Median 6.08e+04 4.27e+08 3.88e+14 4.17e+08 1.19e+07
DECC-CG Worst 1.10e+05 1.78e+09 7.75e+14 6.55e+08 1.73e+07Mean 4.85e+04 6.07e+08 4.26e+14 4.27e+08 1.10e+07
Best 1.00e+06 1.33e+04 5.36e+10 3.27e+08 9.05e+07Median 1.03e+06 3.18e+04 1.36e+12 7.12e+08 9.19e+07
IHDELS Worst 1.05e+06 8.03e+04 2.52e+12 8.44e+08 9.26e+07Mean 1.03e+06 3.46e+04 1.36e+12 6.74e+08 9.16e+07
Algorithm Measure f11 f12 f13 f14 f15
Best 4.68e+10 9.80e+02 2.09e+10 1.91e+11 4.63e+07Median 1.60e+11 1.03e+03 3.36e+10 6.27e+11 6.01e+07
DECC-CG Worst 7.16e+11 1.20e+03 4.64e+10 1.04e+12 7.15e+07Mean 2.46e+11 1.04e+03 3.42e+10 6.08e+11 6.05e+07
Best 5.59e+06 2.63e-13 1.90e+06 9.41e+06 1.41e+06Median 9.87e+06 5.16e+02 4.02e+06 1.48e+07 3.13e+06
IHDELS Worst 2.23e+07 9.11e+02 5.15e+06 2.90e+07 4.58e+06Mean 1.07e+07 3.77e+02 3.80e+06 1.58e+07 2.81e+06
Comparing against MOS
1.2e5 6.00E+005 3.00E+006
Function MOS IHDELS MOS IHDELS MOS IHDELS
F1 2.99E+007 1.29E+001 1.38E+000 1.81E-023 0.00E+000 4.80E-029F2 2.59E+003 1.77E+003 1.77E+003 1.27E+003 8.36E+002 1.27E+003F3 7.77E+000 2.00E+001 4.09E-011 2.00E+001 9.10E-013 2.00E+001F4 3.58E+010 1.77E+010 2.46E+009 2.24E+009 1.56E+008 3.09E+008F5 6.80E+006 1.12E+007 6.79E+006 1.02E+007 6.79E+006 9.68E+006F6 3.11E+005 1.05E+006 1.39E+005 1.04E+006 1.39E+005 1.03E+006F7 3.28E+008 2.86E+008 8.07E+006 6.90E+006 1.62E+004 3.18E+004F8 3.72E+014 1.74E+014 8.56E+013 1.70E+013 8.08E+012 1.36E+012F9 f4.32E+008 7.34E+008 3.89E+008 7.24E+008 3.87E+008 7.12E+008F10 1.24E+006 9.39E+007 1.18E+006 9.35E+007 1.18E+006 9.19E+007F11 2.78E+009 7.15E+009 7.79E+008 4.55E+008 4.48E+007 9.87E+006F12 1.02E+004 1.82E+003 2.02E+003 1.24E+003 2.46E+002 5.16E+002F13 7.34E+009 1.20E+010 7.64E+008 7.32E+008 3.30E+006 4.02E+006F14 4.46E+010 6.81E+010 1.24E+008 1.53E+008 2.42E+007 1.48E+007F15 1.43E+007 5.95E+007 6.25E+006 1.72E+007 2.38E+006 3.13E+006
Number of functions each algorithm is the best one
Evaluations IHDELS MOS
1.25e + 05 5 10
6e + 05 9 6
3e + 06 3 12
Comparing against MOS
1.2e5 6.00E+005 3.00E+006
Function MOS IHDELS MOS IHDELS MOS IHDELS
F1 2.99E+007 1.29E+001 1.38E+000 1.81E-023 0.00E+000 4.80E-029F2 2.59E+003 1.77E+003 1.77E+003 1.27E+003 8.36E+002 1.27E+003F3 7.77E+000 2.00E+001 4.09E-011 2.00E+001 9.10E-013 2.00E+001F4 3.58E+010 1.77E+010 2.46E+009 2.24E+009 1.56E+008 3.09E+008F5 6.80E+006 1.12E+007 6.79E+006 1.02E+007 6.79E+006 9.68E+006F6 3.11E+005 1.05E+006 1.39E+005 1.04E+006 1.39E+005 1.03E+006F7 3.28E+008 2.86E+008 8.07E+006 6.90E+006 1.62E+004 3.18E+004F8 3.72E+014 1.74E+014 8.56E+013 1.70E+013 8.08E+012 1.36E+012F9 f4.32E+008 7.34E+008 3.89E+008 7.24E+008 3.87E+008 7.12E+008F10 1.24E+006 9.39E+007 1.18E+006 9.35E+007 1.18E+006 9.19E+007F11 2.78E+009 7.15E+009 7.79E+008 4.55E+008 4.48E+007 9.87E+006F12 1.02E+004 1.82E+003 2.02E+003 1.24E+003 2.46E+002 5.16E+002F13 7.34E+009 1.20E+010 7.64E+008 7.32E+008 3.30E+006 4.02E+006F14 4.46E+010 6.81E+010 1.24E+008 1.53E+008 2.42E+007 1.48E+007F15 1.43E+007 5.95E+007 6.25E+006 1.72E+007 2.38E+006 3.13E+006
Number of functions each algorithm is the best one
Evaluations IHDELS MOS
1.25e + 05 5 10
6e + 05 9 6
3e + 06 3 12
Comparing against MOS
1.2e5 6.00E+005 3.00E+006
Function MOS IHDELS MOS IHDELS MOS IHDELS
F1 2.99E+007 1.29E+001 1.38E+000 1.81E-023 0.00E+000 4.80E-029F2 2.59E+003 1.77E+003 1.77E+003 1.27E+003 8.36E+002 1.27E+003F3 7.77E+000 2.00E+001 4.09E-011 2.00E+001 9.10E-013 2.00E+001F4 3.58E+010 1.77E+010 2.46E+009 2.24E+009 1.56E+008 3.09E+008F5 6.80E+006 1.12E+007 6.79E+006 1.02E+007 6.79E+006 9.68E+006F6 3.11E+005 1.05E+006 1.39E+005 1.04E+006 1.39E+005 1.03E+006F7 3.28E+008 2.86E+008 8.07E+006 6.90E+006 1.62E+004 3.18E+004F8 3.72E+014 1.74E+014 8.56E+013 1.70E+013 8.08E+012 1.36E+012F9 f4.32E+008 7.34E+008 3.89E+008 7.24E+008 3.87E+008 7.12E+008F10 1.24E+006 9.39E+007 1.18E+006 9.35E+007 1.18E+006 9.19E+007F11 2.78E+009 7.15E+009 7.79E+008 4.55E+008 4.48E+007 9.87E+006F12 1.02E+004 1.82E+003 2.02E+003 1.24E+003 2.46E+002 5.16E+002F13 7.34E+009 1.20E+010 7.64E+008 7.32E+008 3.30E+006 4.02E+006F14 4.46E+010 6.81E+010 1.24E+008 1.53E+008 2.42E+007 1.48E+007F15 1.43E+007 5.95E+007 6.25E+006 1.72E+007 2.38E+006 3.13E+006
Number of functions each algorithm is the best one
Evaluations IHDELS MOS
1.25e + 05 5 10
6e + 05 9 6
3e + 06 3 12
Outline
1 Developing ideas for the proposal
2 IHDELS
3 Results and comparisons
4 Conclusions
Conclusions
We have proposed a new algorithm for LSGO: IHDELS.
Apply iteratively DE+LS
SaDE as the DE algorithm.
It is used a LS Pool that adapt the LS method to applyduring the run.
Results
IHDELS obtains good results.
It seems that IHDELS converges prematurely.
Issues to improve
Restart mechanism to improve the search.
The LS Pool with di�erent strategies.
Study di�erent parameter values.
Conclusions
We have proposed a new algorithm for LSGO: IHDELS.
Apply iteratively DE+LS
SaDE as the DE algorithm.
It is used a LS Pool that adapt the LS method to applyduring the run.
Results
IHDELS obtains good results.
It seems that IHDELS converges prematurely.
Issues to improve
Restart mechanism to improve the search.
The LS Pool with di�erent strategies.
Study di�erent parameter values.
Conclusions
We have proposed a new algorithm for LSGO: IHDELS.
Apply iteratively DE+LS
SaDE as the DE algorithm.
It is used a LS Pool that adapt the LS method to applyduring the run.
Results
IHDELS obtains good results.
It seems that IHDELS converges prematurely.
Issues to improve
Restart mechanism to improve the search.
The LS Pool with di�erent strategies.
Study di�erent parameter values.
Questions?