458 interlude (optimization and other numerical methods) fish 458, lecture 8
Post on 19-Dec-2015
225 views
TRANSCRIPT
458
Numerical Methods Most fisheries assessment problems are
mathematically too complicated to apply analytical methods. We often have to resort to numerical methods. The two most frequent questions requiring numerical solutions are:
Find the values for a set of parameters so that they satisfy a set of (non-linear) equations.
Find the values for a set of parameters to minimize some function.
Note: Numerical methods can (and do) fail – you need to know enough about them to be able to check for failure.
458
Minimizing a Function The problem:
Find the vector so that the function is minimized (note: maximizing is the same as minimizing ).
We may place bounds on the values for the elements of (e.g. some must be positive).
By definition, for a minimum:
( )f ( )f
( )f
( )0
opt
f
458
Analytic Approaches Sometimes it is possible to solve the
differential equation directly. For example:
Now:
2
2
ˆ ˆ( ) ;
( )
i i i i
i i
SSQ y y y a b x
y a b x
2
2
2 ( ) 0
2 ( ) 0 ( )
;( )
i i i i
i i i i i i i
i i i i i i
i i i
dSSQy a bx y aN b x
dadSSQ
y a bx x x y a x b xdb
N x y x y y b xb a
N x x x N
458
Linear Models – The General Case
Generalizing:
The solution to this case is:
Exercise: check it for the simple case:
2,ˆ ˆ ˆ( ) ; ;i i i i j iSSQ y y y X b y Xb
1ˆ ( )T Tb X X X y
1 1
12 2
2
1
1; ;
1 NN
y x
by x
b
xy
y X b
458
Analytical Approaches-II Use analytical approaches whenever possible.
Finding analytical solutions for some of the parameters of a complicated model can substantially speed up the process of minimizing the function.
For example: q for the Dynamic Schaefer model:
21
1
ˆ ˆ ˆ ˆ( n n ) ; ( ) / 2
2( n n[ ]) 0
n( / ) n
exp{ n( / )}
t t t t t t
t t
t t
t tN
SSQ I I I q B B qB
dSSQI qB
dq q
I B N q
q I B
458
Numerical Methods (Newton’s Method)
Single variable version-I
We wish to find the value of x such that f(x) is at a minimum.
1. Guess a value for x2. Determine whether increasing or decreasing x
will lead to a lower value for f(x) (based on the derivative).
3. Assess the slope and its change (first and second derivatives of f) to determine how far to move from the current value of x.
4. Change x based on step 3.5. Repeat steps 2-4 until no further progress is
made.
458
Numerical Methods (Newton’s Method)
Single variable version-II
Formally:
Note: Newton’s method may diverge rather than converge!
0
2
2
1. Set
( ) ( ) '( )2. Compute '( ) ; "( )
'( )3.Modify to
"( )
4.Re peat steps 2 and 3 until stops changing
x x
df x d f x df xf x f x
dx dx dxf x
x x xf x
x
458
Minimize: 2+(x-2)^4-x^2
-50
150
350
550
-4 -2 0 2 4 6 8
x
f(X
)
This is actually quite a nasty function –
differentiate it and see!
Convergence took 17 steps in this case.
Minimum: -6.175 at x = 3.165
458
Numerical Methods (Newton’s Method)
If the function is quadratic (i.e. the third derivative is zero), Newton’s method will get to the solution in one step (but then you could solve the equation by hand!)
If the function is non-quadratic, iteration will occur.
Multi-parameter extensions to Newton’s method exist but most people prefer gradient free methods (e.g. SIMPLEX) to deal with problems like this.
458
Calculating Derivatives Numerically-I
Many numerical methods (e.g. Newton’s method) require derivatives. You should generally calculate the derivatives analytically. However, sometimes, this gets very tedious. For example, for the Dynamic Schaefer model:
I think you get the picture…
1
2
21 11 1 1 1 12 2 2
1 112 2
1[ n n( )] 2 [ n n( )]
[ (1 / ) ] [(1 ) ( ) ]2 2
(1 )t
tt t t t
t t t
t t tr r rt t t t tK
B t tr rtk
dBdSSQ dI qB I qB
dr dr B dr
dB C Cd dB B B K B B
dr dr drdB dB
Bdr dr
458
Calculating Derivatives Numerically-II
The accuracy of the approximation depends on the number of terms and the size of x / y (smaller – but not too small – is better)
2
2 2
2
( ) ( ) ( ) ( )'( ) ; '( )
2
( ) ( ) 2 ( ) ( )
( )
( , ) ( , ) ( , ) ( , ) ( , )
4
f x x f x f x x f x xf x f x
x x
df x f x x f x f x x
dx x
df x y f x x y y f x x y y f x x y y f x x y y
dx dy x y
458
Calculating Derivatives Numerically-III
(back to Cape Hake)
2
2
2
2
0.03; 123556
0.00; 0.0077
dSSQ d SSQ
dr dr
dSSQ d SSQ
dK dK
458
Optimization – some problems-I
Local minima
-3000
-2000
-1000
0
1000
2000
-6 -4 -2 0 2 4 6
X
f(X
)
Global minimum
Local minimum
458
Optimization – some problems-II
Problems calculating numerical derivatives.
Integer parameters. Bounds [either on the function
itself (extinction of Cape Hake hasn’t happened - yet) or on the values for the parameters].
458
Optimization – Tricks of the Trade-I
To keep a parameter,x, constrained between a and b, transform it to a+(0.5+arctan(y)/)(b-a).
0.9
1.2
1.5
1.8
2.1
-5 -3 -1 1 3 5
y
x
x=1+(0.5+arctan(y)/)
x~[1-2]
458
Optimization – Tricks of the Trade-II
Test the code for the model by fitting to data where the answer is known!
Look at the fit graphically (have you maximized rather than minimizing the function)?
Minimize the function manually; restart the optimization algorithm from the final value; restart the optimization algorithm from different values.
When fitting n parameters that must add to 1, fit n-1 parameters and set the last to 1-sum(1:n-1).
In SOLVER, use automatic scaling and set the convergence criterion smaller.
458
Solving an Equation(the Bisection Method)
Often we have to solve the equation f(x)=0 (e.g. the Lokta equation). This can be treated as an optimization problem (i.e. minimize f(x)2).
However, it is usually more efficient to use a numerical method specifically developed for this purpose.
We illustrate the Bisection Method here but there are many others.
458
Solving an Equation(the Bisection Method)
1 2 1 2
1 2
1
2
1.Find and such that ( ) 0and ( ) 0.
2.Set ( ) / 2and compute ( ).
3. If ( ) 0, replace by .
4. If ( ) 0, replace by .
5.Repeat steps2 4 until ( ) 0.
x x f x f x
x x x f x
f x x x
f x x x
f x
458
Solving an Equation(the Bisection Method)
-100
0
100
200
300
400
500
4 9 14 19
Step Number
F(x
)
Find x such that 10+20(x-2)^3+100(x-2)^2=0
458
Note Use of Numerical Methods
(particularly optimization) is an art. The only way to get it right is to practice (probably more than anything else in this course).