559

Fish 559; Lecture 25

Root Finding Methods

559

What is Root Finding-I? Find the value for such that the

following system of equations is satisfied:

This general problem emerges very frequently in stock assessment and management.

We will first consider the case i=1 as it is the most common case encountered.

x

( ) 0; 1,2,....if x i n

2

559

What is Root Finding-II? Typical examples in fisheries

assessment and management include: Find K for a Schaefer model so that if the

Schaefer model is projected from K in year 0 to year m, the biomass in year m equals Z.

Find the catch limit so that the probability of recovery equals a pre-specified value.

Find F0.1 so that:

0.1 0.10 0

0.1 0.1 0F F F F F F

dY dY dY dY

dF dF dF dF

3

559

Methods for Root Finding There are several methods for finding

roots, the choice of among these depends on: The cost of evaluating the function. Whether the function is differentiable (it

must be continuous and monotonic for most methods).

Whether the derivative of the function is easily computable.

The cost of programming the algorithm.

4

559

The Example We wish to find the value of x which

satisfies the equation:

35 0xe x

-6

-4

-2

0

2

4

6

8

10

-1.5 -1 -0.5 0 0.5 1 1.5

X

Y

5

559

Derivative-free methods

6

559

The Bisection Method-I

1 2 1 2

1 2

1

2

1.Find and such that ( ) 0and ( ) 0.

2.Set ( ) / 2and compute ( ).

3. If ( ) 0, replace by .

4. If ( ) 0, replace by .

5.Repeat steps2 4 until ( ) 0.

x x f x f x

x x x f x

f x x x

f x x x

f x

7

559

The Bisection Method-II

X

Y

-1.0 -0.5 0.0 0.5 1.0

-4-2

02

46

8

Function Call #

Fu

nct

ion

va

lue

5 10 15 20

-4-2

02

46

8

8

559

The False Positive Method-I

1 2 1 2

1 2 1 1 1 2

1

2

1.Find and such that ( ) 0and ( ) 0.

2.Set ( ) ( ) /( ( ) ( )) and compute ( ).

3.If ( ) 0, replace by .

4.If ( ) 0, replace by .

5.Repeat steps 2 4until ( ) 0.

x x f x f x

x x x x f x f x f x f x

f x x x

f x x x

f x

9

559

The False Positive Method-II

X

Y

-1.0 -0.5 0.0 0.5 1.0

-4-2

02

46

8

Function Call #

Fu

nct

ion

va

lue

5 10 15 20-4

-20

24

68

X

Y

-1.0 -0.5 0.0 0.5 1.0

-4-2

02

46

8

Function Call #

Fu

nct

ion

va

lue

0 10 20 30 40 50

-40

-30

-20

-10

0

The initial vectors need not bound the solution

10

559

Brent’s Method(The method of choice)

The false positive method assumes approximate linear behavior between the root estimates; Brent’s method assumes quadratic behavior, i.e.:

The number of function calls can be much less than for the bisection and false positive methods (at the cost of a more complicated computer program).

Brent’s method underlies the R function uniroot.

( ) ( ) ( ) ( ) ( ) ( )

[ ( ) ( )][ ( ) ( )] [ ( ) ( )][ ( ) ( )] [ ( ) ( )][ ( ) ( )]

f a f b c f a f c b f b f c ax

f c f a f c f b f b f a f b f c f a f b f a f c

11

559

Brent’s Method

X

Y

-1.0 -0.5 0.0 0.5 1.0

-4-2

02

46

8

Function Call #

Fu

nct

ion

va

lue

5 10 15

-4-2

02

46

8

12

559

Derivative-based methods

13

559

Newton’s Method-I(Single-dimension case)

Consider the Taylor series expansion of the function f:

Now for “small” values of and for “well-behaved” functions we can ignore the 2nd and higher order terms. We wish to find so:

Newton’s method involves the iterative use of the above equation.

2

2( ) ( ) '( ) "( ) ...f x f x f x f x

( ) 0f x

1

( )

'( )n

n nn

f xx x

f x

14

559

Newton’s Method-II

X

Y

-1.0 -0.5 0.0 0.5 1.0

-4-2

02

46

8

Function Call #F

un

ctio

n v

alu

e

1 2 3 4 5 6 7

02

46

8

Note that Newton’s method may diverge rather than converge. This makes it of questionable value forgeneral application.

15

559

Ujevic et al’s method

1

1

1. Set , , and 0

2. Set 1

3. Set ( ) / '( )

3. If | ( ) | then '( )( ) /( ( ) ( ))

4. If | ( ) | then 1

5. Set ( ) ( ) /[ ( ) ( )]

6.Repeat steps 2 5until ( ) 0.

k k k k

k k k k k k k

k k

k k k k k k k k

x k

k k

z x f x f x

f x f x z x f z f x

f x

x x z x f x f x f z

f x

16

559

Multi-dimensional problems-I

g=0

g=0

g=0f=0

f=0

( , ) 0

( , ) 0

f x y

g x y

There is no general solution to this type

of problem

17

559

Multi-dimensional problems-II

There are two “solutions” to the problem: find the vector so that the following system of equations is satisfied:

Use a multiple-dimension version of the Newton-Raphson method;

Treat the problem as a non-linear minimization problem.

( ) 0; 1,2,3...if x i N

x

18

559

Multi-dimensional problems-III

(the multi-dimensional Newton-Raphson method)

The Taylor series expansion about is:

This can be written as a series of linear equations:

x

( ) ( ) 0ii i j

j j

ff x x f x x

x

1 1 1

1 2

2 2 2

1 2

1 2

1 1

2 2

( )

( )

( )

N

N

N N N

N

f f fx x x

f f fx x x

f f fN N

x x x

x f x

x f x

x f x

19

559

Multi-dimensional problems-IV

(the multi-dimensional Newton-Raphson method)

Given a current vector , it can be updated according to the equation:

1 1 1

1 2

2 2 2

1 2

1 2

1

11 1

22 1

( )

( )

( )

N

N

N N N

N

f f fnew old

x x x

f f fnew oldx x x

new old f f fNN N x x x

f xx x

f xx x

f xx x

oldx

20

559

Multi-dimensional problems-V

(use of optimization methods)

Rather than attempting to solve the system of equations using, say, Newton’s method, it is often more efficient to apply an optimization method to minimize the quantity:

2( )ii

SS f x

21