Block 4 Nonlinear Systems Lesson 14 – The Methods of Differential Calculus

The world is not only nonlinear but is changing as well

1

Narrator: Charles EbelingUniversity of Dayton

Some Most Important Topics

Slopes and Derivatives Newton-Raphson Higher Order Derivatives Taylor Series Partial Differentiation

These are excellent topics. The

exceptional student will want to stay awake

for this.

2

The Slope of a line passing through a curve

( ) ( )y f x x f xm

x x

3

The Slope of a line tangent at a point on the curve

0 0

( ) ( )lim limx x

dy y f x x f x

dx x x

4

What is the Derivative? The derivative is one of the two central

concepts of calculus. The derivative gives the slope of the line

tangent to a function at a point. In this way, derivatives can be used to determine many geometrical properties of the function such as concavity or convexity.

The derivative provides a mathematical formulation of the instantaneous rate of change; it measures the rate at which the function's value changes as the function's argument changes. 5

Why do we care?

Analyzing the behavior of functions increasing versus decreasing convex versus concave

The basis for nonlinear optimization finding local max and min

Algorithmic development finding roots of nonlinear equations

Newton-Raphson method Deriving empirical models

curve-fitting using least-squares Approximating complex functions

Taylor series expansion

… and I need you to take the

first partial derivatives, set them equal to zero, and find my maximum profit point.

6

Let’s compute our first derivative!

0 0

1 2 2 1

0

1 2 2 1 1

0

( )

( ) ( ) ( )lim lim

( 1)...

1 2lim

( 1)lim ...

1 2

n

n n

x x

n n n n n n

x

n n n n n

x

f x x

dy f x x f x x x x

dx x xn n

x nx x x x nx x x x

xn n

nx x x nx x x nx

recall the binomialtheorem?

7

The Notation

All of the following are equivalent when y = f(x):

( )'( ) ' ( )x x

d f xdyf x y D y D f x

dx dx

8

Three Rules to live by

1. 0

2. ( ) '( )

3. ( ) ( ) '( ) '( )

dc

dxdcf x cf x

dxdf x g x f x g x

dx

These are terrific rules but do you

have a problem that can use

them?

9

Let’s work a problem

Find the equation of the line tangent to y = f(x) = 2x2 + 2x + 3 at the point (1,7)

f’(x) = (2)2x1 + 2x0 + 0 = 4x + 2f’(1) = 6

Using the point-slope form of a straight line: y – 7 = 6(x – 1) or y = 6x + 1

10

The Chain Rule

I hope you don’t talk about the

chain rule. I hate the chain

rule!

dy dy du

dx du dx

22

2 2

2

2 3

2 3 ;

2 2 4 3 2 2 3 4 3

y x x

let u x x y u

dy duu u x x x x

dx dx

y=f[u(x)]

11

Alternate approach

dy dy du

dx du dx

22 4 3 2

3 2

2 3 4 12 9

16 36 18

y x x x x x

dyx x x

dx

2 3 2 2

3 2

2 2 3 4 3 2 8 6 12 9

16 36 18

x x x x x x x

x x x

12

The Engineer-Manager’s very own Table of Derivatives – the top 5

1

2

1.

2.

3.

14. ln

5.

n n

u u

d duu nu

dx dxd dv duuv u v

dx dx dxdu dvv ud u dx dx

dx v v

d duu

dx u dxd due e

dx dx

With these 5 rules and the chain rule, you can solve

91.4% of all the engineering management

differential calculus problems.

13

Table of Derivatives – the next 5

1

16. log log

7. ln( )

8. ln( )

9. sin cos

10. cos sin

a a

u u

v v v

d duu e

dx u dxd dua a a

dx dxd du dvu vu u u

dx dx dxd du

u udx dxd du

u udx dx

As an ENM student, I have had only an

occasional use of the next 5 rules but seldom

numbers 9 and 10.

14

Let’s work another problem

4

2

3

2 2

23

22 2

2 5; find

1

2 5 2 54

1 1

1 2 2 5 22 54

1 1

s dzz

s ds

dz s d s

ds s ds s

s s ss

s s

15

2

surely you recall rule3!

3.

du dvv ud u dx dx

dx v v

and some more…

2

2

/ 2

/ 2

1( ) ; find '( )

21

'( )2

x

x

f x e f x

f x e x

4

4

( ) ln(2 10) .01

2'( ) .01 4

(2 10)

x

x

f x x e

f x ex

16

The successful student will work the drill problems

Our First Application of Derivatives

Analyzing Functions

Analyze this!

17

Some Well Known FactsGiven f(x) is continuous and differentiable in the interval

(a,b), thenIf f’(x) = 0, x (a,b), then f(x) is constantIf f’(x) > 0, x (a,b), then f(x) is strictly increasingIf f’(x) < 0, x (a,b), then f(x) is strictly decreasing

+ +

-

18

Convex and Concave Functions

If f’(x) is increasing, x (a,b), then f(x) is convex in the interval (a,b)If f’(x) is decreasing, x (a,b), then f(x) is concave in the interval (a,b)

19

Example of Some Well Known Facts

f(x) = x3 – 2x + 1 and f’(x) = 3x2 – 2

2 2 2

2 2

3 2 0 2 / 3 ( ) 2 / 3

2 / 3 2 / 3

3 2 0 2 / 3 2 / 3 2 / 3

x x or x

x or x

x x or x

20

( ) increasing for 2 / 3

( ) decreasing for 2 / 3 2 / 3

( ) increasing for 2 / 3

f x x

f x x

f x x

Our Second Application of the Derivative

Finding Your Roots…

21

Newton-Raphson Method

Problem: solve for x: f(x) = 0 when f(x) is not linear or quadratic.

Assume f(x) is continuous on the interval [a,b] and f(a) and f(b) have opposite sign, then the equation f(x) = 0 has at least one real root between a and b.

22

Newton-Raphson Method -2

x2 x1

f(x1)

1

1 2

1 2 1

2 1 1

1 12 1 1

1

f(x ) 0=

f(x )

f(x )

f(x ) f(x )

f '(x )

mx x

m x x

mx mx

x x xm

23

Newton-Raphson Method - 3

12 1

1

( )

'( )

f xx x

f x

Repeat the above in order to obtain a better approximation

To find the solution to f(x) = 0

( )

'( )old

new oldold

f xx x

f x Recursion formula

24

Newton-Raphson Method - 4

Find the root of f(x) = x4 – 4x + 1 = 0 in the interval (0,1)f(0) = 1 and f(1) = -2 (a sign change, therefore at least

one root)f’(x) = 4x3 - 4

xold f(x) f'(x) xnew0 1 -4 0.25

0.25 0.00391 -3.93750 0.250990.250992 0.00000 -3.93675 0.250990.250992 0.00000 -3.93675 0.250990.250992 0.00000 -3.93675 0.250990.250992 0.00000 -3.93675 0.250990.250992 0.00000 -3.93675 0.25099

start

xold f(x) f'(x) xnew0.9 -1.9439 -1.084 -0.89327

-0.893266 5.20974 -6.85103 -0.13283-0.132833 1.53165 -4.00938 0.249180.249182 0.00713 -3.93811 0.250990.250992 0.00000 -3.93675 0.250990.250992 0.00000 -3.93675 0.250990.250992 0.00000 -3.93675 0.25099

start

( )

'( )old

new oldold

f xx x

f x

25

A Profit Example

Given the following profit function, determine the break even point :

P(x) = 96x -.01x2 –750 x.4 –100,000P’(x) = 96 - .02x –(.4)(750)x.4-1 = 96 - .02x -300x-.6

For the break-even point, find P(x) = 0:

xold P(x) P'(x) xnew200 -87444 79.5117 1299.7651299.76 -5317.5 65.9421 1380.4031380.4 -59.123 64.4735 1381.321381.32 -0.0077 64.4567 1381.321381.32 -1E-10 64.4567 1381.321381.32 0 64.4567 1381.321381.32 0 64.4567 1381.32

26

A Profit Example

P(x)

-$150,000

-$100,000

-$50,000

$0

$50,000

$100,000

$150,000

0 1000 2000 3000 4000 5000 6000 7000

x = 1382

x = 4700

27

Those problems are child’s play. Can’t you do something with a little more

substance?

Yes, how about second order

derivatives? I bet you can’t work

those?

28

Higher Order Derivatives

First derivative

y’ f’(x)

Second derivative

y’’ f’’(x)

Third derivative

y’’’ f’’’(x)

Fourth derivative

Y(4) f(4)

dy

dx2

2

d y

dx3

3

d y

dx4

4

d y

dx

Since a derivative is a function, it can have its own derivative

29

An Example

f(x) = 6x3 – 12x2 + 6x – 2

f’(x) = 18x2 – 24x + 6

f’’(x) = 36x – 24

f’’’(x) = 36

f(4) = f(5) = … = 030

Our Third Application of Derivatives

Approximating Complex Functions

31

Taylor’s Series Approximations

Taylor’s Theorem:Let a be any point on the real line and n be any non-negative integer. Then for any value of x such that f(x) and its first n+1 derivatives exists and are continuous at every point between a and x inclusive

2 3

( )1

( ) ( ) '( ) "( ) "'( )1! 2! 3!

... ( )!

n

nn

x a x a x af x f a f a f a f a

x af a R

n

32

The implication of Taylor’s theorem is that you can approximate a wide range of functions

at a point with a polynomial.

Hmmm…This could useful?

33

Example:f(x) = 100/x2

f’(x) = -200/x3

f’’(x) = 600/x4

f’’’(x)= -2400/x5

fiv(x) = 12000/x6

approximate f(x) at x = 1f(x) 100 – 200(x-1) + 300(x-1)2 – 400(x-1)3 +(12000/24) (x-1)4

Taylor Series Example x f(x) approx f(x) error

0.9 123.45679 123.35 0.106790120.91 120.758363 120.688795 0.069567520.92 118.147448 118.10432 0.043128020.93 115.620303 115.595195 0.025107930.94 113.173382 113.15992 0.013461620.95 110.803324 110.796875 0.00644910.96 108.506944 108.50432 0.002624440.97 106.28122 106.280395 0.000825110.98 104.123282 104.12312 0.000161970.99 102.030405 102.030395 1.0061E-05

1 100 100 01.01 98.0296049 98.029595 9.9407E-061.02 96.1168781 96.11672 0.000158121.03 94.2595909 94.258795 0.000795911.04 92.4556213 92.45312 0.00250131.05 90.7029478 90.696875 0.006072851.06 88.999644 88.98712 0.0125241.07 87.3438728 87.320795 0.023077831.08 85.733882 85.69472 0.039162031.09 84.1679993 84.105595 0.06240433

34

Partial Derivatives

We double our pleasure as we go from a single

variable function to a 2 variable function.

35

Partial Derivatives

0

0

Given ( , )

( , ) ( , )( , ) lim

( , ) ( , )( , ) lim

xh

yh

z f x y

z f x h y f x yf x y

x hz f x y h f x y

f x yy h

To find fx(x,y), treat y as a constant and differentiate f with respect to xin the usual way.To find fy(x,y) treat, x as a constant and differentiate f with respect to yin the usual way. 36

Example Problems2 2

2

2

( , )

( , ) 2 ; (3,4) 40

( , ) 2 ; (3,4) 33

x x

y y

f x y xy x y

f x y y xy f

f x y xy x f

2 2 3

2 2

( , , )

( , , ) 2

( , , ) 2

( , , ) 3

x

y

z

f x y z x y z z

f x y z x

f x y z yz

f x y z y z

37

More Notation

00 0

0

00 0

0

0 0 0 0( , )

0 0 0 0

( , )

( , ) ( , )

( , ) ( , )

( , ) ( , )

( , ) ( , )

x

y

xx x x yy y

yx x x yy y

zf x y f x y

x xz

f x y f x yy y

z zf x y f x y

x x x

z zf x y f x y

y y y

38

One Last Topic – Higher Order Partial Derivatives

Hurry, he is going to find all of the second partial

derivatives of a two-variable function.

ENM students hurrying to class.

39

Here it is…

2 2 2

2

2 2

2

2

( , )

( , ) 2 2

( , ) 2

( , ) 2 2

( , ) 2

( , ) ( , ) 2 4

x

y

xx

yy

xy yx

f x y x y x y

f x y xy xy

f x y x x y

f x y y y

f x y x

f x y f x y x xy

1st partials

2nd partials

40

Taylor Series revisited – 2 variables

f(x,y) = f(x0,y0) + fx(x0,y0) (x – x0) + fy(x0,y0) (y – y0)

+ ½ fxx (x0,y0) (x – x0)2 + fxy (x0,y0) (x – x0) (y – y0)

+ ½ fyy (x0,y0) (y – y0)2 + higher order terms

A second order approximation of f(x,y) in the neighborhood about the point (x0,y0):

41

Matrix-Vector Representation

00 0 0 0 0 0

0

0 0 0 0 00 0

0 0 0 0 0

( , ) ( , ) ( , ) ( , )

( , ) ( , )1( , ) ( , )2

x y

xx xy

yx yy

x xf x y f x y f x y f x y

y y

f x y f x y x xx x y y

f x y f x y y y

f(x,y) = f(x0,y0) + fx(x0,y0) (x – x0) + fy(x0,y0) (y – y0) + ½ fxx (x0,y0) (x – x0)2 + fxy (x0,y0) (x – x0) (y – y0)

+ ½ fyy (x0,y0) (y – y0)2

Hessian

gradient

42

This cries out for an example…

2 2 2

2

2 2

2

2

( , ) ; (2,3) 48

( , ) 2 2 ; (2,3) 48

( , ) 2 ; (2,3) 28

( , ) 2 2 ; (2,3) 24

( , ) 2 ; (2,3) 8

( , ) ( , ) 2 4 ; (2,3) 28

x x

y y

xx xx

yy yy

xy yx yx

f x y x y x y f

f x y xy xy f

f x y x x y f

f x y y y f

f x y x f

f x y f x y x xy f

2 2

2 24 28 21( , ) 48 48 28 2 3

3 28 8 32

148 48( 2) 28( 3) 24( 2) 56( 2)( 3) 8( 3)

2

x xf x y x y

y y

x y x x y y

43

This concludes an adventure in differentiation

Next time – optimize with the best!

The adventurous student will now

work all the problem exercises.

44