an introduction to the quality of computed solutions

64
1 An Introduction to the Quality of Computed Solutions Sven Hammarling NAG Ltd, Oxford [email protected]

Upload: venice

Post on 19-Jan-2016

21 views

Category:

Documents


0 download

DESCRIPTION

An Introduction to the Quality of Computed Solutions. Sven Hammarling NAG Ltd, Oxford [email protected]. Plan of Talk. Introduction Floating point numbers and IEEE arithmetic Condition, stability and error analysis with examples Implications for software. LAPACK and NAG Other approaches - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: An Introduction to the Quality of Computed Solutions

1

An Introduction to the Quality of Computed Solutions

Sven Hammarling

NAG Ltd, Oxford

[email protected]

Page 2: An Introduction to the Quality of Computed Solutions

2

Plan of Talk

• Introduction

• Floating point numbers and IEEE arithmetic

• Condition, stability and error analysis with examples

• Implications for software. – LAPACK and NAG

• Other approaches

• Summary

Page 3: An Introduction to the Quality of Computed Solutions

3

Introduction to NAG

Page 4: An Introduction to the Quality of Computed Solutions

4

NAG History• 1970 - Nottingham Algorithms Group

• 1971 - Mark 1 NAG Library

• 1973 - NAG moved to Oxford, renamed Numerical Algorithms Group

• 1976 - NAG Ltd, a non-profit company

• 1978 - NAG Inc established in USA

• 1980 - NAG Ltd financially self-sufficient

• 1990 - NAG GmbH established

• 1998 - Nihon NAG KK established

Page 5: An Introduction to the Quality of Computed Solutions

5

What does non-profit mean?

No shareholders, no owner

surplus is re-invested in the Company

Page 6: An Introduction to the Quality of Computed Solutions

6

How many people work at NAG?

85 worldwide, 65 in Oxford

12 in Downers Grove

Distributors worldwide

Page 7: An Introduction to the Quality of Computed Solutions

7

NAG Products and Services

• Product Lines– Numerical Libraries– Statistical Systems– NAGWare: Compiler and tools– Visualization and Graphics– PDE Solutions

• Consultancy• Customer Support www.nag.co.uk or www.nag.com

Page 8: An Introduction to the Quality of Computed Solutions

8

NAG Numerical Libraries

F77

F90

C

Parallel

SMP MPI

FL90Plus

Page 9: An Introduction to the Quality of Computed Solutions

9

Quality of Computed Solutions

Page 10: An Introduction to the Quality of Computed Solutions

10

Quality of Computed Solutions

The quality of computed solutions is concerned with assessing how good a computed solution is in some appropriate measure

Page 11: An Introduction to the Quality of Computed Solutions

11

Software Quality

Quality software should implement reliable algorithms and should provide measures of solution quality

Page 12: An Introduction to the Quality of Computed Solutions

12

Floating Point Numbers

Floating point numbers are a subset of the real numbers that can be conveniently represented in the finite word length of a computer, without unduly restricting the range of numbers represented

For example, the IEEE standard uses 64 bits to represent double precision numbers in the approximate range

Page 13: An Introduction to the Quality of Computed Solutions

13

Floating Point Numbers - Representation

Page 14: An Introduction to the Quality of Computed Solutions

14

Floating Point Numbers - Example

Page 15: An Introduction to the Quality of Computed Solutions

15

Floating Point Numbers - Example (cont’d)

0 1 2 3

Page 16: An Introduction to the Quality of Computed Solutions

16

IEEE Arithmetic Standard

ANSI/IEEE Standard 754-1985 is a standard for

binary arithmetic (b = 2). The standard specifies:• Floating point number formats• Results of the basic floating point operations• Rounding modes• Signed zero, infinity ( ) and not-a-number

(NaN)• Floating point exceptions and their handling• Conversion between formats

Most modern machines use IEEE arithmetic.

Page 17: An Introduction to the Quality of Computed Solutions

17

IEEE Arithmetic Formats

Format ecision Exponent Approx Range Approx precision

Single bits 8 bits

Double 53 bits 11 bits

Extended

Pr

24 10 10

10 10

64 15 10 10

38 8

308 16

4932 20

Page 18: An Introduction to the Quality of Computed Solutions

18

Why Worry about Computed Solutions?

• Tacoma bridge collapse• North Sea oil rig collapse• Vancouver stock exchange index, Jan 1982

to Nov 1983• Ariane 5 rocket, flight 501, failure, 4 June

1996• Patriot missile, 25 Feb 1991• Auckland Bridge• London millennium bridge

Page 19: An Introduction to the Quality of Computed Solutions

19

Tacoma Bridge

Page 20: An Introduction to the Quality of Computed Solutions

20

Auckland Bridge, 1975

Page 21: An Introduction to the Quality of Computed Solutions

21

Millennium Bridge, 2000

Page 22: An Introduction to the Quality of Computed Solutions

22

Web Sites

• Disasters attributable to bad numerical computing: http://www.math.psu.edu/dna/disasters/

• Numerical problems: RISKS-LIST: http://catless.ncl.ac.uk/Risks/

• London millennium bridge: http://www.arup.com/MillenniumBridge/

Page 23: An Introduction to the Quality of Computed Solutions

23

Example - Means

Mean of a set of numbers can be outside

their range. Using 3 figure arithmetic:

5.01 5.03 2 10.0 2 5.00

Does this matter? If so, can we guarantee

an answer within range?

Page 24: An Introduction to the Quality of Computed Solutions

24

Example - Sample Variance

22

1

2

2 2

1 1

11

1

1 12

1

For 10000,10001,10002 using 8 figure arithmetic

1 gives: 1.0, 2 gives: 0.0

Are either of these reasonable? (Actually (1) is the

correct ans

n

n ii

n n

n i ii i

T

s x xn

s x xn n

x

wer)

Page 25: An Introduction to the Quality of Computed Solutions

25

Excel Example:Standard Deviation

Microsoft Excel Worksheet

Page 26: An Introduction to the Quality of Computed Solutions

26

Overflow/Underflow Example: Modulus of a Complex Number

2 2

2

1 , where max , , min ,

0 if 0

r i

r i r i

x x x

ba a x x b x x

a

x a

Page 27: An Introduction to the Quality of Computed Solutions

27

Excel Example:Standard Deviation - Overflow

Microsoft Excel Worksheet

Page 28: An Introduction to the Quality of Computed Solutions

28

Stability

The stability of a method for solving a problem is concerned with the sensitivity of the method to (rounding) errors in the solution process

A method that guarantees as accurate a solution as the data warrants is said to be stable, otherwise the method is unstable

Page 29: An Introduction to the Quality of Computed Solutions

29

Condition

The condition of a problem is concerned with the sensitivity of the problem to perturbations in the data

A problem is ill-conditioned if small changes in the data cause relatively large changes in the solution. Otherwise a problem is well-conditioned

Page 30: An Introduction to the Quality of Computed Solutions

30

Condition Examplesx x x

x x x

x x x

x x x

x x x

x x x i

3 2

1 2 3

3 2

1 2 3

3 2

1 2 3

21 120 100 0

1 10

0 99 21 120 99 99 0

1 1117 9 04

101 21 120 100 01 0

1 9 90 104

,

. .

, . , .

. .

, , . .

Page 31: An Introduction to the Quality of Computed Solutions
Page 32: An Introduction to the Quality of Computed Solutions
Page 33: An Introduction to the Quality of Computed Solutions

33

Condition Examples (cont’d)

When

When

I ae be dx

a b

I

a b

I

x x

z( )

,

, .

10

10

1

0

1 101

220

Page 34: An Introduction to the Quality of Computed Solutions
Page 35: An Introduction to the Quality of Computed Solutions

35

Condition Examples (cont’d)

1 2

1 2

1 2

1 2

1 2

1 2

99 98 197

100 99 199

1

98.99 98 197

100 99 199

100, 99

x x

x x

x x

x x

x x

x x

Page 36: An Introduction to the Quality of Computed Solutions
Page 37: An Introduction to the Quality of Computed Solutions

37

Condition Number for Linear Equations

1

1

1

, ( )

( ) , so that

giving

( ) ,

where ( ) is the condition number of

with respect to matrix inversion

Ax b A E x b

A x x Ex x x A Ex

x x EA E A

x A

A A A

A

Page 38: An Introduction to the Quality of Computed Solutions

38

Condition Number Example

1

1 1

1

2 41

99 98, 199

100 99

99 98, 199

100 99

199 4 10

A A

A A

A

Page 39: An Introduction to the Quality of Computed Solutions

39

Stability Examples2

2

1 2

2 1

1.6 100.1 1.251 0

Four significant figure arithmetic on the standard formula

( 4 ) (2 )

gives

62.53, 0.03125,

but using

( )

gives

x x

x b b ac a

x x

x c ax

x

2 0.01251

1 2

Exact solution is

62.55, 0.0125x x

Page 40: An Introduction to the Quality of Computed Solutions

40

Stability Examples (Contd)1

0

1 0

(1 )

Easy to show that

0 1 ( 1)

Integrating by parts gives

1 , 1 1 0.632121

n xn

n

n n

y e x e dx

y n

y ny y e

recursive.m

Page 41: An Introduction to the Quality of Computed Solutions

41

Stability Examples (Contd)

In general

~

~ ~

~ ~

~ ~

~ !

y y

y y y

y y y

y y y

y y nn n

n

0 0

1 0 1

2 1 2

3 2 3

1

1 2 2

1 3 6

1

bg

Page 42: An Introduction to the Quality of Computed Solutions

42

Stability Examples (cont’d)

1

1

1

2 2

1

1

1

In general

1 1 1

n n

n m n m

n m n m

n m

n m n m

m n

n n

y y n

y y

y y n m

y n m

y y n m n m

y y n m n m n

brecursive.m

Page 43: An Introduction to the Quality of Computed Solutions

43

Error Analysis

Error analysis is concerned with establishing whether or not an algorithm is stable for the problem in hand

A forward error analysis is concerned with how close the computed solution is to the exact solution

A backward error analysis is concerned with how well the computed solution satisfies the problem to be solved

Page 44: An Introduction to the Quality of Computed Solutions

Example,

99 98 1 1 , ,

100 99 1 1

2.97Consider

2.99

1.97 0.01 , but

1.99 0.01

1.01Now consider ,

0.99

0.01 , but

01

0.

Ax b r b Ax

A b x

x

x x r

x

x x r

1.97

1.97

Page 45: An Introduction to the Quality of Computed Solutions

45

The Purpose of Error Analysis“The clear identification of the factors determining the stability of an algorithm soon led to the development of better algorithms. The proper understanding of inverse iteration for eigenvectors and the development of the QR algorithm by Francis are the crowning achievements of this line of research.

“For me, then, the primary purpose of the rounding error analysis was insight.”

Wilkinson, 1986Bulletin of the IMA, Vol 22, p197

Page 46: An Introduction to the Quality of Computed Solutions

46

Page 47: An Introduction to the Quality of Computed Solutions

47

Backward Error and Perturbation Analysis

If we know how perturbations in affect the

solution then we can estimate the accuracy of the

computed solution That is, an estimate of the

backward error allows us to estimate the forward error.

Ax b A E x b

A

x

x

x x

xA

E

A

, ~

,~.

~

~

b g

bg

Page 48: An Introduction to the Quality of Computed Solutions

Condition and Error Analysis

forward error

condition number backward error

Page 49: An Introduction to the Quality of Computed Solutions

49

LAPACK

• Systems of linear equations

• Linear least squares problems

• Eigenvalue and singular value problems, including generalized problems

• Matrix factorizations

• Condition and error estimates

• The BLAS as a portability layer

Linear Algebra PACKage for high-performance computers

Dense and banded linear algebra for Shared Memory

Page 50: An Introduction to the Quality of Computed Solutions
Page 51: An Introduction to the Quality of Computed Solutions

LAPACK and Error Bounds

“In Addition to providing faster routines than previously available, LAPACK provides more comprehensive and better error bounds. Our goal is to provide error bounds for most quantities computed by LAPACK.”

LAPACK Users’ Guide, Chapter 4 – Accuracy and Stability

Page 52: An Introduction to the Quality of Computed Solutions

Error Bounds in LAPACK

The Users’ Guide gives details of error bounds and code fragments to compute those bounds. In many cases the routines return the bounds directly

Page 53: An Introduction to the Quality of Computed Solutions

Error Bounds in LAPACK: Example

DGESVX is an 'expert' driver for solving

Estimate of 1/

Estimated forward error for

Componentwise

j

AX B

A

X

DGESVX( ..., RCOND, FERR, BERR, WORK,...)

RCOND -

FERR(j) -

BERR(j) - relative backward error for

(smallest relative change in any element of and

that makes an exact solution)

Reciprocal of pivot growth factor (1/ )

j

j

X

A B

X

WORK(1) -

Page 54: An Introduction to the Quality of Computed Solutions

54

ODE Software Example

NAG routine D02PZF returns global error estimates for the Runge-Kutta solvers D02PCF and D02PDF:

D02PCZ( RMSERR, ERRMAX, TERRMX,…)

RMSERR: Approximate Root mean square error

ERRMAX: Max approximate true error

TERRMX: Point at which max approximate true error occurred

(These routines are based upon RKSUITE)

Page 55: An Introduction to the Quality of Computed Solutions

What to do if Software does not Provide Error Estimates?

• Run the problem with perturbed data

• Better still, use a software tool such as PRECISE which allows you to perform a stochastic analysis

• Put pressure on developers to provide estimates

Page 57: An Introduction to the Quality of Computed Solutions

Quality of Reliable Software

Chaitin-Chatelin & Frayssé define the quality index

of a reliable algorithm at the computed solution as

( ) ( ) ,

where ( ) is the backward error and is the machine

precision.

(Lecture

x

B xJ x

B x

s on Finite Precision Computations)

Page 59: An Introduction to the Quality of Computed Solutions

59

Cancellation

4 4

4 4

1.000 1.000 10 1.000 10

1.000 10 1.000 10

0

s

Using four figure decimal arithmetic

Page 60: An Introduction to the Quality of Computed Solutions

60

Interval Analysis

• Interval arithmetic works on intervals that contain the exact solution

• Interval analysis can provide automatic error bounds for a number of problems

• See www.cs.utep.edu/interval-comp/main.html

4 4 4 4

4 4 4 4

1.000 1.000 1.000 10 1.000 10 1.000 10 1.000 10

1.000 10 1.0010 10 1.000 10 1.000 10

0 10

Page 61: An Introduction to the Quality of Computed Solutions

61

Summary• Error analysis gives insight

• In linear algebra we usually aim for a backward error analysis

• Be concerned about the quality of computed solutions

• Use quality software, complain about low quality software

• Write quality software and be proud of your results

Page 62: An Introduction to the Quality of Computed Solutions

62

Quote

“You have been solving these damn problems better than I can pose them.”

Sir Edward Bullard, Director NPL, in a remark to Wilkinson. (mid 1950s.)

See NAG Newsletter 2/83, page 11

Page 63: An Introduction to the Quality of Computed Solutions

References[1] E. Anderson, Z. Bai, C. H. Bischof, J. Demmel, J. J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, S. Blackford & D. C. Sorensen. LAPACK Users’ Guide, 3rd Edition, SIAM, 1999.

[2] F. Chaitin-Chatelin & V. Frayssé. Lectures on Finite Precision Computations, SIAM, 1996.

[3] G. H. Golub & C. F. Van Loan. Matrix Computations, 3rd Edition, The Johns Hopkins University Press, 1996.

[4] N. J. Higham. Accuracy and Stability of Numerical Algorithms, 2nd edition, SIAM, 2002.

[5] www.cs.utep.edu/interval-comp/main.html

[6] www.nag.co.uk/numeric/numerical_libraries.asp

Page 64: An Introduction to the Quality of Computed Solutions

References (cont’d)[7] F. S. Acton. Numerical Methods that Usually Work. Harper and Row, 1970.

[8] G. E. Forsythe. What is a Satisfactory Quadratic Equation Solver. In Constructive Aspects of the Fundamental Theorem of Algebra, B. Dejon and P. Henrici, P., editors, pages 53-61, Wiley, 1969.

[9] G. E. Forsythe. Pitfalls in Computation, or Why a Math Book isn't enough. Amer. Math. Monthly, 9: 931-995, 1970.

[10] J. H. Wilkinson. The Perfidious Polynomial. In Studies in Numerical Analysis, G. H. Golub, editor, volume 24 of Studies in Mathematics, Mathematical Association of America, Washington D.C., pages 1-24, 1984.