numerical methods in scientific computationjinbo/courses/numericalmethods_fall...exactly how will...
TRANSCRIPT
Numerical Methods in Scientific Computation
Lecture 2 Programming and SoftwareIntroduction to error analysis
Numerical Methods, Fall 2011 Lecture 2 1 Prof. Jinbo Bi
CSE, UConn
Packages vs. Programming• Packages
• MATLAB• Excel• Mathematica• Maple
• Packages do the work for you• Most offer an interactive environment
Numerical Methods, Fall 2011 Lecture 2 2 Prof. Jinbo Bi
CSE, UConn
MATLAB• Software product from The MathWorks,
Inc.• Originally focused on matrix manipulations• Interactive tool to do numerical functions,
and visualization• Commands can be saved into user scripts
call and m-file• Graphics and graphical user interfaces
(GUI) are built into programNumerical Methods, Fall 2011 Lecture 2 3 Prof. Jinbo Bi
CSE, UConn
MATLAB• Base product is extended with a variety of
“toolboxes”• Optimization• Statistics• Curve fitting• Image processing
Numerical Methods, Fall 2011 Lecture 2 4 Prof. Jinbo Bi
CSE, UConn
MATLAB• Example: Find roots of>> p = [1 -6 -72 -27]>> r = roots (p)>> r =
12.1229-5.7345-0.3884
27726 23 −−− xxx
Numerical Methods, Fall 2011 Lecture 2 5 Prof. Jinbo Bi
CSE, UConn
• Advantages:• Large library of functions to evaluate a wide
variety of numerical problems• Interactive tool allows immediate evaluation
of results• Disadvantages
• Expensive• Not as fast as C/C++/FORTRAN code
MATLAB
Numerical Methods, Fall 2011 Lecture 2 6 Prof. Jinbo Bi
CSE, UConn
Programming language concepts• Interpreted Languages
• MATLAB• Usually integrated with an interactive GUI• Allows quick evaluation of algorithms• Ideal for debugging and one-time analysis
and in cases where high speed is not critical
Numerical Methods, Fall 2011 Lecture 2 7 Prof. Jinbo Bi
CSE, UConn
Programming language concepts• Compiled Languages
• FORTRAN, C, C++• Source code is created with an editor as a plain text
file• Programs then needs to be “compiled” (converted
from source code to machine instructions with relative addresses and undefined external routines still needed).
• The compiled routines (called object modules) need then to be linked with system libraries.
• Faster execution than interpreted languages
Numerical Methods, Fall 2011 Lecture 2 8 Prof. Jinbo Bi
CSE, UConn
• Data representation• Constants, variables, type declarations
• Data organization and structures• Arrays, lists, trees, etc.
Programming concepts
Numerical Methods, Fall 2011 Lecture 2 9 Prof. Jinbo Bi
CSE, UConn
• Mathematical formulas• Assignment, priority rules
• Input/Output• Stdin / Stdout, files, GUI
Programming concepts
Numerical Methods, Fall 2011 Lecture 2 10 Prof. Jinbo Bi
CSE, UConn
• Structured programming• Set of rules that prescribe good
programming style• Single entry point and exit point to all control
structures• Flexible enough to allow creativity and
personal expression• Easy to debut, test and update
Programming concepts
Numerical Methods, Fall 2011 Lecture 2 11 Prof. Jinbo Bi
CSE, UConn
• Control structures• Sequence• Selection• Repetition
• Computer code is clearer and easier to follow
Structured programming
Numerical Methods, Fall 2011 Lecture 2 12 Prof. Jinbo Bi
CSE, UConn
• Flowcharts• Visual representation• Useful in planning
• Pseudocode• Simple representation of an algorithm• Basic program constructs without being
language-specific• Easy to modify and share with others
Algorithm expression
Numerical Methods, Fall 2011 Lecture 2 13 Prof. Jinbo Bi
CSE, UConn
• Sequence• Implement one
instruction at a time
Control structures
Numerical Methods, Fall 2011 Lecture 2 15 Prof. Jinbo Bi
CSE, UConn
• Selection• Branching
• IF/THEN/ELSE• CASE/ELSE
Control structures
Numerical Methods, Fall 2011 Lecture 2 16 Prof. Jinbo Bi
CSE, UConn
• Repetition• DOEXIT loops (break loop)
• Similar to C while loop
• DOFOR loop (count-controlled)
Control structures
Numerical Methods, Fall 2011 Lecture 2 19 Prof. Jinbo Bi
CSE, UConn
Control structures• pretest• posttest• midtest
Numerical Methods, Fall 2011 Lecture 2 20 Prof. Jinbo Bi
CSE, UConn
• Break complicated tasks into manageable parts
• Independent and self-contained• Well-defined modules with a single entry
point and single exit point• Always a good idea to return a value for
status or error code
Modular programming
Numerical Methods, Fall 2011 Lecture 2 22 Prof. Jinbo Bi
CSE, UConn
• Makes logic easier to understand and digest
• Allows black-box development of modules
• Requires well-defined interfaces with no side effects
• Isolates errors• Allows for reuse of modules and
development of libraries
Modular programming
Numerical Methods, Fall 2011 Lecture 2 23 Prof. Jinbo Bi
CSE, UConn
• Specification: A clear statement of the problem, the requirements, and any special parameters of operation
• Algorithm/Design: A flow chart or pseudo code representation of how exactly how will the problem be solved.
Software engineering approach
Numerical Methods, Fall 2011 Lecture 2 24 Prof. Jinbo Bi
CSE, UConn
• Implementation: Breaking the algorithm into manageable pieces that can be coded into the language of choice, and putting all the pieces together to solve the problem.
• Verification: Checking that the implementation solves the original specification. In numerical problems, this is difficult because most of the time you don’t know what the correct answer is.
Software engineering approach
Numerical Methods, Fall 2011 Lecture 2 25 Prof. Jinbo Bi
CSE, UConn
• How do you solve the sum of integers problem?
a) Simplestsum = 1 + 2 + 3 + 4 + 5 + .... + N• Coded specifically for specific values of N.
Example case
∑=
=n
iiS
0
Numerical Methods, Fall 2011 Lecture 2 26 Prof. Jinbo Bi
CSE, UConn
b) Intermediate solution• Does not require much thought, takes
advantage of looping ability of most languages:• C/C++:
• MATLAB
Possible solutions
Numerical Methods, Fall 2011 Lecture 2 27 Prof. Jinbo Bi
CSE, UConn
c) Analytical solution• Requires some thought about the sequence
remember back to one of your math classes.
Possible solutions
2)1(* +
=nnsum
Numerical Methods, Fall 2011 Lecture 2 28 Prof. Jinbo Bi
CSE, UConn
• What can fail in the above algorithms. (To know all the possible failure modes requires knowledge of how computers work). Some examples of failures are:
• For (b): This algorithm is fairly robust. • But, when N is large execution will be slow
compared to (c)
Verification of algorithm
Numerical Methods, Fall 2011 Lecture 2 29 Prof. Jinbo Bi
CSE, UConn
Algorithm (c)• This is most common algorithm for this
type of problem, and it has many potential failure modes. For example:• (c.1) What if N is less than zero?
Still returns an answer but not what would be expected. (What happens in (b) if N is negative?).
Numerical Methods, Fall 2011 Lecture 2 30 Prof. Jinbo Bi
CSE, UConn
Algorithm (c)• (c.2) In which order will the multiplication
and division be done.For all values of N, either N or N+1 will be even
and therefore N*(N+1) will always be even but what if the division is done first? Algorithm will work half of the time.
If division is done first and N+1 is odd, the algorithm will return a result but it will not be correct. This is a bug. For verification, it means the algorithm works sometimes but not always.
Numerical Methods, Fall 2011 Lecture 2 31 Prof. Jinbo Bi
CSE, UConn
Algorithm (c)• (c.3) What if N is very large?What is the maximum value N*(N+1) can
have?There are maximum values that integers can be represented in a computer and we may overflow. What happens then? Can we code this to handle large values of N?
Numerical Methods, Fall 2011 Lecture 2 32 Prof. Jinbo Bi
CSE, UConn
Verification• By breaking the program into small
modules, each of which can be checked, the sum of the parts is likely to be correct but not always..
• Problems can be that the program only works some of the time or it may not work on all platforms.
• The critical issue to realize all possible cases that might occur.
Numerical Methods, Fall 2011 Lecture 2 33 Prof. Jinbo Bi
CSE, UConn
Introduction to error analysis• Error is the difference between the exact
solution and a numerical method solution• In most cases, you don’t know what the exact
solution is, so how do you calculate the error• Error analysis is the process of predicting
what the error will be even if you don’t know what the exact solution
• Errors can also be introduced by the fact that the numerical algorithm has been implemented on a computer
Numerical Methods, Fall 2011 Lecture 2 34 Prof. Jinbo Bi
CSE, UConn
Significant digits• Can a number be used with confidence?• How accurate is the number?• How many digits of the number do we
trust?
Numerical Methods, Fall 2011 Lecture 2 35 Prof. Jinbo Bi
CSE, UConn
Significant digits• Can the speed be estimated to
one decimal place?
Numerical Methods, Fall 2011 Lecture 2 36 Prof. Jinbo Bi
CSE, UConn
• The significant digits of a number are those that can be used with confidence• The digits that are known with certainty plus
one estimated digit• zeros are not always significant digits
– 0.0001845, 0.001845, 0.01845– 4.53x104, 4.530x104, 4.5300x104
• Exact numbers vs. measured numbers• Exact numbers have an infinite number of
significant digits• π is an exact number but usually subject to
round-off
Significant digits
Numerical Methods, Fall 2011 Lecture 2 37 Prof. Jinbo Bi
CSE, UConn
Accuracy and precision• Accuracy is how close a computed or
measured value is to the true value• Precision is how close individual computed
or measured values agree with each other• Reproducibility
• Inaccuracy/Bias vs. Imprecision/Uncertainty• Inaccuracy: systematic deviation from the truth• imprecision: magnitude of the scatter
Numerical Methods, Fall 2011 Lecture 2 39 Prof. Jinbo Bi
CSE, UConn
• The level of accuracy and precision required depend on the problem
• Error to represent both the inaccuracy and imprecision of predictions
Accuracy and precision
Numerical Methods, Fall 2011 Lecture 2 41 Prof. Jinbo Bi
CSE, UConn
• Two general types of errors• Truncation errors – due to approximations of
exact mathematical functions• Round-off errors – due to limited significant
digit representation of exact numbers
Et = true value – approximation
Error definitions
Numerical Methods, Fall 2011 Lecture 2 42 Prof. Jinbo Bi
CSE, UConn
• Et does not capture the order of magnitude of error
• 1V error probably doesn’t matter if you’re measuring line voltage, but it does matter if you’re measuring the voltage supply to a VLSI chip
• Therefore, its better to normalize the error relative to the value
Error definitions
Numerical Methods, Fall 2011 Lecture 2 43 Prof. Jinbo Bi
CSE, UConn
• Example:• Line voltage
• Chip supply voltage
Error definitions
Numerical Methods, Fall 2011 Lecture 2 44 Prof. Jinbo Bi
CSE, UConn
• What if we don’t know the true value?• Use an approximation of the true value
• How do we calculate the approximate error?
• Use an iterative approach• Approximate error = current approximation –
previous approximation• Assumes that the iteration will converge
Error definitions
a
Numerical Methods, Fall 2011 Lecture 2 45 Prof. Jinbo Bi
CSE, UConn
• For most problems, we are interested in keeping the error less than specified error tolerance
• How many iterations do you do, before you’re satisfied that the result is correct to at least n significant digits?
Error Definitions
Numerical Methods, Fall 2011 Lecture 2 46 Prof. Jinbo Bi
CSE, UConn
• Infinite series expansion of ex
• As we add terms to the expansion, the expression becomes more exact
• Using this series expansion, can we calculate e0.5 to three significant digits?
Example
Numerical Methods, Fall 2011 Lecture 2 47 Prof. Jinbo Bi
CSE, UConn
• Calculate the error tolerance
• First approximation
• Second approximation
• Error approximation
Example
Numerical Methods, Fall 2011 Lecture 2 48 Prof. Jinbo Bi
CSE, UConn
• Third approximation
• Error approximation
Example
Numerical Methods, Fall 2011 Lecture 2 49 Prof. Jinbo Bi
CSE, UConn