matlab programming script english
DESCRIPTION
Matlab Programming Script EnglishTRANSCRIPT
Programming
Udo Ernst
Vorlage: Jan Wiersig, WS2006
Update: Udo Ernst, WS2008
Update: Udo Ernst, WS2009
Update: Udo Ernst, SS2010
Translation: Daniel Harnack, WS2012
Institute for Theoretical Physics
University of Bremen
[email protected] 12, 2013
Contents
I. Programming in Matlab 1
1. Basic Structure of a Computer 31.1. Hardware Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2. Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3. Programming Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Representation of Numbers in the Computer 52.1. Bits and Bytes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2. Representation of Real Numbers and Numerical Errors . . . . . . . . . . . . . 7
2.2.1. Range Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2.2. Rounding Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3. Basic Commands and Variables 93.1. Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2. Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.3. The Colon-Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.4. Mathematical Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.5. Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.6. Program Control Commands and Function Definition . . . . . . . . . . . . . . 14
3.6.1. Loops (’for’-Loops) . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.6.2. Conditional Loops (’while’-Loops) . . . . . . . . . . . . . . . . . . . 163.6.3. Bifurcations/Conditional Execution (’if’-Expressions) . . . . . . . . . 163.6.4. Data Input and Output . . . . . . . . . . . . . . . . . . . . . . . . . 173.6.5. Example of a Simple Program: Orthogonality of Two Vectors . . . . . 19
3.7. Self-defined Functions and Sub-programs . . . . . . . . . . . . . . . . . . . . 193.7.1. Functions as Input Arguments . . . . . . . . . . . . . . . . . . . . . . 213.7.2. Recursive Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4. Graphics 234.1. Simple Plotting Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.2. Multiple Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254.3. Parametric Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.4. Three-dimensional and Color-coded Displays (Images) . . . . . . . . . . . . . 28
5. Systematic Programming 315.1. Planning the Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
iii
5.2. Examples of Program Development . . . . . . . . . . . . . . . . . . . . . . . 335.2.1. Finding Prime Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 335.2.2. Simulation of Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . 38
6. Vectorization and Vector Calculus in Matlab 416.1. Basic Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416.2. Definition of Matrices and Arrays . . . . . . . . . . . . . . . . . . . . . . . . 426.3. Simple, predefined Vector Operations . . . . . . . . . . . . . . . . . . . . . . 446.4. Reshaping, Copying and Duplication of Matrices/Arrays . . . . . . . . . . . . 45
6.4.1. What Size does an Array have? . . . . . . . . . . . . . . . . . . . . . 476.5. Example: Vectorization of a Program . . . . . . . . . . . . . . . . . . . . . . 48
II. Numerical Methods in Matlab 51
7. Integration and Differentiation 537.1. Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
7.1.1. Midpoint Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537.1.2. Trapezoidal Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557.1.3. Simpson’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567.1.4. Integration with Matlab . . . . . . . . . . . . . . . . . . . . . . . . . 577.1.5. Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7.2. Numerical Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587.2.1. Right-sided Differentiation . . . . . . . . . . . . . . . . . . . . . . . . 587.2.2. Centered Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . 60
8. Differential Equations 638.1. Ordinary Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . 63
8.1.1. Euler Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 648.1.2. Runge-Kutta Method . . . . . . . . . . . . . . . . . . . . . . . . . . 698.1.3. Runge-Kutta Method of 4. Order . . . . . . . . . . . . . . . . . . . . 718.1.4. Adaptive Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 728.1.5. Matlab Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
9. Data Analysis 759.1. Fourier Analysis and Filter Operations . . . . . . . . . . . . . . . . . . . . . . 75
9.1.1. Fourier Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . 759.1.2. The Convolution Theorem . . . . . . . . . . . . . . . . . . . . . . . . 80
9.2. Probabilities and Distribution Functions . . . . . . . . . . . . . . . . . . . . . 859.2.1. Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 859.2.2. Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
9.3. Regression Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 899.3.1. General Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 899.3.2. Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . 909.3.3. Goodness of the Fit . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
9.3.4. Non-linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Part I.
Programming in Matlab
1
1. Basic Structure of a Computer
1.1. Hardware Architecture
The nowadays generally applied principle, termed ’Von-Neumann-architecture’ following its
first description by John von Neumann in 1946, defines the main components of a computer,
which are illustrated in figure 1.1.
CPUprocessing unit
bus unit
control unit
input/output units memory
Figure 1.1.: Schematic illustration of the Von-Neumann-architecture.
The task of the processing unit consists, as the name suggests, in the processing of the data,especially the application of arithmetic and logic operations. Among the most important func-
tions of the control unit is the timely coordination of the sequences in the computer. Forthis, the control unit has to retrieve commands from the memory, decode them and control
their execution. In today’s computers, the processing unit and the control unit are typicallyintegrated in the same component: the CPU (Central Processing Unit).
Within the computer, data is processed in various hardware components. The transfer of the
data between the components is mediated by the bus unit.
The Memory consists of an numbered array of cells, in which every cell is capable of storinga small amount of information. This information is stored as a binary number, i.e. sequence
of ones and zeros (yes/no information), in the memory cells.
Input and output units connect the computer with the outside world. Examples for input units
are keyboard, mouse or joystick. Output units are for example screen and printer.
3
1.2. Software
The term ’Software’ encompasses both computer programs and the data used by these pro-grams. We define three types:
• System software, that is needed for the proper functioning of the computer. Among thisis especially the operating system (e.g. Unix, Linux and Windows) and also supplemen-
tary software such as hardware drivers, etc.
• Programming languages
• Application software (e.g. text editor, browser, ...)
• Games!
1.3. Programming Languages
A programming language subserves the function of transmitting commands to the computer
and providing a well defined sequence in which these commands are to be executed. At thispoint, we want to distinguish between three generations of programming languages. The first
generation consists of the machine languages. A machine language is made up of simplecommands that are represented as binary numbers. For every computer type, more precisely
for its CPU, a special code is necessary. The direct programming in a machine language isnowadays rarely done any more.
The second generation are the assembler languages. Inspite of binary numbers, coding is doneby symbols. An assembler directive is translated 1:1 to a machine command. Thus, also
assembler programs are specific to the type of processor.
The third generation of programming languages is machine independent, i.e. can be executedon every CPU. These higher programming languages utilize more complex commands and
program structures, allowing for problem-oriented working and reducing possible errors andlack of clarity to which machine oriented languages are prone. However, also programs written
in such higher programming languages have to be ultimately translated to machine language.The programs supplying the translation are called compiler respectively interpreter. Examples
for higher programming languages are Pascal, Fortran, C and C++. An interpreter, on theother hand, interprets and directly executes every command. Here, examples are Basic and
Lisp. Interpreter programs are easier to write, since debugging is more straight forward.However, the execution of these programs is slower.
The higher programming languages have developed in various directions. Especially for the
natural sciences, an interesting branch are the interactive calculation tools, like for exampleMatlab, Maple and Mathematica. These tools facilitate the computer based work on scientific
problems and questions by providing mathematical partial solutions and graphical representa-tion. It is distinguished between numerical (Matlab) and symbolic (Maple and Mathematica)
calculation tools. The latter are also termed ’computer-algebra systems’.
4
2. Representation of Numbers in theComputer
2.1. Bits and Bytes
In digital computers, information is represented by discrete states of circuit elements. The
smallest possible unit of information is one bit (binary digit). It can be in two different states,that may be described differently:
true ←→ falseyes ←→ no
1 ←→ 0...
A bit is easily realized physically:
current flowing ←→ current not flowing
voltage there ←→ voltage gonemagnetized ←→ not magnetized
...
For example, in the communication between integrated circuits, information is traded viachanges in voltage, while the bits on the hard disk are written by magnetization of ferromag-
netic materials. Between electronic musical instruments, bits are exchanged through currentloops (MIDI-Standard). In the stone age of computers, the fifties, bits were even once realized
as air bubbles in a liquid medium.
By combining several bits, complex information like numbers, letters (e.g. 8 bit-ASCII-code),
images, color, music, etc, can be represented. Much like in the decimal system, where wholenumbers are made up of the digits 0-9, the same numbers can be written in the so called dual
system by sequences of 0 and 1:
5
dual system decimal system0 01 110 211 3100 4101 5110 6111 7
......
Two further examples for 8-bit numbers:
01010101 = 1 ⋆ 20 + 0 ⋆ 21 + 1 ⋆ 22 + 0 ⋆ 23 + 1 ⋆ 24 + 0 ⋆ 25 + 1 ⋆ 26 + 0 ⋆ 27 = 85
10101010 = 0 ⋆ 20 + 1 ⋆ 21 + 0 ⋆ 22 + 1 ⋆ 23 + 0 ⋆ 24 + 1 ⋆ 25 + 0 ⋆ 26 + 1 ⋆ 27 = 170
To represent negative numbers, a small trick is necessary: one specific bit codes for the sign
of the number. In an n-bit number, one would for example reserve the n-th it for the sign,and the remaining n−1 bits code, as usual, a binary number z: If the n-th bit is not set, then
the result is by default +z. If the n-th bit is set, the result is −(2n−1) + z. For illustration,again a small table:
dual system decimal system01111111 +12701111110 +126
......
00000010 +200000001 +100000000 +011111111 -111111110 -2
......
10000010 -12610000001 -12710000000 -128
Certain bit lengths have names:
1 Byte = 8 Bit
1 Word = 16 Bit1 Kilobyte = 1024 Byte
1 Megabyte = 1024 Kilobyte1 Gigabyte = 1024 Megabyte
1 Terabyte = 1024 Gigabyte
6
2.2. Representation of Real Numbers and Numerical
Errors
Real numbers are, by their nature, analogue quantities. Hence we would expect the handling
of these numbers on digital computers not to be completely problem-free. Present digital
computers usually represent real numbers as floating-point numbers.
floating-point number = mantissa × basisexponent .
Thereby, the precision, with which the real number can be represented, is determined by the
number of available bits. ’Simple precision’ requires 4 Bytes, for ’double precision’ 8 Bytes areneeded. The latter is the default configuration in Matlab. The IEEE format of double precision
uses 53 Bits for the mantissa, 11 Bits for the exponent and for the basis the remaining 2.One Bit of the mantissa respectively the exponent are used for the sign of the quantity. Thus,
the exponent can vary between−1024 and +1023. The mantissa always represents a value inthe interval [1, 2[ in the IEEE notation. Here, the 52 Bits are utilized to add up fractions of
exponents of 2. The value of the mantissa yields mantissa=1 +∑52
i=1 bi2−i, with bi = 1 , if
the i-th bit in the mantissa is set.
2.2.1. Range Error
The maximal range of the floating-point numbers is determined by the number of bits usedto code for the exponent. A typical number for single precision is
2±127 ≈ 10±38
and for double precision2±1023 ≈ 10±308 .
Via application of arithmetic operations on these numbers, the range can be exceeded. The
error occurring in that case is named a range error. As an example we consider the Bohr radiusin SI units
a0 =4πε0h
2
mee2≈ 5.3× 10−11m .
The quantity h is Planck’s quantum of action divided by 2π. Bohr’s radius is in the range ofsingle precision floating-point numbers. However, the same does not hold for the numerator
4πε0h2 ≈ 1.24×10−78KgC2m and the denominatormee
2 ≈ 2.34×10−68KgC2. I.e. neither the
numerator nor the denominator can be represented as a single precision floating-point number.Hence, the calculation of Bohr’s radius by the given formula can be problematic. A simple
solution of this problem lies in the use of natural units, such as Bohr’s radius, for distances, etc.
An even bigger problem can be illustrated by the calculation of the factorial. The factorial isdefined as
n! = n · (n− 1) · (n− 2) · . . . 3 · 2 · 1 .
7
In Matlab, it can be easily verified by using the function factorial(n), that the factorial for
n > 170 can not be represented, even with double precision numbers. A way out is providedby the use of logarithms, since the logarithm of a bigger number still gives moderately small
values, e.g. log10(10100) = 100. It ensues that
ln(n!) = ln(n) + ln(n− 1) + . . .+ ln(3) + ln(2) + ln(1) .
For bigger n, the evaluation of this expression is, however, to laborious. If n > 30, one is
advised to use Stirling’s formula
ln(n!) = n ln(n)− n +1
2ln(2πn) + ln
(
1 +1
12n+
1
288n2+ . . .
)
.
The factorial n! can than be written as the following
n! = mantissa× 10exponent .
To get the mantissa and the exponent, we form the logarithm to the basis 10 (reminder:log10(x) = ln(x)/ ln(10))
log10(n!) = log10(mantissa) + exponent .
We now associate the integer part of log10(n!) with the exponent. The post-decimal places
are associated with the mantissa, i.e. mantissa = 10a with a = log10(n!)− exponent.
From these examples we learn that range errors can usually be circumvented with a littlecreativity.
2.2.2. Rounding Error
Rounding errors stem from the finite precision of the mantissa. The following program illus-
trates this fact:
x = 1;
while (1+x ∼=1)x = x/2;
end
x = x*2;
One might think that this constitutes an infinite loop. To the contrary, the loop will be left
in finite time. The result for double precision is x ≈ 2 × 10−16 (= Matlab variable eps).eps is the smallest number with 1 + eps > 1, and is the ’machine accuracy’. Rounding
errors of this order of magnitude occur on a regular basis. For example, Matlab calculatessin π ≈ 1.2246× 10−16. It shall be mentioned hat the machine accuracy for double precision
is exactly eps = 2−52, since 52 bits (plus one bit for the sign) are used for the mantissa. Thisrounding error might appear to be small and negligible. However, if further calculations are
performed with rounded numbers, the rounding errors can accumulate with each calculationand grow to a significant value.
8
3. Basic Commands and Variables
Matlab is an interactive system used for numerical calculations and visualization in almostevery field of technical and scientific research. Matlab excels in its ease of use, good graphics
and high efficiency.
This introduction explains the basics mandatory to perform simple numerical calculations.
First, we use Matlab as a command line interpreter. Entering of single commands will follow aprompt, here >>. An important command is help. This command returns descriptions and
syntax for functions and commands. For example
help sin
shows information regarding the sine function. Also a link to the detailed documentations can
be found, that can directly be called alternatively with the command doc. Short descriptionsof commands, that match certain search pattern, can be obtained by lookfor. Further useful
commands includequit and exit, with which the Matlab session is terminated.
3.1. Variables
Variables are introduced by assignment and do not have to be declared or defined. Names ofthe variables can be chosen freely. The only forbidden things are the use of special characters
and digits at the beginning of the name. A distinction between uppercase and lowercase lettersis not made. Examples:
a = 3; 3 is assigned to a (no output)
a result output
b = -2.5 −2, 5 is assigned to b (with output)
Note that Matlab uses a point to separate pre-decimal places and post-decimal places, followingthe conventions of the English language.
A major advantage of Matlab in comparison to a conventional programming language is thepossibility to directly work with vectors. This concept shall only be touched briefly here, since
it will be extensively treated in a separate chapter:
a = [3;1]; defines a column vector ~a =
(
31
)
a(k) means the vector component ak
b = [0,3,-4]; defines a row vector ~b = (0, 3,−4) .
9
The commas in square brackets can also be replaced by blank spaces.
Matlab cannot only calculate with vectors, but also with matrices. Thus, linear system of
equations, such as
x+ 2y = 3
3x+ y = 1 ,
can be formulated elegantly:(
1 23 1
)(
xy
)
=
(
31
)
respectively
D~x = ~b
with the 2× 2 matrix
D =
(
1 23 1
)
← 1. line← 2. line
ր տ1. line 2. line
and the column vectors
~x =
(
xy
)
and ~b =
(
31
)
.
Here, ~x is the solution vector being sought. The formal solution of the equation D~x = ~b is
given by
~x = D−1~b ,
where D−1 denotes the inverse of the matrix D.
Matrices are introduced in Matlab as the following
C = [1,1;0,1]; defines the 2× 2 matrix C =
(
1 10 1
)
D = [1,2;3,1]; defines the 2× 2 matrix D =
(
1 23 1
)
.
The single elements Ckl of a matrix C can be accessed by C(k,l). Here, k denotes the k-th
row and l the l-th column of the matrix. Interestingly, all variables that are used by Matlabare matrices, which is how the name is derived (’Matrix Laboratory’).
The following table lists useful commands to manage variables.
10
who List of all used variableswhos List of all used variables with respective size
clear Variable Delete a variableclear all Delete all variables
load ’filename’ Load variable(s) from a filesave ’filename’ variable Save variable to a file
! execute a Linux/MSDOS-command
Some specific variables are predefined:
ans Last resultpi πi Imaginary unit
√−1
Inf Infinity (z.B. 1/0)NaN ’Not-a-Number’ (e.g. 0/0)
Note that the predefined variable can also be overwritten. Sometimes this happens unintend-edly and leads to unpredictable results.
3.2. Operations
The basic arithmetic operations like addition (+), subtraction (-), multiplication (*), division(/) and powers (ˆ) are defined in a natural manner. Examples using the earlier defined variables
are
z = 10-8; assigns the value 2 to z
t = b*[1;2;3]; assigns the scalar product −6 = 0 + 6− 12 to t
F = C+D; assigns the matrix to F
(
2 33 2
)
=
(
1 10 1
)(
1 23 1
)
to
x = zˆ3; assigns the value 8 = 23 to x
3.3. The Colon-Operator
The colon is used in Matlab to generate sequences of numbers, e.g
x=1:10;
generates a row vector ~x = (1, 2, . . . , 10), which holds the numbers from 1 to 10 in increments
of 1. The increment can be set between the start and the end value:
x=0:0.1:1;
The result is a row vector ~x = (0, 0.1, . . . , 1).
The colon-operator is especially handy to access defined elements of vectors, e.g.
11
x(3:6)
gives the elements (0.2, 0.3, 0.4, 0.5). When the colon-operator stands on its own, e.g. x(:),
all elements of the vector are referenced. This becomes especially interesting in the case ofmatrices. As a case in point we consider the matrix
A =
(
1 −1 23 0 1
)
,
from which we want to determine the first and the third column.
A = [1,−1, 2; 3, 0, 1];a = A(:,1);
b = A(:,3);
The result is ~a =
(
13
)
and ~b =
(
21
)
.
3.4. Mathematical Functions
Matlab provides a wealth of mathematical and numerical functions. All functions are appliedto the whole vector respectively matrix. The following tables summarize the basic functions.
abs(x) absolute valuesqrt(x) square root
sin(x), cos(x), tan(x) sine, cosine, tangentatan2(y,x) arc tangent of y/x in the interval [0, 2π)exp(x) exponential function
log(x), log10(x) natural logarithm, logarithm to the basis 10besselj(n,x) Bessel function n-th orderrem(x,y) rest after division (z.B. rem(10.3,4) = 2.3)round(x) rounding (z.B. round(3.2) = 3)floor(x) round down (z.B. floor(3.2) = 3)ceil(x) round up (z.B. ceil(3.2) = 4)
norm(x) norm (length) of a vectorsum(x) sum over vector elementsprod(x) product over vector elements
max(x), min(x) maximum, minimum of vector elementslength(x) dimension of a vectormean(x) meanstd(x) standard deviation
12
size(A) dimension of the matrix Adet(A) determinant of the matrix Arank(A) rank of the matrix Arand(N) uniformly distributed random numbers between [0, 1). N ×N matrixrandn(N) Gaussian distributed random numbers with mean = 0 and variance = 1.
3.5. Complex Numbers
An important feature of Matlab is that it provides a variable type for complex numbers. Aslong as it is not already used in another context, like a loop, the variable i is predefined as the
complex unit. In general, complex numbers can be defined by invoking the command complex
with two arguments, that represent the real- respectively the imaginary part of the number,
e.g.:
i = complex(0, 1);
Computations with complex numbers can be carried out just like with normal, real values.Hence, the following to lines
c = complex(17, 42);
c = 17+i*42;
give the same results.
In the imaginary plane, the complex number can be represented as a vector with a length amp,that starts in the origin and points in the direction phi. These quantities can be derived from
the complex numbers through the commands abs and angle. The commands
amp = abs(c)
phi = angle(c)
give the output 45.3100 and 1.1862. Conversely, a complex number can be defined by alength and an angle in the complex plane:
c = amp*exp(i*phi);
The real and imaginary part of a complex number can then be extracted by the commandreal and imag:
real(c), imag(c)
for example returns the output 17 and 42. The command complex can also handle matricesand arrays as input. Thus
c array = complex(zeros(42, 1), rand(42, 1));
defines a a 42x1-matrix filled with complex numbers, that have a random imaginary part
between 0 and 1.
13
3.6. Program Control Commands and Function
Definition
Programming in Matlab is carried out via script files and function files. The filenames endin ’.m’, which is why these files are also named ’M-files’. You should not use blank spaces,
umlauts or mathematical operators (e.g. -) in the file names. One should also avoid namingthe files after a build-in Matlab command, since this command will then be overwritten.
Simply put, script and function files contain a list of commands. These commands will thensimply be executed in succession. More complex program structures are realized with loops and
bifurcations. Essentially, there are the following possibilities to control the flow of a program– These constructs are found in every programming language, not only in Matlab.
Figure 3.1.: Different possibilities to control the flow of a program, respectively the successionof commands contained in a program.
14
3.6.1. Loops (’for’-Loops)
Reoccurring operations are executed within loops. If it is desired to execute a specific set of
commands n-times in a row, the use of a so called for-loop is advisable. As an example, wewant to create a vector ~p = (p1, p2, p3, p4, p5) with pk = k2, where k = 1, 2, 3, 4, 5. The
symbol % induces commentaries:
for k=1:5 % begin of the loop
p(k) = kˆ2; % entries in the row vector
end % end of the loop
The result is a row vector ~p = (1, 4, 9, 16, 25). Note that frequently, loops in Matlab can bereplaced by the use of the colon-operator, here p = [1:5].ˆ2. The latter is definitely prefer-
able, since the execution is faster. In analogy to the colon-operator, an increment differentform 1 can be set, e.g.
for k=1:2:5
15
3.6.2. Conditional Loops (’while’-Loops)
The difference between the the while-loop and the for-loop, where the number of times which
the body of the loop is executed has to be known beforehand, is that the execution of thecommands in the while-loop is tied to a condition: prior to every iteration this condition is
tested. If the test turn out positive, the loop will be run through. Otherwise the loop willbe left (or not even entered in the first place), and the commands following the loop will be
executed. Example:
x=5;
while(x>1) % begin of the loop (infinite loop possible)
x=x/2;
end % end of the loop
The result is x = 5/8 = 0.625.
3.6.3. Bifurcations/Conditional Execution (’if’-Expressions)
Conditional execution of commands is also mediated by if-expressions. The result of the bi-
furcation depends on the result of the result of the if-statement. Example:
if (x>5)
z = 1;
end
Provided the relation x > 5 is fulfilled, set z = 1. The following operators are available toconstruct relations:
> greater< smaller>= greater or equal<= smaller or equal== equal∼= unequal& logical ’and’| logical ’or’∼ logical ’not’
The command if comes with the command else. Example:
if (0 < x & x < 1)
16
status = 1;
else
status = 0;
end
If 0 < x and x < 1, then set status = 1, else set status = 0. The command else thus allows
to react to a unfulfilled condition. If this shall be coupled to a further condition, the commandelseif (has to be written as one word) can be used.
3.6.4. Data Input and Output
Keyboard and Screen
For input from the keyboard, the command input is provided:
x = input(’Input x: ’);
The command disp displays text and values of variables on the screen:
disp(’The value of x is ’);
disp(x);
Formatted output is achieved by the command fprintf, e.g.
fprintf(’The value of x is %g \n’,x);
In this connection, %g is a placeholder for the variable x. The symbol \n forces the following
output to begin in a new line. Another Example is
fprintf(’x = %6.2f \n’,x);
Here, the first number after the percent symbol sets the minimal total number of digits orcharacters that shall be printed. The second number is the desired number of post-decimal
places.
Storage Media
The output to the hard disk or other storage media takes place in a similar way. Prior to
writing to or reading from a file, this file has to be opened. This is done with the command
17
fopen. Example:
>> fp = fopen(’example.dat’, ’w’);
example.dat is the name of the data being opened and w indicates that we want to writesomething into the data (=write). Upon successful execution of the command, Matlab returns
a so called file handle. This can be understood as a pointer when writing into the file, whichtells the computer where the data is supposed to go.
Similar to printing to the screen, data can now be written into the file:
>> fprintf(fp, ’x = %6.2f \n’,x);
Note that the ’file handle’ fp is passed as the first argument to the fprintf-command.
After writing of the data, the file is closed by the command fclose. In the same manner, thefile handle has to be passed to the command:
>> fclose(fp);
The read-in of formatted text works along the same lines; However, when using fopen, theoption ’r’ for reading of data (=read) has to be set, and for read-in, the command fscanf
has to be utilized with a suitable format.
Data can also be written and read in a binary format. Here, the data of the computer is written
into a file in its internal binary representation. An advantage of this type of data storage is
its usually low memory demand. A drawback is that the files produced in this manner cannot be opened and the contents inspected by a conventional text editor . In addition, it can
occur that the data transfer between two computers with different machine architecture ishampered, in case they utilize different representations of floating-point numbers and integers.
The respective commands to read and write binary data are fread and fwrite. The files areopened and closed as usual via fopen and fclose.
Additionally, Matlab provides a third possibility to store data: the commands save and load.The syntax of these commands is
>> load(’filename.mat’, ’var1’, ’var2’, ..., ’varN’);
and
>> save(’filename.mat’, ’var1’, ’var2’, ..., ’varN’); .
The variables with the names var1, var2 until varN are saved respectively loaded. The suffix
of a Matlab-file should always be .mat. Loss-free compression and system-wide compatibilityare the advantages of this method. Unfortunately, files created by different Matlab versions
are sometimes incompatible.
18
Note: if no variable names are specified, all variables that momentarily reside in the working
memory will be saved.
3.6.5. Example of a Simple Program: Orthogonality of Two Vectors
The following script file probes the orthogonality of two vectors ~a and ~b, i.e. it is tested
whether the scalar product of the two vectors is zero. Please try to understand the programwith the help of the given commentaries.
% orthog - program to test the orthogonality of two vectors (3D)
clear all; % delete all variables
% initalize the vectors a and b
a = input(’give 1. vector: ’);
b = input(’give 2. vector: ’);
% evaluate the scalar product
ab = 0;
for i=1:3
ab = ab+a(i)*b(i);
end
% tell whether the vectors are orthogonal
if (abs(ab) < 1.0e-8)
disp(’vectors are orthogonal’);
else
disp(’vectors are not orthogonal’);
fprintf(’scalar product = %g \n’, ab);
end
Matlab provides an own editor, that facilitates the writing of programs. In our case, the scriptfile would be named ’orthog.m’. Given that the file is in the correct folder, the program can
be invoked with the command orthog.
3.7. Self-defined Functions and Sub-programs
New, own functions can be defined as a function file. In the same way it is possible to
superimpose existing implementations of Matlab-functions with own functions. Assuming thatwe want to implement the Gaussian bell-curve
fσ(x) =1√2πσ
e−x2
2σ2 .
For this end we write the following file ’gauss.m’:
19
function y = gauss(x,sigma)
% function to calculate the Gaussian bell-curve
y = exp(-x.ˆ2/(2*sigmaˆ2))/(sqrt(2*pi)*sigma);
return;
The parameter y is an output parameter, while x and σ are input parameters. Note that therespective file name only differs from the name of the function the file defines by the suffix
’.m’. The function gauss is thus to be defined in ’gauss.m’, nowhere else. The self-definedfunction gauss will hence be used just like a built-in Matlab-function. For example with
x = -4:0.01:4;
plot(x,gauss(x,1));
the function can be plotted. Even the help functions help gauss and lookfor gauss work.
Printed on the screen are in this case the comment lines preceding the first command in thefile ’gauss.m’.
Naturally, new functions can also be provided with several output parameters. For this avector-valued output parameter is used, which components hold the actual output parameters.
Self-defined functions can also be called within any script or function file. This can be usedto structure programs into a main program and sub-program to increase the clarity and the
readability of the code ( see e.g. fig. 3.2). Conveniently, variables defined within a functionare always local, i.e. unknown to other program parts. Hereby, unwanted ’side effects’ are
avoided.
Figure 3.2.: Schematic example of the subdivision of a program (Calculation and illustrationof two oscillation modes of a physical system) into a main program and varioussub-programs.
20
3.7.1. Functions as Input Arguments
In many applications, one may want to not only give parameters to sub-programs, but also
functions. The evaluation of the function passed to the sub-program is done with the commandfeval. Consider an example of a sub-program which plots a given function in a predefined
interval including annotation of the axes, etc:
function plotfunction(f,a,b) % no output parameter
x = a:0.01:b;
y = feval(f,x);
plot(x,y);
xlabel(’x’);
ylabel(’y’);
return;
This sub-program is invoked for example by
plotfunction(’sin’,0,2*pi);
or
plotfunction(@(x)sin(x),0,2*pi);
3.7.2. Recursive Functions
Recursive functions constitute an elegant method to integrate iterative calculations in a simplefunction expression, that is capable of solving complex tasks. A recursive function calls itself
during the sequence of execution of its program code. As can easily be guessed, programmingrecursive functions involves the risk that a function calls itself over and over again, but never
returns to the calling program. Thus, a suiting termination criterion has to be provided, thatensures that after a specific iteration, no further recursion is initiated.
As an example we consider a program implementing a nested intervals method to evaluate thesquare root of a number x. We begin with an interval in which the sought for square root will
certainly be, thus for example [0, x]. Now we divide the interval in half and check in whichhalf the square root has to be. This half we take as the new interval and call the function
anew. The procedure is repeated until the interval is smaller than a given numerical accuracyand the square root is thus determined with sufficient precision:
% recursion_eng.m
% ===========
% calculates the square root of a variable x
% using a recursive nested intervals method.
%
% Invoke: x_sqrt = recursion_eng(x);
%
function x_sqrt = recursion_eng(x, x_low , x_high)
21
% no boundaries set? Then set plausible boundaries!
if ~exist(’x_low’)
x_low = 0;
x_high = x;
end
% Interval bigger than the accuracy of Matlab?
if (abs(x_low.^2-x) > 4*eps)
% yes, find the midpoint of the inteval
x_mid = (x_high+x_low)/2;
if (x_mid^2 > x)
% the square root is in the lower half...
x_sqrt = recursion_eng(x, x_low , x_mid);
else
% the square root is in the upper half...
x_sqrt = recursion_eng(x, x_mid , x_high);
end
else
% ...no, square root is found!
x_sqrt = x_low;
end
22
4. Graphics
4.1. Simple Plotting Functions
In order to represent data graphically, there exist numerous possibilities of visualization. The
simplest realizations are x-y-plots, where data points (x, y) are plotted in a Cartesian coordi-nate system. Examples:
t = 0:0.01:2*pi;
plot(t, sin(t));
x = 0:0.01:1;
plot(x, x.ˆ2);
The result of the last two commands is the graph of the function f(x) = x2 in the interval
x ∈ [0, 1]. Note the point in front of the power ˆ, which ensures that the operation isperformed element-wise.
You can even omit the last argument of plot, in which case one vector with N single function
values vector will be plotted dependent on the set of indices 1, 2, . . . , N :
plot(vector);
Generally, one would avoid the short syntax for small test programs and rather give the x-axisexplicitly.
The correct depiction of a function also needs clear annotation of the axes with the physical
units of the plotted quantities. The x-axis is annotated with the command xlabel(string);,the y-axis with ylabel(string);. As the name suggests, string is a string, i.e. a sequence
of characters , special characters and numbers, that has to be given in between quotationmarks. By title(string);, the graph can be given a title. Examples for this are:
t = 0:pi/120:2*pi;
plot(t, sin(t));
xlabel(’angle [rad]’);
ylabel(’amplitude [V]’);
title(’The sine is a pretty boring function’);
If one desires to plot more than one function into the same graph, Matlab can be advisedto do so by the keyword hold on; after plotting the first function. This command prevents
Matlab from drawing new axes for the following plots. All further plotting commands now addfunctions to the already existing graphs, until the command hold off; is given:
plot(t, sin(t));
23
Figure 4.1.: Simple example plot.
hold on;
plot(t, cos(t).*sin(t), ’r’);
...
(further plotting commands)
...
hold off;
When several function are plotted into the same graph, optical means of discrimination between
the functions is important. The main possibilities are changing the line style (continuous,dotted, dashed, ...) or the color of the plot symbols. Matlab accepts these options as a
combined string following the function values as a further argument in the plotting command.Some possibilities of constructing this sting are the following:
Colors: ’k’, ’r’, ’g’, ’m’, ’y’,...Line styles: ’-’, ’–’, ’.’, ’-.’,...
Plot symbols: ’o’, ’x’, ’.’, ’d’,...
As an example we consider the addition of a cosine function as a violet, dashed line to the
previous example plot:
plot(t, cos(t), ’m--’);
Since multiple annotation of an axis is not possible, a legend can be produced when several
functions are displayed in one graph. The command syntax for this is
legend(string1 string2 ... );
An example for the three functions in the previous example plot is:
legend(’sine’ ’sine * cosine’ ’cosine’);
Among further options, extensively listed by help plot;, are the manipulation of the line
width and the size of plotting symbols. These have to be give as a pair of arguments to
24
Figure 4.2.: Three functions in one graph.
the plot function: first a string, that unambiguously states which plot parameter is to beset, and second the value that is to be assigned to this parameter. The line width is set via
’LineWidth’, and the string addressing the size of plot symbols is ’MarkerSize’. Assumingwe would also like to depict the exponential function by a yellow continuous line of line width
4 and circles of the size 10 as plot symbols, we would have to write:
hold on;
plot(t, exp(-t), ’yo-’, ’LineWidth’, 4, ’MarkerSize’, 10);
hold off;
The intervals of the axes shown in the graph can be set by axis([x min x max y min
y max]);. The option tight tells axis to choose the intervals such that the plot win-dow circumscribes the plotted functions as close as possible. axis off; hides the axes of the
plot. Examples:
axis tight; axis([-5 15 -2 2]);
Did the function turn out nicely and is sought to be saved, the currently active plot window
can be written to a pixel or vector graphics file by print. The syntax for this command isprint(string, ’dFORMAT’, ’-rRESOLUTION’);. Here, string should be replaced by a
file name (without suffix, Matlab will append it automatically!), FORMAT by the desired formatand RESOLUTION by the desired resolution in pixel per inch. Examples for supported formats
(all others can be found by the help function) are:
FORMAT: ps, psc, eps, epsc, tiff, jpeg, png,...
4.2. Multiple Graphs
Multiple functions do not have to be plotted in the same graph, Matlab also allows to openseveral plot windows. A new plot window is created by figure. The windows are numbered
by Matlab. Thus, if one wants to access a specific plot window n (for example to add a graph
25
Figure 4.3.: Example of the definition of two plotting region in a 3x4 raster.
or annotation), the window is addressed by figure(n);. If the designated window does not
exist yet, a new window will be opened.
A window can be closed by close(n);; close all; closes all windows at the same time.
Furthermore the possibility exist to position several graphs next to each other in the same
plot window. For this, Matlab internally subdivides the window into nx times ny rectangles,that are numbered in ascending order from 1 to nx*ny from left to right and from top to
bottom. A new graph is created in one such rectangle by the command subplot(ny, nx,
k);. The output of the following plotting command will then be shown in the rectangle with
number k. It is also possible to extend graphs across several rectangles that together form abigger rectangle. To achieve this, a vector with two components has to be given as the third
argument to subplot, which holds the index of the upper left rectangle and the one of thelower right of the area where the graph is supposed to go.
Other useful functions are:
grid on, grid off;: activate/deactivate grid in the background of the current plot
loglog(x, y);, semilogx(x, y);, semilogy(x, y);: plotting in logarithmic resp. semilog-
arithmic representation
polar(angle, radius);: plotting in polar coordinates (important for polar bears and pen-
guins!).
26
−1 −0.5 0 0.5 1−1
−0.5
0
0.5
1
fx(t)
f y(t)
Figure 4.4.: Parametric function.
4.3. Parametric Functions
Parametric functions are also easily plotted. First, an independent variable, on which com-
ponents of the function depend, has to be defined as a sufficiently finely discretized vector.Second, the components of the function have to be calculated in dependency on this quantity
and are plotted against it.
An example is the harmonic oscillator in two dimensions (2D pendulum), that, due to friction,
slowly comes to a halt (note that Matlab, when creating annotations, even understands a bitof TeX, which comes in handy for example to set indices or Greek symbols!):
t = 0:0.1:100;
fx = exp(-t/20).*sin(t);
fy = exp(-t/20).*cos(t);
plot(fx, fy);
xlabel(’f x(t)’);
ylabel(’f y(t)’);
You can also animate plots nicely by drawing a sequence of graphs in quick succession. Atten-
tion! Following every complete plotting command, you should then use the directive drawnow;,to make Matlab display the plot immediately. Otherwise, Matlab will hold back the display
of the plot until the command interpreter is no longer busy and waits for new user input. Theplot that then appears will thus only be the last of the intended animation...
Example, based on the previous function definition:
for i=1:numel(t);
plot(fx(i), fy(i), ’ko’);
axis([-1 1 -1 1]);
drawnow;
end
27
4.4. Three-dimensional and Color-coded Displays
(Images)
Three-dimensional trajectories can be drawn with the plot3-command:
plot3(fx, fy, t);
Two-dimensional functions or surfaces can be depicted as a mesh grid:
[x grid, y grid] = meshgrid(-20:20, -20:20);
figure(1);
mesh(x grid);
title(’slanted plane’);
axis off;
z = exp(-(x grid.^2+y grid.^2)/100);
figure(2); mesh(z);
title(’bell-curve’);
axis off;
Figure 4.5.: Examples for grid functions.
A useful command is set(gca, ’DataAspectRatio’, [1 1 1]);, which scales all axes
such that their size relationship on the screen is approximately equal.
The command imagesc(array); depicts the two-dimensional function or matrix array asa color-coded top-view. The color code can be displayed next to the plot by the additional
command colorbar;:
imagesc(z);
title(’Bell-curve, top-view’);
colorbar;
axis off;
28
imagesc is a handy tool to display images, that are saved as arrays. If the array has a third
dimension of size 3, the entries of the array will be interpreted as color components accordingto the Red-Green-Blue (RGB) system. Attention: the y-axis will be reversed, which can be
fixed with the command set(gca, ’YDir’, ’normal’).
Instead of a color-coded illustration also contour lines can be used. The correspondent com-mand is contour(array);. Optionally, in a second argument the number of contour lines
used for the plot can be set:
Figure 4.6.: Examples for a top-view and a contour plot.
contour(z, 100);
title(’bell-curve, contour’); axis off;
The following table summarizes the most important graphics commands.
plot(x, y) graph of the vectors x and yloglog(x, y) logarithmic depiction
semilogx(x,y), semilogy(x,y) semi-logarithmic depictionpolar(theta, rho) graph in polar coordinates
contour(z) 2D contour line plot of matrix zmesh(z) 3D grid plot of matrix z
title(’text’) set figure titlexlabel(’x’),ylabel(’y’) annotation of the axes
grid on/off grid lines on/offlegend(text1, text2,...) legend
axis([xmin, xmax, ymin, ymax]) scaling of the x- and y-axissubplot(m, n, p) division in m× n subplots
clf delete current plotting windowfigure(n) create plot with number nhold on/off keep/let go of previous graph
print(’filename’) print/save figure
29
5. Systematic Programming
Computers are at the same time incredibly fast and incredibly stupid. They will do exactlywhat they were told, but usually not what you want. To change this or even prevent it
from happening, second to the thorough knowledge of the syntax of a programming languageand the available commands, a careful planning is mandatory – this becomes more and more
important the bigger the programs and the more complex the problems in question become.
Simply put, programming is similar to building a model with Lego bricks. From small piecessomething is put together, the final appearance of which has a wholly different character than
the pieces it is made from. In analogy to building Lego models, one has to think about whichpieces to use fist, and which to use later... and of course it has to be the correct pieces!
In the following, the systematic path form a problem to the final program is outlined. Twoexamples, finding prime numbers and simulating a diffusion process, complete this chapter.
5.1. Planning the Program
The following steps are reasonable to plan and finally write a program.
(a) Problem definition and solution
The first step is the exact definition of the program, the computer is supposed to solve.It should be clarified, which quantities constitute the input the computer has to process
and which output quantities the computer should return. Also one should be aware bywhich means or methods the problem shall be solved (In most cases, an arbitrary amount
of thought and time can be used up by this step. However, for this course, this aspect isregarded secondary, since in most given problems the ansatz for the solution is already
lined out. We want to focus on programming per se).
As a simple Example we consider the simple conversion of a monetary amount from Euroto US-Dollar. Input quantities are the amount in Euro as well as the exchange rate. The
output quantity is the amount in US-Dollar. This simple problem of conversion can besolved by trivial multiplication.
(b) Division into sub-steps
In the second step, the (calculation-)problem is divided into individual and clearly ar-
ranged sub-steps. These sub-steps should be chosen and organized according to thefollowing aspects:
- Which steps have to be executed first, which later?
31
The order of commands in a program is always important. For example, before
executing a command, all variables the command uses need to be known and avalid value assigned to them. Complex calculation on partial solutions can only be
carried out if the partial solutions were computed before. In our simple case, the
amount in Dollar can only be calculated, if the exchange rate has been assigned toa specific variable beforehand.
- Which steps have to be iterated several times resp. applied in a similarmanner on different quantities?
If steps are to be executed several times, loops are a good solution.
If similar tasks occur at different places during the solution of the problem, it is
advisable to use a function in the program code.
- Which steps have to be carried out dependent on a specific condition?
Always when a distinction of cases is due, program code will be executed only
under certain conditions. For example the absolute value of a number x is foundby multiplying it with −1 – but only if x is negative! Another example: the e-
mail client on your computer will only retrieve emails from a mail-server in casethey do not already exist as local copies on your hard drive (avoiding unnecessary
operation).
- Which steps demand user interaction or an informative feedback?
Some steps might need the confirmation of the user prior to their execution. For
example a program that formats your hard disk should ask you before it startsrunning.
Feedback is also important if errors occur during the execution of the program –for example when dividing a number by 0.
- Make sure that the solution can be found in finite steps.
This is also a very important point for the success of the problem solution: Thecomputer should be done after a finite number of steps, ergo in finite time. If this
is not given, a different solution procedure has to be found.
(c) Creating a flow chart (optional)
In addition to the division in sub-steps, you can also create a flow chart of your program,
where the individual sub-steps are connected by directed arrows.
Flow charts may be very helpful to review more complex program logics. Often problem
become apparent when creating a flow chart, that you were not aware of when mentally
dividing into sub-steps. For the creation of this diagrams, even a DIN norm exists (DIN66001). In the following an illustration will be used that is close to this norm and uses
the flow control elements and boxes introduced in the previous chapter(s).
(d) Transformation of sub-steps into program commands
Only now the carefully structured problem is translated to a specific programming lan-
32
guage. Here, also some guidelines exist that should be adhered to:
- How are data, data structures and intermediary results represented?
Not every mathematical algorithm is portable 1:1 to the computer. Complex num-
bers, for example are unknown to many programming languages – these will berepresented by a real part and an imaginary part in two floating-point numbers.
When working with Matlab is is important to consider when to use scalar quan-tities, vectors, two dimensional arrays or higher dimensional arrays. Important is
also which type of data is used. A floating-point number for example is unsuitableto represent a string. Also one should always be aware of the memory consumption
of the program.
- Replace every sub-step by at least one program command
- Unexpectedly complex sub-steps should be further broken down
- Adhere to the syntax of commands and expressions
For this, it is mandatory to use the help function! – especially for beginners or
when changing to another programming language. Every command has to beunambiguous, so the computer can understand what you want it to do.
After finalizing your program, it needs to be tested: Does the code perform as intended? If not,
that is it back to the drawing board: First, all the syntax errors are eliminated (somewhereis a comma instead of a dot, in a comparison the double equality sign was not used, an
intermediate result was not assigned to a variable, etc), and then the logic of the program ischecked – are the results plausible?
Assistance for this step – the so called ’debugging’ – provides the debugger, which is part of
many programming languages. Matlab also offers a debugging functions, however, many errorscan be solved by command line based interaction with the (with an error message) aborting
program. A good idea is to test the program with a reduced data set or problem, for exampleto run a search for prime numbers first until n = 17 before making the program return all
prime numbers between one and one trillion.
A lot of the aforementioned steps are not always clearly separable in practice. Most likely,
while breaking up a problem in smaller steps, you will already think about useful commandsyou know and how these might help. Program development is thus a constant interaction
between these different levels of planning.
5.2. Examples of Program Development
5.2.1. Finding Prime Numbers
The task our example program has to solve is the calculation of all prime numbers in theinterval from 1 to N . Reminder: a prime number is a natural number that is only divisible
without remainder by itself and by 1. 1 itself is no prime number.
33
Figure 5.1.: Flow chart of a ’brute-force’ prime number finder
The simplest method to determine whether i is a prime number is the following: One triesevery divisor from 2 to i− 1. If a divisor is found that divides without remainder, then i is no
prime number. This so called ’brute-force’-ansatz (leads to an end, but is not very smart) canbe easily implemented in a program. We check, in a loop, whether for a particular number
i a divisor can be found. If this is the case, a so-called flag is set (a variable, that is either0 or 1 and signal, whether a certain event occurred). If the flag is not set after the loop is
finished, no divisor has been found and the number is remembered as a prime number. If theflag is not set, the program does nothing. After this, the next number i+1 is scrutinized and
so forth until all natural numbers until N have been tested. The latter will also be done in aloop. Figure 5.1 shows the flow chart, the following gives the correspondent Matlab code:
34
%%% TASK:
%%% ========
%%% find all prime numbers between 1 and N and save them
%%% in a vector ’prime_numbers ’
%%%
%%% solution : brute force
%%% until which natural number shall be calculated?
n = 10000;
%%% there are maximally m<=n-1 prime numbers for all natural
%%% numbers until n
prime_numbers = zeros([1 n-1]);
%%% The next prime number shall be saved in the vector component
%%% with the index idx_prime_number
idx_prime_number = 1;
%%% check all numbers
for i=2:n
%%% try all divisors from 2 to i-1
found = 0;
for divisor =2:i-1
%%% is the number i divisable by divisor without remainder?
if (i/divisor == fix(i/divisor ))
found = 1;
%%% terminate the loop , one divisor is enough!
break; %%%WITH OPTIMIZATION
end
end
%%% No divisor found? Then the number i is a prime number!
if (gefunden == 0)
%%% remember prime number , go to next index
prime_numbers(idx_prime_number) = i;
idx_prime_number = idx_prime_number+1;
end
end
%%% shorten the result vector ,
%%% only keep the found prime numbers
prime_numbers = prime_numbers(1: idx_prime_number -1);
35
Figure 5.2.: Flow chart for the ’Sieve of Eratosthenes’.
For large N this code will run for a long time, that is to say (running Matlab 6.1, Intel DualCore
processor, n = 10000) approximately 141 seconds. When scrutinizing the program code, apoint can be found at which the code can be optimized: As soon as one divisor is found, it
does not make any sense to further execute the inner loop. It can be left with break. This isespecially sensible for large i: If i is already divisible by 2, the divisors from 3 to i− 1 do not
have to be tested any more! By this, the execution time of the program reduces itself to 16seconds.
36
A much more sophisticated method to find prime numbers is the ’Sieve of Eratosthenes’. The
principle is easy to grasp: First a list of all natural numbers is generated from 1 bis n, fromwhich successively all non-prime numbers are deleted. Except for 1 at first all numbers are
candidates form prime numbers. Now every number i from 2 to n is checked. If i is already
marked as a non-prime number, it is skipped. If it is not marked (like for example 2 at thebeginning of the procedure), all multiples of this number are marked as non-prime numbers,
i.e. 2i, 3i, 4i until n. Left over will only be the prime numbers!
See the flow chart of the correspondent program (figure 5.3) and the Matlab code:
%%% TASK:
%%% ========
%%% find all prime numbers between 1 and N and save them
%%% in a vector ’prime_numbers ’
%%%
%%% solution : sieve of Eratosthenes
%%% until which natural number shall be calculated?
n = 10000;
%%% define so called flags for all natural numbers
%%% until n. Is the flag equal to 1 at the end of the program
%%% the number is a prime number
flag_prime_number = ones([1 n]);
%%% by definitioin , 1 is not a prime number
flag_prime_number(1) = 0;
for i=2:n
%%% is the current number i a prime number?
if (flag_prime_number(i) == 1)
%%% yes , mark all multiples as non-prime numbers
for j=i+i:i:n % WITHOUT OPTIMIZATION
flag_prime_number(j) = 0; % WITHOUT OPTIMIZATION
end % WITHOUT OPTIMIZATION
%%% flag_prime_number(i+i:i:n) = 0; % WITH OPTIMIZATION
end
end
%%% find the indices of all entries =1 in the vector
%%% the indices are the sought prime numbers
prime_numbers = find(flag_prime_number);
The run time underlines the ingenuity of the sieve-idea: 0.22 seconds instead of 16 seconds!
Additionally, the program code can even be optimized by replacing the inner loop by a vector-
37
ized command (comment all lines marked with WITHOUT OPTIMIZATION and uncomment all
lines marked with WITH OPTIMIZATION): 0.05 seconds!
With this, in comparison to the first attempt at this program, the execution time could beshortened by a factor of 6400! Important to note is that both programs are completely correct
and yield identical results, depite their different appearance and run time.
5.2.2. Simulation of Diffusion
A second example is a simulation of a so-called ’random walk’. A random walk describes amovement of a particle (e.g. molecule, protein) that is determined by a random process. In
our case the particle starts at the location x = 0 and moves in every time step by a distance ofdx either to the left or to the right. Both possibilities have the same probability of happening.
The task for the computer is to simulate n = 1000 trials with the particle and stop thesimulation each time the particle crosses a barrier at x = −1 or x = 1. The output of our
program shall be the mean number of steps needed to reach one barrier, calculated from the
n iterations.
This is the flow chart (figure 5.3) and the program code:
%%% TASK:
%%% ========
%%% perform a random walk with step width dx starting at
%%% x=0 until x crosses either -1 or +1
%%%
%%% perform this many trials:
n_trials = 1000;
%%% elementary step width
dx = 0.5;
%%% initialize result vector
steps = zeros([1 n_trials ]);
for i=1: n_trials
%%% start random walk at x=0
x = 0;
%%% initialize counting variable
n_steps = 0;
%%% perform random walk until +1 or -1 is reached
while abs(x) <= 1
%%% draw random number.
%%% choose direction with equal probability
38
if (rand >= 0.5)
%%% to the left...
direction = -1;
else
%%% to the right...
direction = 1;
end
%%% update position and number of steps
x = x+direction*dx;
n_steps = n_steps +1;
end
%%% return number of steps
fprintf ([’In the %i-th trial , %i steps needed ’ ...
’until boundary !\n’], i, n_steps );
%%% remember number of steps for trial i
steps(i) = n_steps;
end
%%% evaluate the mean of the steps and return the value
fprintf(’the mean number of steps used is %f ...\n’, ...
sum(steps)/ n_trials );
39
Figure 5.3.: Flow chart for the simulation of a diffusion process.
40
6. Vectorization and Vector Calculus inMatlab
A great advantage of Matlab is that all variables are matrices (or, more general, arrays).
The benefits of this principle are manifold: They encompass the simplicity of the syntax to
formulate complex mathematical operations, avoidance of tedious for-loops and optimizationof internal processing of huge amounts of data resulting in considerable acceleration of the
program.
6.1. Basic Syntax
The easiest definition of vectors and matrices is to put the elements of a matrix between
square brackets. Column vectors are treated in Matlab as matrices of dimension N × 1 ,row vectors as matrices of dimension 1 × N and scalars (i.e. plain numbers) as matrices of
dimension 1×1.Rows are divided from other rows by the use of a semi-colon, and columns areseparated by commata (oder simply by a blank space, as in the last of the following examples).
Examples:
scalar: a scalar = 1; 1x1-matrixcolumn vector: a column = [1; 2; 3]; 3x1-matrix
row vector: a row = [1, 2, 3]; 1x3-matrixmatrix: a fullmatrix = [1 2 3; 4 5 6; 7 8 9]; 3x3-matrix
The same rules that subsume single elements or variables to a vector also apply for strings andso-called cell-arrays:
b = [’A ’ ’character’ ’string’ ’of four elements’];
c = rand([4 4]) 1:17 complex(42) ’this is a cell of a cell-array!’;
For the ones interested to dive deeper into Matlab: Note that cell-arrays are extremely useful
to save several aspects of data in the same variable. An example from the laboratory wouldbe a variable that holds the measured data, a date for the day of recording, a description of
the experimental setup in a string and the parameters of recording. Furthermore, structuresexist, whose constituent parts have names that can be accessed with a syntax similar to the
programming language C.
41
6.2. Definition of Matrices and Arrays
Simple vectors can be defined with the already mentioned colon-operator. The syntax is startvalue:step width:end value. If the step width is omitted, the default step width of 1 is
used. Examples:
1:7 creates a vector with the numbers 1, 2, 3, 4, 5, 6, 7.
1:2:7 creates a vector with the numbers 1, 3, 5, 7.
4:-0.3:2.8 creates a vector with the numbers 4.0, 3.7, 3.4, 3.1, 2.8.
The vectors defined in this way shall be assigned to a variable:
p = 0:0.01:100;
Such vectors, if used in loops, will be processed element-wise. In the following example, a
vector with the numbers 1, 17 and 42 is printed to the screen element-wise:
to process = [1 17 42];
for k=to process;
fprintf(’k is %i\n’, k);
end
The output is:
k is 1
k is 17
k is 42
Furthermore, the colon operator can be used to access columns, rows or arbitrary dimensions
of an array. The colon by itself selects all elements of the respective dimension. In the syntaxintroduced above, the colon operator can also be used to select a specific set of indices of a
matrix:
b = a(4, 7:5:17); selects, from the 4. row of the matrix a the entries of the columns 7, 5
and 17. It creates a vector from these elements and assigns it to the variable b.
Bigger matrices and arrays are defined by the commands zeros, ones and eye. The two
arguments of these functions assign the number of columns and rows, e.g. fullofzeros =
zeros(17, 42);. zeros fills the created matrix with zeros, ones with ones, and eye createsan identity matrix. More than two arguments result in multidimensional arrays (tensors).
There are several shortened notations of these commands. Some examples:
z = zeros([1 17]); correct, creates 1x17-matrix.
z = zeros(1, 17); correct, creates 1x17-matrix.
z = zeros([1, 17]) correct, creates 1x17-matrix.
z = zeros(1:17) wrong, tries to create a 1x2x3x4x5x6x7x8x9x10x11x12x13x14x15x16x17-
matrix and does not find enough memory!
A similar problem can occur when invoking the commands with only one argument n. Matlab
will try to create a nxn-Matrix instead of a 1xn-vector (which might have been intended).
42
z = zeros(1e6); wrong, tries to create a 1000000x1000000-matrix!
z = zeros(1, 1e6); correct, creates a 1x1000000-vector.
If you are in doubt, use, as always, the help function! Multidimensional arrays lend themselvesfor example to save the color planes of a digital image. A matrix for that purpose would be
created by a = zeros([1024 768 3]);.
As the last command in this series we will cover meshgrid.meshgrid takes two vectors as
arguments and duplicate the first vector in the rows of the output matrix. The second vectoris duplicated along the columns of the output matrix. The number of duplications of a vector
is determined by the number of elements of the other vector. Example:
[x matrix, y matrix] = meshgrid(-10:10, -10:10); creates two matrices, whose en-tries of the columns resp. of the rows range from −10 to 10. This command is suitable to
define coordinate axes for two-dimensional plots. An example can be found in the chapter’Graphics in Matlab’.
When working with initializations and arrays, it is important to be aware of how the contentsof these arrays are administrated in the computer. Matlab hides many tasks form our eyes,
that have to be attended to in many other programming languages first before being able towork with variables and arrays.
One of these tasks is the declaration of variables. In a ’proper’ programming language, one
has to determine the type of a variable when introducing it. Different types are floating-point numbers, integers, strings, etc. If a numerical value is assigned to a variable, Matlab
automatically assumes that the type of the variable is intended to be a double-precision floating-point number. Other types have to be created by an explicit type conversion.
Another task is the allocation and deallocation) of memory: An array needs memory to berepresented in the computer. Before the variable can be used, the computer has to be advised
first to reserve a fitting space in the memory for this variable (just as a real estate has to bebought first before one is allowed to start building a house on it). In the same manner, memory
has to be freed again if it is no longer needed (to prevent that empty real estates are left in thememory that block the space from other house builders). Matlab hides these processes and
does all the administrative work for us. This can become critical if one writes into a variable
that was previously undefined. For example the command clear; a(100000)=pi; creates avector with 100000 entries, and only in the 100000-th entry the number π is written. If the
range of the vector, for which only 100000 elements were reserved, is now exceeded by writinga(100001)=pi;, the following will happen:
• a new vector with 100001 elements is created.
• into element 100001, 2π is written.
• the elements 1 bis 100000 are copied from the old to the new vector.
• the memory allocated by the old vector is deallocated.
• the new vector takes the name of the old vector.
...a lot of effort to append an element. This is why you should always keep in mind to properly
43
define and initialize arrays (with zeros or ones) before using them!
6.3. Simple, predefined Vector Operations
Matlab is capable to process all elements of a vector or a matrix in one go:
a = rand([1 17]);
b = rand([1 17]);
c = a+b;
The mathematical equivalent is ci = ai + bi, ∀i = 1, . . . , 17.
Vectors and matrices have to match in their dimensions; A prerequisite for the Addition ofarrays is that all dimensions have the same size. The same holds for subtraction.
Element-wise multiplication, division and exponentiation is indicated by a preceded dot:
c mult = a.*b;
c div = a./b;
This convention was introduced in Matlab to tell apart the element-wise multiplication fromthe matrix multiplication.
C = [1,1;0,1]; defines the 2× 2 matrix C =
(
1 10 1
)
D = [1,2;3,1]; defines the 2× 2 matrix D =
(
1 23 1
)
G = C*D; gives G =
(
4 33 1
)
.
The mathematical equivalent is Gkl =∑
j CkjDjl.
Transposing a matrix Ctkl = Clk is done by
K = C’; gives K =
(
1 01 1
)
The function transpose(C) can be utilized alternatively.
Inverting a matrix is accomplished by
M = Dˆ-1; gives M =
(
−0.2 0.40.6 −0.2
)
44
resp. M = inv(D);. Assuming a system of equations D~x = ~b and the vector ~b with
~b =
(
31
)
is given, the solution, by using the inverse, is found in Matlab by:(
xy
)
= M
(
31
)
=
(
−0.2 · 3 + 0.4 · 10.6 · 3− 0.2 · 1
)
=
(
−0.21.6
)
.
The solution of a system of equations can also be found by using the non-inverted matrix and
one of the symbols /, \ resp. the commands mldivide and mrdivide.
Important in a physical or geometrical context is the cross product of two vectors, which is
realized in Matlab by the command cross:
a = [0; 0; 1];
b = [1; 0; 0];
c = cross(a, b);
c
The output of these lines is:
c =
0
1
0
6.4. Reshaping, Copying and Duplication of
Matrices/Arrays
A useful command to reorganize data is reshape. The content of the array is not altered bythis command, only the organization in columns and rows, respectively the other dimensions.
The syntax is result = reshape(array, [dim1 dim2 dim3 ... dimN]);. The resultis an array of dimensions dim1 to dimN. The product of these dimension has to equal the
product of the old dimension of the array.
In the following situations, the command is sensibly applied:
• Reshaping of arrays to a form that is suitable for vector operations. Let us assume thatwe have 8 vectors of 10 constellations each in a 3x8x10-array wit the name r. Now we
want to rotate the vectors with a rotation matrix D ,to simulate the movement of the
night sky:
dummy = D*reshape(r, [3 80]);
r = reshape(dummy, [3 8 10]);
We first reshaped the 8x10-matrix of vectors to form a 800-element vector, then we
applied the rotation and finally brought the result matrix back to its original form.
45
• Interpretation of data, that was initially written from a file into a one-dimensional array.
Example:
f = fopen(’digitalfphoto.bin’, ’r’);
a = fread(f, ’uint8=>uint8’);
fclose(f);
imagesc(reshape(a, [500 375 3]));
set(gca, ’DataAspectRatio’, [1 1 1]);
The second to last command reshaped the array to enable the usage of imagesc to
display the image correctly.
In the last example, the Matlab user might be annoyed that the imagesc-command projects
the first dimension of the array (with length 500, thus e.g. the x-axis of a photo in landscape
format!) onto the y-axis. The versatile command permute remedies this situation – thiscommand swaps single dimensions of an array. It is important to understand that not only the
organization of an array in rows and columns is changed, but also the sequence of the entries inthe memory! The syntax of the command is permuted = permute(array, [dimidx dim1,
dimidx dim2, ..., dimidx dimN]);. Here, the second argument holds the numbers from1 to N (where N is the number of dimensions of an array, i.e. N = 2 for a matrix), that specify
the permutation of the dimensions. In the array permuted, the first dimension will be thedimidx dim1-th dimension of the old array array, the second dimension the dimidx dim2-th
dimension of the old array, etc.
In order to display the digital photo from our example correctly, we thus have to type:
a permuted = permute(reshape(a, [500 375 3]), [2 1 3]);
imagesc(a permuted);
This interchanges the x- and the y-direction, while the color information is preserved in the
third dimension.
repmat, another important command, duplicates an array or matrix along one or several
dimensions. The syntax is similar to reshape,
result = repmat(array, [rep dim1 rep dim2 rep dim3 ... rep dimN]);
The matrix array is cloned rep dim1-times along the first dimension, rep dim2-times along
the second dimension, etc.
46
For example, the following commands,
a = [1 2; 3 4];
repmat(a, [3 4]) ,
give the output:
ans =
1 2 1 2 1 2 1 2
3 4 3 4 3 4 3 4
1 2 1 2 1 2 1 2
3 4 3 4 3 4 3 4
1 2 1 2 1 2 1 2
3 4 3 4 3 4 3 4
Using this command makes sense, if matrix operations are repeated with different operandson the same initial vector. An example may be found in the section ’vectorization’.
6.4.1. What Size does an Array have?
Especially when manipulating a lot of different arrays, when a function is supposed to operate
on an array of previously unknown size or when one wants to find out whether two arrays are
compatible to serve as an input to a matrix operation, the following commands will be helpful:
size, length and numel.
It follows a short explanation for their function:
• size: returns the size of an array in a vector that has as many elements as the arrayhas dimensions. Example:
size(zeros([17 42]))
ans =
17 42
• length: returns the longest dimension of an array, irrespective of which dimension thisis. Example:
length(zeros([17 42]))
ans =
42
length(zeros([42 17]))
ans =
42
• numel: returns the total number of all elements of an array and is identical with the
construct prod(size(array));. Example:
numel(zeros([17 42]))
ans =
47
714
6.5. Example: Vectorization of a Program
To close this chapter, we will discuss two examples of what to do when vectorizing a program.First, consider the problem of generating a NxN -Matrix with entries aij = i · j. The tedious
way via loops looks like this:
N = 17;
a1 = zeros([N N]);
for i=1:N, for j=1:N; a1(i, j) = i*j; end; end
It is much simpler to first create a row vector of the numbers 1 bis N , and subsequentlyduplicate this this vector via repmat to obtain a NxN -matrix. Finally, this matrix is multiplied
element-wise with its transpose:
N = 17;
onefold = 1:N;
manifold = repmat(onefold, [N 1]);
a2 = manifold.*manifold’;
However, this is still not yet the pinnacle of elegance that can be reached. Thinking a littleabout this problem, one might come to the conclusion that the mathematical operation that
is intended can be described as a matrix multiplication of a Nx1-matrix with a 1xN -matrix(the so-called outer product of two vectors):
N = 17;
onefold = 1:N;
a3 = onefold’*onefold;
As a second example we consider the following program: It defines two vectors with x- andy-coordinates, performs a translation of these coordinates and finally a rotation.
48
%%% Example: before vectorizing
x = [ 0 -1 0 -2 0 -3 0 -4 0 -5 0 0 ...
0 5 0 4 0 3 0 2 0 1 0];
y = [ 6 4 4 2 2 0 0 -2 -2 -4 -4 -6 ...
-4 -4 -2 -2 0 0 2 2 4 4 6];
%%% Translation in x- and y-direction
trans_x = 4;
trans_y = -4;
%%% Rotation
phi = pi/4;
n = numel(x);
for i=1:n
%%% apply translation to x,y
x(i) = x(i)+ trans_x;
y(i) = y(i)+ trans_y;
end
for i=1:n
%%% apply rotation to x,y
x_temp = x(i)*cos(phi)+y(i)*sin(phi);
y_temp = -x(i)*sin(phi)+y(i)*cos(phi);
x(i) = x_temp;
y(i) = y_temp;
end
As a first step, we can subsume the 23 coordinate-tuple to a2x23-matrix, that we call r:
r = [x; y];
Here we used the semicolon to stack the row vectors on top of each other. Every column of
the matrix r now holds a coordinate tuple. In the next step, we rewrite the translation vectoras well as a column vector:
trans = [trans x; trans y];
Now we might be tempted to replace the first for-loop by an element-wise addition of transand r. But beware! The dimensions of both variables are not compatible, trans has only one
column. Thus we use the command repmat, to duplicate trans 23 times. After this we canperform the addition:
r = r+repmat(trans, [1 size(trans, 2)]);
49
Here, we used the command size to avoid having to use the constant 23 – the code will thus
also work with other, longer or shorter variables x or y.
Next up is the rotation. The rotation matrix D is quickly defined,
D = [cos(phi) sin(phi); -sin(phi) cos(phi)];,
and equally quickly applied to r:
r = D*r;
In the end, we assign the result again to the variables x and y . For this we use the colon-
operator that selects whole rows of the matrix r:
x = r(1, :);
y = r(2, :);
We can even integrate rotation and translation:
r = D*(r+repmat(trans, [1 size(r, 2)]));
This gives the vectorized program code:
%%% Example: AFTER vectorization
x = [ 0 -1 0 -2 0 -3 0 -4 0 -5 0 0 ...
0 5 0 4 0 3 0 2 0 1 0];
y = [ 6 4 4 2 2 0 0 -2 -2 -4 -4 -6 ...
-4 -4 -2 -2 0 0 2 2 4 4 6];
%%% Translation in x- and y-direction
trans_x = 4;
trans_y = -4;
%%% Rotation
phi = pi/4;
%%% Definition of the vectors
r = [x; y];
trans = [trans_x; trans_y ];
D = [cos(phi) sin(phi); -sin(phi) cos(phi)];
%%% Two to three loops vectorized:
r = D*(r+repmat(trans , [1 size(r, 2)]));
%%% assign the result
x = r(1, :);
y = r(2, :);
50
Part II.
Numerical Methods in Matlab
51
7. Integration and Differentiation
7.1. Numerical Integration
In many cases, the antidervative can not be expressed in closed form via standard functions.This makes numerical methods for the approximation of the antiderivative necessary (’quadra-
ture’). The intention of this chapter is to introduce and explain the basic concepts of numericalintegration. For this, we consider definite integrals of the form
I =∫ b
af(x)dx
with −∞ < a < b < ∞ and ’sufficiently smooth’ function f(x). Such an integral can be
interpreted as the area (with sign) under the curve f(x) (see fig. 7.1).
x
y f(x)
a b
Figure 7.1.: Definite integral with boundaries a and b as the area under a curve f(x).
An example for which numerical integration is useful are one-dimensional ordinary differential
equations, that can be written as definite integrals:
dy
dx= f(x) and y(a) = 0 ⇒ y(b) =
∫ b
af(x)dx = I .
7.1.1. Midpoint Rule
The simplest way of numerical integration is via the midpoint rule, or also rectangle method.
Here, the interval of integration is split into sub-intervals as shows in fig. 7.2. The sub-intervalshave a width of ∆x = (b−a)/N , the midpoints of these sub-intervals are xi = a+(i−1/2)∆x
with i = 1, . . . , N . The points xi are also called supporting points.
53
x
y
a bx x x1 2 3
x∆
Figure 7.2.: Scheme of the midpoint rule.
The integral is then replaced by the Riemann sum:
I ≈ IM =N∑
i=1
f(xi)∆x
To illustrate the accuracy of this approximation, consider the following integral of the Gaussianbell-curve
I =∫ 1
0
2√πe−x2
dx .
Due to the regular occurrence of this integral in many contexts, the corresponding primitive,
the so called ’error’-function erf(x), was defined. In the literature, we find
I = erf(1) = 0.842701...
The following table shows the comparison of the exact solution with the result of the midpointrule for different numbers of supporting points N .
N IM ∆I = I − IM5 0.844088 -0.0013869110 0.843047 -0.0003461320 0.842787 -0.0000864960 0.842711 -9.61 10−6
From the table we can derive that the error is proportional to 1/N2 respectively ∆x2. This
can also be shown analytically. For this we first examine the part integral
Ii =∫ xi+∆x/2
xi−∆x/2f(x)dx .
The sub-interval [xi−∆x/2, xi+∆x/2] should be chosen small enough, such that the functionf(x) varies sufficiently little in the interval. Within this interval, we can limit the consideration
to the first terms of the Taylor series:
f(x) = f(xi) + f ′(xi)(x− xi) + f ′′(xi)(x− xi)
2
2+ f ′′′(xi)
(x− xi)3
6+O((x− xi)
4) .
54
Substitution shows that some terms vanish
Ii = f(xi)∆x+ f ′(xi)(x− xi)
2
2
∣
∣
∣
xi+∆x/2
xi−∆x/2+ f ′′(xi)
(x− xi)3
6
∣
∣
∣
xi+∆x/2
xi−∆x/2
+f ′′′(xi)(x− xi)
4
24
∣
∣
∣
xi+∆x/2
xi−∆x/2+O(∆x5)
= f(xi)∆x+ f ′′(xi)∆x3
24+O(∆x5) .
Writing now with central derivation
f ′′(xi) =f ′(xi +∆x/2)− f ′(xi −∆x/2)
∆x+O(∆x2) .
Insertion yields
Ii = f(xi)∆x+ [f ′(xi +∆x/2)− f ′(xi −∆x/2)]∆x2
24+O(∆x5) .
Taking the sum I =∑N
i=1 Ii, many terms will vanish, thus
I − IM =∆x2
24[f ′(b)− f ′(a)] +O(∆x4) .
Hence, the error is proportional to ∆x2 and depends on the derivations at the boundaries of
the interval.
7.1.2. Trapezoidal Rule
Instead of approximating the area by little rectangles, here, trapezoids are used (see fig. 7.3).
x2x = a1 x = b4x3 x
y
x∆
Figure 7.3.: Scheme of the trapezoid rule. The dashed line illustrates the division of a trapezoidinto a triangle and a rectangle.
The width of the sub-intervals is ∆x = (b − a)/(N − 1). The supporting points lie at theborders of the sub-intervals: xi = a + (i− 1)∆x with i = 1, . . . , N .
55
In the part integral I1, the area of the trapezoid can be split up into an area of a triangle and
the area of a rectangle (see fig. 7.3),
I1 =∫ x2
x1
f(x)dx ≈ f(x1)∆x+1
2[f(x2)− f(x1)]∆x =
1
2[f(x1) + f(x2)]∆x
It follows the ’compound’ trapezoid rule:
I ≈ IT =1
2[f(x1) + f(xN )]∆x+
N−1∑
i=2
f(xi)∆x .
Note that the trapezoid rule contains the boundaries a and b of the interval. This is referred
to as a closed formula. In contrast, the midpoint rule does not contain the boundaries of theinterval (open formula). The latter is an advantage, in case that integrable singularities exist
at x = a or x = b.
The error approximation gives, in analogy to the procedure used for the midpoint rule,
I − IT = −∆x2
12[f ′(b)− f ′(a)] +O(∆x4) .
Thus, the trapezoid rule is less accurate by a factor of two than the midpoint rule. This isa surprising result, considering the two figures 7.2 and 7.3 which suggest the opposite. The
seeming contradiction is resolved when realizing that the procedure of the midpoint rule partlycompensates positive and negative deviations from the exact solution.
7.1.3. Simpson’s Rule
Simpson’s rule is often used since it is more accurate than the midpoint or the trapezoid rule.The idea behind Simpson’s rule is to approximate the integrand function as a parabola between
three supporting points. For this parabola, the integral can be found analytically.
As in the trapezoid rule, the supporting points xi = a+(i−1)∆x are used with i = 1, . . . , N
and N odd. The width of the sub-intervals is again ∆x = (b− a)/(N − 1). The ’compound’Simpson’s rule reads
IS =∆x
3(f(x1) + 4f(x2) + 2f(x3) + . . .+ 2f(xN−2) + 4f(xN−1) + f(xN)) .
The error shall be given without strict derivation:
I − IS = O(∆x4) .
56
The error term is of higher order as compared to the trapezoid rule. Thus we expect the
error to be much smaller with the Simpson’s rule. Figure. 7.4 confirms this expectation. Withthe same number of supporting points, Simpson’s rule gives far more accurate values for the
integral than the trapezoid rule.
Figure 7.4.: Comparison of the absolute errors between the trapezoid rule and Simpson’s rulewhen evaluating the integral I =
∫ 10
2√πe−x2
dx as a function of the number ofsupporting points N . Note the double logarithmic plotting. For large N , roundingerrors dominate.
7.1.4. Integration with Matlab
Matlab provides several routines for integration. The most important one for one-dimensional
integrals is
y = quad(f,a,b);
Here, f is the integrand function and a and b are the boundaries of the interval. This routinesuses an adaptive Simpson rule. It is adaptive in the sense that more supporting points are
used where the function varies strongly. Invoking the routine can be done by
y = quad(@(x)exp(x),0,1);
or
y = quad(@(x)1.0./(1.0+x.ˆ2),0,1);
57
7.1.5. Miscellaneous
Further interesting methods and aspects of numerical integration are
• Non-uniformly distributed supporting points:
– adaptive quadrature methods
– Gaussian quadrature
• Improper integrals, i.e. the function or the interval of integration is boundless
• Multidimensional integrals, e.g.
∫ ∫
f(x, y) dxdy
7.2. Numerical Differentiation
Many applications demand the approximate differentiation of functions via a numerical method.
Finding the analytical differentiation is for example impossible if the function to be differenti-ated is by itself only given numerically.
7.2.1. Right-sided Differentiation
The differentiation of a function f(x) is defined by
f ′(x) = limh→0
f(x+ h)− f(x)
h.
Since h = 0 is numerically impossible, one has to choose h ’suitably small’. Geometrically thismeans that the sought tangent on f at the position x is approximated by the secant through
x and x+ h, see fig. 7.5(a).
To estimate the resulting approximation error, we consider the Taylor series up to the secondorder
f(x+ h) = f(x) + f ′(x)h +f ′′(x)
2h2 +O(h3) .
Figure 7.6 illustrates the Taylor series of the sine function at x = 0, sin (h) = h − h3/6 +h5/120 + O(h7). It can be seen that the Taylor series becomes a better and better approx-
imation of the sine function with increasing order. Close to the point of construction of theTaylor series (here x = 0), the original function can be replaced in a controlled manner by its
Taylor series of a suitable order.
Rewriting the above Taylor series results in the relation
f ′(x) =f(x+ h)− f(x)
h+O(h) .
58
(b)
x+hx−h x
f(x+h)
(a)
x x+h
f(x)
f(x+h)
f(x−h)
Figure 7.5.: Approximation of the slope of the tangent on the curve f at the position x by theslope of the secant (a) in x and x + h resp. (b) in x− h and x + h. The latteris a decidedly better approximation.
0 0.5 1 1.5 2 2.5 30
0.2
0.4
0.6
0.8
1
1.2
h
Figure 7.6.: Sine function (continuous line) and the corresponding Taylor series including the1. order(+), 3. order (o) and 5. order (x).
The left hand side is the exact derivative. The right hand side is the right-handed derivative
and the approximation error, also called truncation error or discretization error, which is linear
in h. The truncation error can thus be made small when choosing h accordingly. Unfortunately,when h gets to small, rounding errors start to contribute significantly such that an optimization
of the trade-off between truncation error and rounding error has to be found. This is illustratedin figure 7.7 for the example of f(x) = x2 at the position x = 1. The total error is plotted on
a logarithmic scale
∆(h) =
∣
∣
∣
∣
∣
f ′(x)− f(x+ h)− f(x)
h
∣
∣
∣
∣
∣
,
as a function of h = 10−i, i = 1, 2, . . . , 25. For f ′(x) the known analytical solution 2x is
59
inserted. The linear decrease of the truncation error for h > 10−8 is apparent. For h ≤ε ≈ 2 × 10−16, the rounding error dominates, where ε is the machine accuracy. The optimalvalue h ≈ 10−8 is explained by the fact that the truncation error O(h) and the rounding error
O(ε/h) (denominator/numerator) are in balance if h ≈ √ε ≈ 10−8.
Figure 7.7.: Total error of the right-sided derivation of f(x) = x2 at x = 1. Note the doublelogarithmic plotting.
7.2.2. Centered Differentiation
Consider the following Taylor series
f(x+ h) = f(x) + f ′(x)h +f ′′(x)
2h2 +
f ′′′(x)
6h3 +O(h4)
f(x− h) = f(x)− f ′(x)h+f ′′(x)
2h2 − f ′′′(x)
6h3 +O(h4) .
Subtraction of both equation gives
f ′(x) =f(x+ h)− f(x− h)
2h+O(h2) .
The right hand side is the centered first derivation approximation. Note that the truncation
error now decreases quadratically with the decrease of the step width h, although the evaluationof this formula is not more extensive than the one of the right-sided derivation. In both cases,
the function f has to be calculated at two points. In contrast to the right-sided derivation,the sought slope of the tangent on f at the position x is now approximated by the slope of
the secant through x− h and x+ h (see fig. 7.5(b)).
60
If we add the Taylor series for f(x+ h) and f(x− h) instead of subtracting them, and then
divide by h2, we obtain the centered second derivation approximation
f ′′(x) =f(x+ h)− 2f(x) + f(x− h)
h2+O(h2) .
The truncation error is also quadratic in h.
61
8. Differential Equations
8.1. Ordinary Differential Equations
In many fields of science, problems occur where one or several quantities vary dependent on
time, e.g. oscillations or decay processes. The description of these processes can be formalizedby differential equations. As a simple example, radioactive decay may serve. An amount of
radioactive material is give at time t = 0. We are looking for a function x(t) that gives the
amount of the material x which is still present at time t. The rate at time t by which theradioactive material decays is proportional to the amount of the material x(t) that exists at
time t. From these considerations, the following differential equation can be derived:
x = −αx
with decay constant α > 0. This differential equation is referred to as ’ordinary’, since it
only includes the derivative with respect to one quantity (here time t). It is furthermore adifferential equation of ’1. order’, since only the first derivative is taken.
The solution of a differential equation is not a number, but a function. Here, the solutionis x(t) = x0e
−αt, which can be proven by taking the first derivative. Unfortunately, in most
practical cases, a differential equation can not be be solved analytically. This holds especiallyfor ordinary systems of differential equations. An example is the Newtonian mechanics of a
particle
m~r = ~F (~r, ~r, t)
with place ~r = (x, y, z), velocity ~v = ~r, acceleration ~r, mass m, force ~F and time t. This is an
ordinary system of differential equations of order 2, since the second derivative with respect totime is taken. Typically one is interested in an initial value problem: Given shall be the place
~r(0) and the velocity ~v(0) at the starting time, sought is the place ~r(t) and velocity ~v(t) at alater time t.
Alternatively to the previous notation, the differential equation can also be interpreted as anordinary system of differential equations of 1. order with double the number of variables
~r = ~v
m~v = ~F (~r, ~v, t) .
Using the ’phase space variable’ ~x = (~r, ~v), it can be transformed to a more compact form
~x = ~F (~x, t) .
63
Here, the first three components of ~F are equal to ~v and the last three components equal to~F/m. One can see that it is sufficient to consider ordinary systems of differential equationsof 1. order. Note that in the following notation, the tilde˜ is suppressed.
8.1.1. Euler Method
Figure 8.1.: Flow plan for the Euler integration scheme.
The aim is to find the solution of the initial value problem of the differential equation system(DES)
~x = ~F (~x, t)
numerically. For this, we first replace the differential quotient by the right sided derivation
~x(t+ τ)− ~x(t)
τ+O(τ) = ~F (~x, t)
64
respectively
~x(t+ τ) = ~x(t) + ~F (~x, t)τ +O(τ 2) .The truncation error is of 2. order in the step width τ . By following notation we indicate the
discretization in time: tn = (n− 1)τ , ~xn = ~x(tn) and ~Fn = ~F (~x(tn), tn), where n = 1, 2, . . ..The method of Euler finally follows:
~xn+1 = ~xn + ~Fnτ .
This method is illustrated in figure 8.2.
Figure 8.2.: This figure shows a comparison between the true solution (continuous line) and anumerical solution found by the Euler method in one time step (dotted).
Schematically, the algorithm is as follows:
• Choose a step width τ
• Specify an initial condition ~x1
• Set the counter n to 1
• While tn ≤ tmax:
– Evaluate the right side of the DES~Fn
– Find the new places and velocities via ~xn+1 = ~xn + ~Fnτ
– Increase the counter by 1
The result is a trajectory ~x1, ~x2, ..., which approximates the (in the general case unknown)true solution x(t).
The local error is characterized as the truncation error per time step. It is of order O(τn),which gives n = 2 for the method of Euler. The global error is calculated from the time
interval of the length needed to calculate the trajectory tmax = Nτ . N is the number of timesteps. An approximation for the maximal error is
global error ∝ N × (local error)
= NO(τn) = tmaxO(τn−1) .
The global error of the method of Euler is thus of order O(τ).
65
Example: The Planar Pendulum
mg
L
x
Θ−y
Figure 8.3.: Schematic depiction of a planar pendulum.
As an example we will discuss the planar pendulum. The pendulum consists of a point-shaped
mass m, that is fixed on a massless rod of length L (see fig.8.3). The only degree of freedomof this system is the angle of deflection Θ(t) ∈ [−π, π]. The total energy is the sum of kinetic
and potential energy
E =1
2m(
x2 + y2)
+mgy
with gravitational acceleration g. From fig. 8.3, the following relations can be extracted:
x = L sinΘ
y = −L cosΘ .
Taking the time derivation while considering L = 0
x = LΘ cosΘ
y = LΘ sinΘ .
This yields
x2 + y2 = L2Θ2(
sinΘ2 + cosΘ2)
= L2Θ2 .
Insertion into the energy equation finally yields
E =1
2mL2Θ2 −mgL cosΘ .
This is the energy of the pendulum, expressed by the angle of deflection Θ and the respective
angular velocity Θ. An equation of motion for the angle can be derived from the conservationof energy
E = mL2ΘΘ +mgLΘ sin Θ!= 0 .
This gives the equation of motion for the planar pendulum
Θ = − g
LsinΘ .
66
In the approximation of small deflections, the problem can be solved analytically. With sinΘ ≈Θ, the differential equation of the harmonic oscillator follows:
Θ = − g
LΘ .
We thus choose the ansatz
Θ(t) = C1 cos (ωt+ C2)
with amplitude C1, phase C2 and angular frequency ω. Taking the second derivative of time
yields
Θ = −ω2Θ ⇒ ω =
√
g
L.
The known result is, that in the approximation of small deflections, the period of oscillationis independent of the deflection.
The general case of arbitrarily big deflections is not as easy to handle. We will solve this
problem numerically. The analysis above shows, that it is sensible to choose√
L/g as the
natural unit of time, i.e.
t = t
√
L
g.
The important point is, t is a dimensionless quantity. This choice of scaling yields
d2Θ
dt2=
d2Θ
dt2g
L= − g
Lsin Θ .
The equation of motion can thus be written with the scaled time as
Θ = − sin Θ .
This equation of motion is free from both units and parameters! Introducing the natural unitthus not only prevents range errors, but also simplifies the problem.
With the generalized speed γ = Θ, the equation of motion can be written as a DES of 1.
order
Θ = γ
γ = − sinΘ .
The following is a program that solves the equations of motion numerically via the method of
Euler.
Θn+1 = Θn + τγn
γn+1 = γn − τ sin Θn
For the sake of clarity, the program is divided into sub-programs.
Script ’pendulum.m’:
67
%pendulum - calculating the dynamics of the pendulum via the method of Euler
clear all;
% set initial conditions and constants
theta0 = input(’initial angle = ’);
x = [theta0*pi /180; 0];
t = 0;
tmax = input(’time = ’);
tau = input(’step width = ’);
% count iterations
i = 1;
while (t <= tmax)
% remember angle and time
t_plot(i) = t;
th_plot(i) = x(1)*180/ pi;
x = euler_integration(t, x, tau , ’derive_pendulum’);
t = t+tau;
i = i+1;
end
% figure
plot(t_plot , th_plot , ’+’);
xlabel(’time’); ylabel(’theta’);
function file ’euler integration.m’:
function xout = euler_integration(t, x, tau, derivs)
xout = x+tau*feval(derivs , t, x);
return;
function file ’derive pendulum.m’:
function rhs = derive_pendulum(t, s)
rhs = [s(2); -sin(s(1))];
return;
All three sub-program can be written in one file. However, for this, the main program has tobe also defined as a function.
Fig. 8.4(a) shows a solution of the equation of motion by the method of Euler with step width
68
τ = 0.1. We observe that the amplitude, and hence the energy, of the oscillation increases
strongly with time. This violates energy conservation and has to be considered an artifactof the discretization of time. The non-physical increase of energy can be slowed down by
choosing a smaller step width. This is demonstrated in fig. 8.4(b) for the case of τ = 0.05.
Unfortunately, the deviation from the true solution is still considerable, despite the small stepwidth.
Figure 8.4.: Numerical solution of the pendulum dynamics following the method of Euler. Theinitial angle is 10◦, and the initial speed is 0. The step width is (a) τ = 0.1 and(b) τ = 0.05. Note the different scaling of the y axes.
8.1.2. Runge-Kutta Method
Now we will discuss a method that is superior to that of Euler, since the error term is of higherorder. The idea is to find the changes in time (derivatives) of the variable at a sampling point
that lies at the half of the interval, over which the integration step is supposed to run. Theactual integration step is then performed with the derivative at this point, and is one order
more precise than the Euler method, as we will show in the following.
We again consider the DES~x = ~F (~x, t) .
In the following we will compare the Taylor series
~x(t+ τ) = ~x(t) + ~x(t)τ + ~x(t)τ 2
2+O(τ 3)
with the method of Euler with half the step width
~x ∗(t+τ
2) = ~x(t) + ~F (~x, t)
τ
2.
It can be seen that
~F(
~x ∗(t+τ
2), t+
τ
2
)
= ~F (~x, t) +∂ ~F
∂~x
∣
∣
∣
(~x,t)~F (~x, t)
τ
2+
∂ ~F
∂t
∣
∣
∣
(~x,t)
τ
2+O(τ 2)
69
= ~x(t) +
∂ ~F
∂~x
∣
∣
∣
(~x,t)~x(t) +
∂ ~F
∂t
∣
∣
∣
(~x,t)
τ
2+O(τ 2)
= ~x(t) + ~x(t)τ
2+O(τ 2) .
This yields the Runge-Kutta method of 2. order
~x(t+ τ) = ~x(t) + ~F(
~x ∗(t +τ
2), t+
τ
2
)
τ +O(τ 3)
~x ∗(t +τ
2) = ~x(t) + ~F (~x, t)
τ
2.
The truncation error is of 3. order in τ , i.e. one order higher as compared to the method of
Euler. The calculation method is illustrated in fig. 8.5.
x
t τ
xn+1
xn
x*
n tt + /2n n+1 t
Figure 8.5.: Scheme of the Runge-Kutta method, 2. order.
To better understand this method, we investigate a very simple differential equation
x = x .
With the initial condition x(0) = 1, the solution is
x(t) = et .
Let us now consider a translation τ in time. We make use of the rules of powers and construct
the Taylor series up to the fifth order
x(t+ τ) = et+τ = et eτ
x(t+ τ) = x(t)
(
1 + τ +τ 2
2+
τ 3
6+
τ 4
24+O(τ 5)
)
.
Comparing this with the result obtained by the method of Euler
xn+1 = xn + τxn = xn(1 + τ)
70
shows that the method of Euler gives the exact solution up to the first order. Now we look at
the Runge-Kutta method of 2. order, which is
xn+1 = xn + τx∗
x∗ = xn +τ
2xn .
Inserting the lower equation in the upper one yields
xn+1 = xn + τxn +τ 2
2xn
xn+1 = xn(1 + τ +τ 2
2) .
Thus we see that the Runge-Kutta method of 2. order represents the exact solution up to the2. order.
8.1.3. Runge-Kutta Method of 4. Order
In complete analogy, the Runge-Kutta method of 4. order can be derived. We will thus skipthis procedure and directly look at the result:
~x(t + τ) = ~x(t) + (~F1 + 2~F2 + 2~F3 + ~F4)τ
6+O(τ 5)
~F1 = ~F (~x, t)
~F2 = ~F (~x+τ
2~F1, t+
τ
2)
~F3 = ~F (~x+τ
2~F2, t+
τ
2)
~F4 = ~F (~x+ τ ~F3, t+ τ)
The truncation error is of 5. order in the step width τ . Let us first consider again the simpledifferential equation x = x. The auxiliary quantities Fi can be found effortlessly,
F1 = x
F2 = x+τ
2F1 = x+
τ
2x
F3 = x+τ
2F2 = x+
τ
2x+
τ 2
4x
F4 = x+ τF3 = x+ τx +τ 2
2x+
τ 3
4x .
71
Inserting and rearranging yields
x(t+ τ) = x(t)
(
1 + τ +τ 2
2+
τ 3
6+
τ 4
24
)
.
We can see that the Runge-Kutta method of 4. order is the exact solution up to the 4. orderof the series. We thus assume that this method needs far less time steps than the method of
Euler. Fig. 8.6(a) confirms this. The result for the planar pendulum with a small deflection
angle of 10◦ solved by the Runge-Kutta method of 4. order is plotted. The case of bigdeflection angles (here 170◦) is shown in fig. 8.6(b). From the comparison with fig. 8.6(a) we
can deduct that the period of oscillation depends on the deflection of the pendulum.
Figure 8.6.: Numerical solution of the dynamics of the planar pendulum via Runge-Kutta, 4.order. the step width is τ = 0.2. The initial angle is (a) 10◦ and (b) 170◦.
8.1.4. Adaptive Methods
Thinking about the previously introduced methods, two questions arise:
• Given a DES, how should the step width τ be chosen?
• Can the step width τ be chose dynamically, when the variables of the DES show less
or more temporal variations? Such a case is given for example for the pendulum with alarge deflection angle near 180◦ (see the following figure). With a flexible step width,
the calculation could be accelerated a lot.
Both questions lead toward the so called adaptive methods, where the step width τ is adjustedin every time time step. The procedure is the following: Find ~x(t + τ) first with a ’big’ time
step τ . This gives a rough approximation ~xg. Then calculate the far better approximation ~xk
with two successive ’small’ time steps τ/2.
With these two approximations, a relative error ∆ can be approximated,
∆ =|~xk − ~xg||~xk|+ ε
,
72
Runge-Kutta method of 4. order: pendulum with initial angle = 179.5◦ and τ = 0.2.
with the machine precision ε. We want that this error is approximately equal to a given value∆0. To achieve this, we recall that the local error of the Runge-Kutta method of 4. order is
proportional to τ 5. Thus, ∆ is proportional to τ 5. The step width has to be adjusted suchthat ∆0 ∝ τ 50 . It follows for the approximated step width:
τ0 = S1τ(
∆0
∆
)1/5
.
Here, S1 < 1 is a ’security factor’, which assures that the approximated step width is not to
big. A second ’security factor’ S2 > 1 is introduced to prevent too large steps τ → τnew,
τnew =
{ τ0 if τ/S2 < τ0 < S2τS2τ if τ0 > S2ττ/S2 if τ0 < τ/S2
Feasible values are S1 = 0.9 and S2 = 1.25. By the way: As the new initial condition for thenext time step ~xk should be used, since this value is more accurate than ~xg.
8.1.5. Matlab Solver
Matlab provides several solvers for DES. In the simplest case, these are invoked by
[t,y] = solver(function, time interval, initial values);
The objects in this command are:
73
• t a column vector of length n (adaptively found by the solver) holding the times of the
output points.
• y a n×m matrix with the m coordinates of the solution vector y(:,i), i = 1, . . . , m.
• solver, z.B.
– ode23 Runge-Kutta Method (2,3) order. The abbreviation ’ode’ stems from’ordinary differential equation’.
– ode45 Runge-Kutta method (4,5) order. First choose, if no information about theproperties of the system is known.
– ode113 a solver of variable order. Ideal, if high accuracy is needed or if the
calculation of the right side of the DES is very intricate.
– ode15s a solver of variable order for stiff DES. DES are called stiff when the indi-
vidual components vary on highly different time scales. The terminology probablyoriginates from the investigation of spring pendula with very stiff springs, i.e. high
spring constants.
• function, In the simplest form, function has the form function dx = function name(t,x).Both arguments dx and x are column vectors, the time t is a scalar. The function value
dx has to give the derivative of the m differential equations.
• time interval, A row or column vector, e.g [t0,t1] or [t0:step width:t1]
• initial values, A row or column vector of length m.
Calling the solver and the graphical representation of the solution can than be accomplished
for example by
[t,y] = ode45(’derivspendulum’,[0,30],[10/180*pi,0]); plot(t,y(:,1)/pi*180);
74
9. Data Analysis
Three aspects of data analysis shall be covered in this chapter: spectral analysis via the Fourier
transformation, generation of random numbers and working with probability distributions, andthe elementary methods to fit models to measured data.
9.1. Fourier Analysis and Filter Operations
The Fourier analysis is the most important integral transformation in physics. It not only
allows to find the frequency content of a temporally varying signal, it also comes in handy tosimplify the calculation of convolutions and thus applying filters efficiently to data. In every
programming language, thus also in Matlab, the Fourier transformation is implemented as the
so called ’Fast Fourier Transform’ (FFT) to optimize calculation time and to spare the userthe effort to program it themselves.
9.1.1. Fourier Transformation
It has to be distinguished between the discrete Fourier transformation and the continuousFourier integral. First, we will define the Fourier integral and then introduce the discrete
version. Although the computer, due to the discretization of any function x(t), always has touse the discrete Fourier transformation, we will use the integral notation to better understand
some basic properties.
Fourier Integral
The Fourier integral can be defined as follows:
x(ω) =1
2π
∫ +∞
−∞x(t) exp (−iωt) dt (9.1)
Attention: in the literature sometimes a pre-factor of 1/√2π is used. The factor in our
definition is chosen such that it is compatible with the normalization of the Matlab-FFT which
is described later on. The Fourier transformation is reversible; the correspondent reversetransformation is
x(t) =∫ +∞
−∞x(ω) exp (iωt) dω . (9.2)
75
If x(t) is a real valued function, the transform x(ω) is usually complex. For the existence of the
Fourier transformation it is sufficient that x(t) is absolutely integrable, i.e.∫+∞−∞ |x(t)|dt < ∞.
The Fourier transformation of a function is a representation of that function as a superpositionof harmonic oscillations with angular frequency ω = 2πf and coefficients x(ω). It is thus
nicely suited for frequency analysis of temporal signals. The complex coefficients x containinformation about phase φ and amplitude A of the oscillation, and yield by using Euler’s
identity:
x = ℜ(x) + iℑ(x) (9.3)
= A exp(iφ) (9.4)
= A cos(φ) + iA sin(φ) (9.5)
Here, i is the imaginary unit and ℜ and ℑ denote the real respectively the imaginary part of
the complex number.
If x(t) is periodic, for example in 2π, or if x(t) is only defined in the interval [0, 2π], this can
be expressed as the Fourier series with coefficients xk:
xk =1
2π
∫ 2π
0x(t) exp (−ikt) dt . (9.6)
The reverse transformation is written as an infinite sum:
x(t) =+∞∑
k=−∞x(t) exp (ikt) . (9.7)
By rescaling of the t-axis by t′ = 2πt/T , a function a(t) defined on an interval [0, T ] can
always be mapped on a function x(t′) in the interval [0, 2π], and thus can be transformed byequation (9.6).
For further properties of the Fourier transformation you may consult the instructive Wikipediaarticle.
Discrete Fourier Transformation
In the computer, functions are defined on discrete sampling points in a finite interval. Let
us assume a function a(t) is sampled at N equidistant points tn = Tn/N , and the valuesat these points are an = a(tn). By the transformation t′ = (2π/T )t we transfer a(t) to a
function x(t′) in the interval [0, 2π] and approximate the integral from equation (9.6) with the
midpoint rule (see chapter Integration an Differentiation):
xk =1
2π
∫ 2π
0x(t′) exp (−ikt′) dt′ ≈ 1
2π
N−1∑
n=0
an exp (−ik2πtn/T )∆t′ . (9.8)
Here, ∆t′ is given by (2π/T )(T/N) = 2π/N , which means that
xk ≈ Ak =1
N
N−1∑
n=0
an exp (−i2πnk/N) . (9.9)
76
This equation describes the discrete Fourier transformation, the implementation of which we
will discuss more extensively in the following paragraph. The corresponding reverse transfor-mation is:
an =N−1∑
k=0
Ak exp (i2πnk/N) . (9.10)
Fast Fourier Transform (FFT)
The discrete Fourier transformation is invoked in Matlab by the command a trans = fft(a);,where a is a function or time series. The fast Fourier transform is an algorithm that is optimized
for a fast execution of the transformation. It is based on the idea to first divide a functionof N sampling points into two function of N/2 sampling points, and then to compute two
Fourier transformations on the partial functions and combining the two results to the soughttransformation of the full function. This procedure can be iteratively extended: further partial
function then have N/4, N/8, . . . sampling points. The reverse transformation, the inverse
Fourier transformation, is invoked by the command ifft.
Calling help fft shows that the Matlab-implementation of the FFT does unfortunately not
divide the result by the number of sampling points, which is the number of element of a vectora. This has to be done ’manually’.
The performance gain of the FFT in comparison to a non-optimized, brute force algorithm isimpressive: the execution time of the FFT is already faster by 4 orders of magnitude(!) for
N = 216 sampling points:
n_power = 12;
t = zeros([ n_power 2]);
for power=1: n_power
fprintf(’N=2^%i...\n’, power);
n = 2^power;
a = rand([n 1]);
%%% Calculates FFT of signal ’a’
tic;
a_fast = fft(a);
t(power , 1) = toc;
%%% Calculate Fourier transform by ’brute force ’
tic;
a_slow = zeros([n 1]);
a_slow (1) = sum(a);
exp_k1 = exp(-complex (0, 1)*2*pi *(0:(n-1))’/n);
exp_k = 1;
for k=1:n-1
77
exp_k = exp_k.*exp_k1;
a_slow(k+1) = sum(a.*exp_k);
end
t(power , 2) = toc;
end
%%% Plot execution times and the
%%% acceleration factor
figure (1);
subplot (2, 1, 1);
plot(1:n_power , t);
legend({’FFT’ ’Fourier brute force’});
xlabel(’power of two’); ylabel(’t [s]’);
subplot (2, 1, 2);
plot(1:n_power , t(:, 2)./t(:, 1));
xlabel(’power of two’); ylabel(’factor’);
Application: The Power Spectrum
This paragraph deals with the question of how the FFT is applied to a signal a(t) , and theinterpretation of the result. For this we consider a sine wave with frequency f and a step
function as possible signals.
n = 100; dt = 0.01; f = 5;
a sine = 42*sin((0:(n-1))*dt*2*pi*f);
a step = (0:(n-1)) >= n/2;
The signals are sampled with a frequency of 1/dt for T = 1s, i.e. the discretization of the
functions has the step width dt. The FFT is calculated by the following:
fft a sine = 1/n*fft(a sine);
fft a step = 1/n*fft(a step);
Now we want to investigate how much energy is contained in the different frequency bands.
For this we calculate the so called power spectrum Φ(f), i.e the square of the sum of oscillation
amplitudes for a specific frequency f . Here one has to be aware of two possible stumblingblocks:
• In the first place we have to make sure, how Matlab stores the calculated coefficients Ak
in the result vector: The first entry, e.g. a sine(1) contains A0, i.e. the mean of the
function a(t). The k-th entry then holds the coefficient Ak−1, for example a sine(17)
would be A16.
• Which frequency does the coefficient Ak correspond to? Ak corresponds to the harmonic
oscillation, for which exactly k periods fit into the time interval of length T . Thefrequency is thus fk = k/T .
• Let us now look at the coefficients Ak and AN−k. From equation (9.9) we can see
78
Figure 9.1.: Power spectrum of a step function.
that exp(−i2π(N −k)n/N)) = exp(i2πkn/N) exp(−i2πn) = exp(i2πkn/N). Hence,
AN−k is equivalent to the coefficient A−k with negative wave number −k and thus
corresponds to the same frequency than Ak.
If we want to know the amplitude of the oscillation with frequency f = 5Hz, we have to lookat the coefficient with the wave number k = Tfk = 5 and k = −5. The correspondent value
Φ(f) is found in Matlab-notation as:
phi 5Hz = (abs(fft a sine(1+5))+abs(fft a sine(n-5)))
The output 1.7640e+03 is indeed the square of the amplitude 42.
A signal with n samples can maximally represent oscillations with n/2 periods. Oscillations of
higher frequencies are reduced by sampling to oscillations of lower frequencies with negative
wave number. This insight is known as the Nyquist-Theorem. When calculating the powerspectrum, one can thus ignore all coefficients from k = n/2 upwards and double all amplitudes
of the low frequency components (except for A0, |Ak| = |A−k| holds). A corresponding Matlabcommand plots the power spectrum for a step function, which has, as expected, a broad band
spectrum:
plot(0:(n/2), [fft a step(1) 2*abs(fft a step(2:(n/2)+1))]);
xlabel(’wave number k’); ylabel(’power’);
79
9.1.2. The Convolution Theorem
The fast Fourier transform is not only useful to determine the frequency content of temporal
signals, but also to perform so called ’convolutions’. A convolution of two functions f(t) andg(t) is defined by the following equation:
h(t) =∫ +∞
−∞f(t′)g(t− t′) dt′ (9.11)
Here, g is often referred to as convolution kernel. The procedure of a convolution can bevisualized as follows: We interpret f as a arbitrary function, that is compared to a template
function g. To account for possible shifts on the time axis, the template is shifted betweent = +∞ and t = −∞ and the comparison is made at every position t. If both functions
match perfectly, the result h(t) is big. If the shifted g does not match with f , the result issmall.
Applications of this equation are numerous in physics, and reach from simple filter operations
to timely resolved frequency analysis (examples will be discussed later). First, we want tounderstand the connection between convolutions and the Fourier transformation. As a short
notation of the application of the Fourier transform F to a function f (resp. its reversetransformation F−1) we introduce:
f(k) = F [f(t)]( k) (9.12)
f(t) = F−1[
f(k)](
t)
(9.13)
Now we apply the Fourier transform to both the left and the right side of the Definition (9.11)
and gain after short computation,
h(k) = 2πf(k)g(k) , (9.14)
or, in short notation, h = 2πF [f ]F [g]. To get the sought result h(t), we apply the inverse
Fourier transform to the equation, which results in
h = F−1 [2πF [f ] · F [g]] . (9.15)
The advantage of equation (9.15) over (9.11) lies in the computation speed of FFT: despitea three-fold transformation, calculating equation (9.15) is faster than computing the integral
in (9.11), since the convolution integral corresponds to an element-wise multiplication of the
Fourier coefficients in the Fourier space k – try to verify!
Non-periodic Functions
The discrete Fourier transformation can be applied to any arbitrary function f , periodicity isnot required. The convolution theorem, on the other hand, does strictly speaking only apply
for the Fourier integral. It thus demands either a definition of the function in [−∞,+∞], or aperiodic function on a finite interval. Only in the latter case, fft/ifft can be directly used,
i.e. without ’post-treatment’, to evaluate a convolution integral.
80
Still, the FFT can be used with non-periodic functions, provided the convolution kernel g is
sufficiently narrow, i.e. its absolute value almost vanishes within m << n sampling pointsaround t = 0. This is for example given for exponentially decaying functions or Gaussian
distributions. Unwanted boundary effects can be suppressed by simply discarding the first and
lastm elements of the result vector. Another possibility is the continuation of a function aboveits borders by concatenating a vector of k zeros. The command is fft(f, n+k), where n is
the length of the vector f . The extension of f by zeros is only sensible in specific situations(see also remarks further down), for example if signal was 0 before and after a period of data
acquisition, or when a filter is successively ’inserted’ into a beginning signal.
Application: Filtering of a Time Series
Let us consider the filtering of a temporal signal as an example for the application of the
convolution theorem. The signal is chosen to be a rectangle function, that is 1 in the firstthird of the data acquisition period, then jumps to 0 and increases to 2 in the last third. The
filter shall be an exponentially decaying function exp(−t/τ), which realizes a low pass withtime constant τ . The following program generates the signal and the filter, applies the filter
operation and displays the result. Border effects caused by the assumed periodicity of thesignals are clearly discernible.
% parameter
n = 1000;
t = (0:(n-1))/n;
tau = 0.05;
% rectangle signal
f_signal = (t < 0.33)+2*(t > 0.66);
g_filter = 1/tau*exp(-t/tau);
f_filtered = real(ifft(1/n*fft(f_signal ).*fft(g_filter )));
% plot
figure (1);
subplot (3, 1, 1);
plot(t, f_signal ); xlabel(’t’); title(’Signal f(t)’);
subplot (3, 1, 2);
plot(t, g_filter ); xlabel(’t’); title(’Filter g(t)’);
subplot (3, 1, 3);
plot(t, f_filtered); xlabel(’t’); title(’Filtered Signal h(t)’);
Equation 9.15 reveals a further, interesting possibility to apply filter operations: The term
F [g], the transform of the filter kernel, acts through the multiplication with F [f ] similarto a ’switch’, that only allows passage of ’permitted’ frequencies and suppresses ’unwanted’
frequencies. This properties can be used to directly design filters with the sought propertiesin the frequency space. As an example, we could set all frequency components of a signal to
0 below a threshold frequency. This transformation realizes a high pass on f :
81
0 0.2 0.4 0.6 0.8 10
1
2
t
Signal f(t)
0 0.2 0.4 0.6 0.8 10
10
20
t
Filter g(t)
0 0.2 0.4 0.6 0.8 10
2
4
t
Filtered Signal h(t)
Figure 9.2.: Low pass filtering
% construct high pass filter
k_min = 5;
g_filter_highpass = zeros([1 n]);
g_filter_highpass(1+k_min:end -(k_min -1)) = 1;
f_filtered_highpass = real(ifft(1/n*fft(f_signal ).* g_filter_highpass));
% plot
figure (2);
plot(t, f_filtered_highpass); xlabel(’t’);
title(’Highpass -Filtered Signal h(t)’);
When defining own filters in frequency space, one has to be cautious: most of these filter are
acausal and change the phase of f also in the not suppressed frequency bands.
To conclude, two technical remarks shall be made. First, the FFT is also defined for higherdimensional functions. The Matlab-commands are (i)fft2, (i)fft3 and (i)fftn. This
is useful for example to filter digital images. Second, the FFT can be applied to a specificdimension n of an N -dimensional array. The syntax is fft(a, [], n), where the empty
brackets indicate that a ’zero-padding’, as described earlier, shall not be made. This form ofthe FFT is worth considering, when several functions, that are stored in the columns or rows
of a matrix, shall be transformed in one go.
82
0 0.2 0.4 0.6 0.8 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1x 10
−3
t
Highpass−Filtered Signal h(t)
Figure 9.3.: high pass filtering
−4 −2 0 2 4−1
−0.5
0
0.5
1
t [s]
g(t)
Figure 9.4.: Wavelet filter g(t) for the frequency analysis of non-stationary signals
Application: Examples for Convolution Operations
To end this section, some further examples for the application of the convolution theorem inphysics and data analysis shall be mentioned:
• Wavelet Transformations. Fourier transformations are ill suited for signals, whose spec-
trum changes over time. This also includes signals which do contain periodic oscillations,but where the frequency is subject to fluctuations.
In wavelet transformations, a signals is convoluted with a filter g, that only contains fewoscillations of a specific period and decays to 0 outside of this range. An example is
depicted in the following picture:
83
A spectrum is obtained that not only depends on the frequency, but also from time. A
much used wavelet is the Morlet-wavelet, which is a harmonic oscillation multiplied witha Gaussian curve.
• Correlation Functions. The cross correlation C(τ) between signals f(t) and g(t) is
defined by the following equation:
C(τ) =∫
f(t)g(t+ τ)dt (9.16)
Here, τ denotes the delay between the ’leading’ signal f and the for τ > 0 ’lagging’
signal g. For f(t) = g(t), C is referred to as the auto-correlation.
Equation (9.16) is not directly a convolution, this means one has to be careful when
applying the convolution theorem. Utilizing that F [f(−t)](k) = f(−k) it holds:
C(τ) = F−1 [ 2πF [f(t)](k)F [g(t)](−k)](− τ) (9.17)
The following examples come from the theoretical neurophysics:
• Reverse Correlation. A simple model for the response properties of neurons in the
visual system assumes that the response r(t) of a neuron is constructed by a linearsuperposition of a stimulus s(x, y, t) at the location (x, y) on the retina, multiplied with
a weight function w(x, y, τ). τ is the delay between stimulus and neuronal response,and g[ ] an additional point-non-linearity:
r(t) ∝ g[∫
xdx∫
ydy∫
τdτw(x, y, τ)s(x, y, t− τ)
]
(9.18)
Again, w can be interpreted as a filter. With this insight, the inner integral over τ isbest numerically solved via the convolution theorem.
When the stimulus s(x, y, t) and the response r(t) is known, the unknown filter w can bedetermined under special conditions. This becomes specifically easy, when the stimulus
is uncorrelated white noise:
w(x, y, τ) ∝∫
r(t)s(x, y, t− τ)dt (9.19)
After application of the convolution theorem, we get:
w(x, y, τ) = F−1 [ 2πF [r(t)](k)F [s(x, y, t)](−k)]( τ) (9.20)
• Recurrent Networks. Neuronal networks usually have a one- or two-dimensional topol-ogy, where a neuron at position x is coupled to a neuron at position x′ by a weight
of magnitude w(x, x′). The dynamics of such a network are usually described as adifferential equation for the neuronal activities A(x, t) in the following form:
τA(x, t) = −A(x, t) + g[I(x, t)] (9.21)
84
I(x, t) denotes the incoming current of the neuron at position x; this is a sum across
the weighted activities of all other neurons:
I(x, t) =∫
w(x, x′)A(x′, t)dx′ (9.22)
If the weight are invariant to translation via w(x, x′) = w(x− x′), the solution of thisintegral is again a case for the well known convolution theorem.
9.2. Probabilities and Distribution Functions
9.2.1. Distributions
Often the problem arises to estimate an underlying distribution ρ(x) from a sample {xi} ofcardinality N (i = 1, . . . , N). The simplest possibility is to create a histogram of the values
xi. For this, a range of the x-axis between [xmin, xmax] is divided into M intervals Ij ofsize ∆x = (xmax − xmin)/M . An approximation of ρ is now given by finding the number of
elements xi that fall within a specific interval Ij = [xmin + (j − 1)∆x, xmin + j∆x[:
hj =∫ xmin+j∆x
xmin+(j−1)∆x
N∑
i
δ(x− xi)dx (9.23)
It thus holds ρ(x) = hj/(N∆x) for x in Ij . For a good approximation, N >> M should begiven.
Matlab provide the function histc, that calculates distributions from samples. The syntax is:
h = histc(x samples, x min:Delta x:x max{, dim});The vector h contains the number of samples that lie in the intervals determined by the second
argument (the intervals do not have to be equidistant!).
Histograms can also be calculated from multi-dimensional distributions of ~x. However, one
needs an extremely large number of samples, or the approximated distribution has to be
smoothed to obtain a sufficiently good estimation. A different problem pose distributionsthat unevenly stretch out across big intervals of x – here one should think about dynamically
adapting the binning width ∆x.
9.2.2. Random Numbers
Creating random numbers is a often required function of a programming language. There are
numerous algorithms to create numbers that are distributed ’as random as possible’, these willhowever not be covered here. Instead we will concern ourselves with creating random numbers
from arbitrary given distributions.
85
Uniformly Distributed Random Numbers
For uniformly distributed random numbers, Matlab provides the command rand. It creates
random numbers in the interval [0, 1[ and can be used in different ways:
a = rand;
b = rand([n 1]);
c = rand(n);
d = rand(size(array));
The first command gives exactly one random number, the second one a vector of n randomnumbers. Attention: the third command directly gives a n x n matrix or random numbers.
The fourth variant gives an array of random numbers with the same size of the array array.An example of the usage see the following paragraph.
It might be a good idea to generate the same sequence of random numbers when restarting
a program. In this way, faulty code can be tested more effectively or different numericalprocedures can be compared with the same realization of a random process. To accomplish
this in Matlab, the ’seed’, i.e. the starting value of an internal random number generator, canbe set to a fixed value by
rand(’state’, start value);
Normally Distributed Random Numbers
Normally distributed random numbers are created by the command randn. The syntax is
analogue to rand. The distribution has the mean 0 and the variance 1.
To generate other uniform or normal distributions, it is best to scale and move the numbers
generated by rand and randn. The following Matlab code for example draws samples from auniform distribution in the interval [2, 4[ and a normal distribution with mean 3 and standard
deviation 0.5, and plots the corresponding distributions via histc:
86
%%% number of samples
n_samples = 12345;
%%% draw random numbers and rescale
x_uniform = 2+2*rand([ n_samples 1]);
x_normal = 3+0.5* randn([ n_samples 1]);
%%% estimate distributions
x_axis = 1.5:0.05:4.5;
rho_uniform = histc(x_uniform , x_axis)/ n_samples/0.05;
rho_normal = histc(x_normal , x_axis)/ n_samples/0.05;
%%% plot exact and estimated distriutions
figure (1);
subplot (2, 1, 1);
bar(x_axis , rho_uniform);
hold on;
plot(x_axis , (x_axis >= 2).*(x_axis <= 4)/2, ’r-’);
hold off;
xlabel(’x’); ylabel(’\rho_{uniform }(x)’);
subplot (2, 1, 2);
bar(x_axis , rho_normal);
hold on;
plot(x_axis , 1/sqrt(2*pi )/0.5*exp( -0.5*( x_axis -3).^2/0.5^2) , ’r-’);
hold off;
xlabel(’x’); ylabel(’\rho_{normal}(x)’);
Random Numbers from Arbitrary Distributions
To generate random numbers from arbitrary distributions ρ(x), we will schematically introduce3 procedures:
The Hit-and-Miss Method. Let ρmax be the maximum of the distribution ρ. To generaterandom numbers from ρ(x), we have to make sure that the value x occurs with the relative
frequency ρ(x)/ρmax. This can be assured by a simple procedure that uses the random number
generator for uniformly distributed random numbers: we first draw a candidate x1 for a randomnumber from the area of definition of ρ(x). Then we draw a second number x2 from [0, 1[.
If this one is smaller than ρ(x1)/ρmax, the first number is accepted, otherwise the procedureis repeated. See also the following picture: only pairs of numbers (x1, x2) are accepted, that
fall into the area under the renormalized distribution ρ.
The hit-and-miss method can be very time consuming. It is easier if the inverse function of
the primitive F of ρ can be calculated. The idea is to start out with a uniformly distributed
87
1 1.5 2 2.5 3 3.5 4 4.5 50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x
ρ unifo
rm(x
)
1 1.5 2 2.5 3 3.5 4 4.5 50
0.2
0.4
0.6
0.8
1
x
ρ norm
al(x
)
Figure 9.5.: Uniform and normal distribution.
random number y and find a transformation x = g(y) such that the probabilities ρu(y) dy andρ(x) dx are equal. With ρu(y) = 1 one can derive
y = F (x) =∫ x
−∞ρ(x′) dx′ , (9.24)
and thus x = g(y) = F−1(y). Is F−1 known, one simply draws the uniformly distributed
random numbers y and calculates the sought random numbers x from the inverse function. IfF−1 can not be found analytically, there are two possibilities:
Interval Nesting. The primitive F can be numerically approximated as the integral of ρ withone of the numerical methods introduced in an earlier chapter for an arbitrary x. By nesting
of intervals, the value of x, that corresponds to a specific random number y, can now beconfined. This procedure is not particularly fast, but can yield a good precision.
Tabulation. If many random number have to be drawn, the inverse function F−1(y) can first
be tabulated. For this one is advised to use equidistant sampling points yj. In order to thendraw a random number x from ρ(x), again first a random number y1 is taken from a uniform
distribution, and then the neighboring values yj and yj+1 are found, for which yj ≤ y1 < yj+1.With another random number y2 from a uniform distribution in the interval [0, 1[ the random
number x is:
x = y2(F−1(yj+1)− F−1(yj)) + F−1(yj) . (9.25)
88
0 2 4 6 8 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x1
x 2 (ρ/
ρ max
)
Figure 9.6.: Hit-and-miss method: draw two uniformly distributed random numbers x1 andx2, and accept x1 if both numbers fall into the blue area. By this, the probabilitydistribution marked by the red line.
9.3. Regression Analysis
Analysis of measured data is an important part in evaluating physical theories and models.Usually, some parameters of the physical models have to be fitted to the data and then
the adapted model shall be scrutinized whether it describes the data well. The mandatorytechniques of parameter fitting as well as the evaluation of the goodness of a fit will be
discussed in the following.
9.3.1. General Theory
Let us assume that data is provided as pairs of values (xi, yi), i = 1, . . . , N . The model
that is to be fitted predicts a functional correlation of the form Y (x; a1, . . . , aM) between thequantities x and y; a1, . . . , aM are here the parameters that have to be found. Of course there
should be much more data points than parameters, mathematically speaking: N ≫ M . Oneexample, that is commonly encountered in the practical courses, is the fitting of a straight line
Y (x; a1, a2) = a1 + a2x
to a data set.
It is the aim to adjust the values of these parameters such that the function Y (x; a1, . . . , aM)
represents the date as well as possible. Intuitively we would assume that in case a fit is good,the graph of the function Y (x; a1, . . . , aM) will lie close to the points (xi, yi) (see figure 9.7).
We can quantify this statement by measuring the deviations between the data points and thefunction
∆i = Y (xi; a1, . . . , aM)− yi .
89
x1 x2 x3 x4
y4
y3
y1y2
x
y
Y(x)∆ i
σ i
Figure 9.7.: Fitting a curve to data points.
We choose our fitting criterion such that the sum of the squares of the deviations becomes
minimal. This means that we have to tune our parameter values a1, . . . , aM such that the’cost function’
χ2(a1, . . . , aM) =N∑
i=1
∆2i =
N∑
i=1
[Y (xi; a1, . . . , aM)− yi]2
is minimized. This method is dubbed least squares method. It is not the only possible method,
but the one that is most commonly used.
Usually the data points have an estimated error bar (the confidence interval), that we denoteas yi ± σi. In this case we should change our fitting criterion such that points with bigger
error bars have a smaller weight. Following this logic we define
χ2(a1, . . . , aM) =N∑
i=1
(
∆i
σi
)2
=N∑
i=1
[Y (xi; a1, . . . , aM)− yi]2
σ2i
. (9.26)
9.3.2. Linear Regression
We now want to fit a straight line to the data points,
Y (x; a1, a2) = a1 + a2x .
This type of fitting is called linear regression. The two parameters a1 and a2 have to be chosensuch that
χ2(a1, a2) =N∑
i=1
[a1 + a2xi − yi]2
σ2i
becomes minimal. The minimum can be found by taking the derivation of the equation andsetting it to zero
∂χ2
∂a1= 2
N∑
i=1
a1 + a2xi − yiσ2i
= 0
∂χ2
∂a2= 2
N∑
i=1
xia1 + a2xi − yi
σ2i
= 0 .
90
We now introduce the following quantities
Σ =N∑
i=1
1
σ2i
, Σx =N∑
i=1
xi
σ2i
, Σy =N∑
i=1
yiσ2i
,
Σx2 =N∑
i=1
x2i
σ2i
, Σxy =N∑
i=1
xiyiσ2i
.
These sums are calculated directly from the data points, are thus known constants. Hence wecan rewrite the previous system of equations as
a1Σ + a2Σx − Σy = 0
a1Σx + a2Σx2 − Σxy = 0 .
This is a system of linear equations with two unknowns a1 and a2. The solutions are
a1 =ΣyΣx2 − ΣxΣxy
ΣΣx2 − (Σx)2, a2 =
ΣΣxy − ΣyΣx
ΣΣx2 − (Σx)2. (9.27)
In a second step we want to estimate the error bars for the parameters a1 and a2. We use the
law of error propagation
σ2aj
=N∑
i=1
(
∂aj∂yi
)2
σ2i .
Insertion of the equations (9.27) yields
σa1 =
√
Σx2
ΣΣx2 − (Σx)2, σa2 =
√
Σ
ΣΣx2 − (Σx)2.
As an example we consider fig. 9.8. Here the fit parameters are a1 = 0.1529 ± 0.2633 and
a2 = 1.0939±0.0670. Note that the error bars σaj do not depend on the yi. These error bars
are thus no quantifier of the goodness of the fit.
9.3.3. Goodness of the Fit
It is clear that the fit is locally good, if the deviation of the function is smaller or approximatelyequal to the error bar. We consider the upper boundary
|yi − Y (xi)| ≈ σi .
Insertion into the definition of χ2 in equation (9.26) yields χ2 ≈ N . The more parameters are
used, the better the fit will be. For the case of N = M , the fit will be exact. For the case ofthe straight line, this simply means that to N = 2 points, we can always exactly fit a straight
line (M = 2). Thus, a good criterion for the goodness of the fit is
χ2 ≈ N −M .
As an example we again refer to fig. 9.8. Here, χ2 ≈ 4.5 and N −M = 8. The goodness of
the fit is thus quite high.
91
0 2 4 6 8 100
2
4
6
8
10
12
xi
y i, Y(x
)
Figure 9.8.: An example of linear regression. A straight line Y (x) is fitted to the data points(xi, yi) .
9.3.4. Non-linear Regression
In many cases, the fitting of a non-linear function can be broken down through a clever variabletransformation to the fitting of a linear function. As the first example, the commonly occurring
caseZ(x;α, β) = αeβx
shall be observed. One thinks for example of exponential decays. With the variable transfor-
mationY = lnZ, a1 = lnα, a2 = β
we get the linear functionY (x; a1, a2) = a1 + a2x .
A second example are power laws of the type
Z(t;α, β) = αtβ .
the variable transformation
Y = lnZ, x = ln t, a1 = lnα, a2 = β
also gives a linear function.
92