introduction to scientific computing · moving sofa problem ... bash c# perl compiled languages...
TRANSCRIPT
Introduction to Scientific Computing
Benson Muite
[email protected]://kodu.ut.ee/˜benson
https://courses.cs.ut.ee/2018/isc/spring
12 February 2018
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 1 / 26
Course Aims
General introduction to numerical and computationalmathematicsReview programming methodsLearn about some numerical algorithmsUnderstand how these methods are used in real worldsituations
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 2 / 26
Course Overview
Lectures Monday J. Livii 2-224 10.15-11.45 Benson Muite(benson dot muite at ut dot ee)Practical Monday J. Livii 2-205 12.15-13.45 Benson Muite(benson dot muite at ut dot ee)Practical To be decided Gul Wali Shah (gulwali at ut dot ee)Homework typically due once a week until project.Expected to start this in the labs.Exam/final project presentation will be scheduled at end ofcourse.Grading: Homework 50%, Exam 30%, Active participation10%, Tests 10%Course Texts:
Solomon “Numerical Algorithms: Methods for ComputerVision, Machine Learning, and Graphics”Pitt-Francis and Whiteley “Guide to scientific computing inC++”
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 3 / 26
Lecture Topics
1) 12 February: Graphing, differentiation and integrationdimension
2) 19 February: programming, recursion, arithmeticoperations
3) 26 February: Linear algebra4) 5 March: Floating point numbers, errors and ordinary
differential equations5) 12 March: Image analysis using statistics6) 19 March: Image analysis using differential equations -
Lecture by Gul Wali Shah7) 26 March: Case study: Application of machine learning to
analyse literary corpora8) 2 April: Case study: DNA simulation using molecular
dynamics
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 4 / 26
Lab Topics
1) 12 February: Mathematical functions, differentiation andintegration, plotting, reading:https://doi.org/10.1080/10586458.2017.1279092
2) 19 February: Sequences, Series summation, convergenceand divergence, Fibonacci numbers and Collatz conjectureor similar experimental mathematics
3) 26 February: Monte Carlo integration, parallel computingintroduction, Matrix Multiply, LU decomposition
4) 5 March: Finite difference method, error analysis, intervalanalysis, image segmentation by matrix differencesReading https://doi.org/10.1080/10586458.2016.1270858
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 5 / 26
Lab Topics
5) 12 March: Eigenvalue computations, singular valuedecomposition, use of matrix operations in statistics -Eigenfaces
6) 19 March: fixed point iteration, image segmentation - solvepartial differential equation from finite differencesdiscretization with implicit timestepping (Mumford-Shahmodel), compare iterative and direct solvers
7) 26 March: Optimization algorithms, deep learning Readinghttps://arxiv.org/abs/1801.05894
8) 2 April: Molecular dynamics simulation using Gromacs
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 6 / 26
Reading
1) 12 February: Solomon chapter 1 andhttps://doi.org/10.1080/10586458.2017.1279092
2) 19 February: Solomon chapters 2 and 33) 26 February: Monte Carlo integration, parallel computing
introduction Solomon chapters 4, 54) 5 March: Solomon chapters 14 and 15, interval analysis
Reading “Differential Equations and Exact Solutions in theMoving Sofa Problem”https://doi.org/10.1080/10586458.2016.1270858
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 7 / 26
Reading
5) 12 March: Matrix Multiply, Eigenvalue computations,singular value decomposition, use of matrix operations instatistics Solomon chapters 6 and 7
6) 19 March: Solomon chapters 11, 13 and 167) 26 March: Word count, clustering algorithms, optimization
algorithms, deep learning Readinghttps://arxiv.org/abs/1801.05894 Solomon chapters 8, 9and 12
8) 2 April: Molecular dynamics simulationhttp://www.bevanlab.biochem.vt.edu/Pages/Personal/justin/gmx-tutorials/lysozyme/index.html
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 8 / 26
Algorithms
Picture of a statute of Al-Khawarizmi fromhttps://commons.wikimedia.org/wiki/File:Khwarizmi_Amirkabir_University_of_Technology.png
Al-Khawarizmi is known for “0” and hand calculationmethods for solving systems of linear equations
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 9 / 26
Algorithms
x + 2y = 52x + 3y = 1
x + 2y = 5y = −4
x = 13y = −4
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 10 / 26
Algorithms
[1 22 3
] [xy
]=
[51
]
[1 20 1
] [xy
]=
[5−4
]
[1 00 1
] [xy
]=
[13−4
]
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 11 / 26
Algorithms
This was Gaussian elimination method for solving asystem of linear equationsAlgorithm is suitable for computer implementationOther algorithms possible for solving systems of linearequations, some are faster than others but may also bemore complicated to program
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 12 / 26
Algorithms
Two competing banks offer interest rate of 1% on depositscompounded daily and 1.2% compounded weekly. Whichbank do you deposit your money in?
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 13 / 26
Algorithms
Two competing banks offer interest rate of 1% on depositscompounded daily and 1.07% compounded weekly. Whichbank do you deposit your money in?
7∏i=0
1.01 = 1.017 = 1.072 > 1.07
r=1.01;funds=1;i=0;while i less than 7 do
funds = funds * r ;i = i + 1 ;
endOutput: funds
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 14 / 26
Algorithms
Methods from scientific computing are used in a variety ofdifferent areas.They typically need to be programmed to run on acomputerDepending on the end use case, one programminglanguage may be more suited than another
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 15 / 26
Programming languages typically used for scientificcomputing
Scripting languagesPythonMatlabOctaveBashJuliaR
Compiled languagesCC++FortranPascal
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 16 / 26
Other programming languages
Scripting languagesPhpRubyJavascriptBashC#Perl
Compiled languagesJavaHaskellPrologValaGuile
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 17 / 26
Typical advantages and disadvantages of scripted andcompiled programs
Scripting languagesShorter programsEasy to debug and writeSlower to executeUsually good for prototyping
Compiled languagesFast to executeMore difficult to debug and writeUsually good for longer lived code
PracticeIn practice usually mix scripting languages (high levelprogram control) and compiled code (optimized libraries forcritical and heavily used functional building blocks)
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 18 / 26
Continuous vs. Discrete Mathematics
Much of scientific computing is motivated by problemswritten in the language of continuous mathematicsComputers are much better at discrete mathematics
∫ b
af (x)dx ≈
n∑i=0
f (xi)∆x
dfdx≈ f (x + ∆x)− f (x)
∆xYou will need to learn to translate between the twolanguages
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 19 / 26
Continuous mathematics
Suppose a third bank offers continuously compoundedinterest of 1.05% per week. How much money would youget at the end of one week?
funds(t + δt) = ((1 + r)δt)funds(t)
funds(t) =
[n∏
i=0
((1 + r)t/n)
]funds(0)
Now
limn→∞
[n∏
i=0
((1 + r)t/n)
]= exp((rt)
funds(t) = exp(rt)funds(0)
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 20 / 26
Continuous mathematics
Suppose a third bank offers continuously compoundedinterest of 1.05% per week. How much money would youget at the end of one week?
funds(t + δt) = ((1 + r)δt)funds(t)
funds(t) =
[n∏
i=0
((1 + r)t/n)
]funds(0)
Now
limn→∞
[n∏
i=0
((1 + r)t/n)
]= exp(rt)
funds(t) = exp(rt)funds(0)
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 21 / 26
Continuous mathematics
Typically we are interested in functions that depend onmore than one variable
Contour map fromhttps://commons.wikimedia.org/wiki/File:Contour_map_(PSF).png
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 22 / 26
Discrete and Continuous mathematics
Can describe height h(x , y) by using a matrix
h(i , j) =
[1 2 3 42 3 4 5
]Can describe change in height in x direction by a matrix
h(i + 1, j)− h(i , j) =
[−1 −1 −1 ?−1 −1 −1 ?
]∂h∂x
= limδx→0
h(x + δx , y)− h(x , y)
δx
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 23 / 26
Discrete and Continuous Mathematics
Can describe height h(x , y) by using a matrix
h(i , j) =
[1 2 3 42 3 4 5
]Can describe change in height in y direction by a matrix
h(i , j + 1)− h(i , j) =
[−1 −1 −1 −1? ? ? ?
]∂h∂y
= limδy→0
h(x , y + δy)− h(x , y)
δy
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 24 / 26
Discrete and Continuous Mathematics
Can define a vector field ∇h(x , y) by
∇h(x , y) =
[∂h∂x∂h∂y
]To find the derivative in a particular direction t can useprojection ∇h(x , y) · t[
∂h∂x∂h∂y
]·[txty
]=∂h∂x
tx +∂h∂y
ty
This is particularly helpful on a computer since onenormally chooses perpendicular coordinate directions thatmay not align with the direction in which one wants todetermine the rate of change.When representing a vector field on a computer usuallyuse several matrices, one for each coordinate direction,though best choice may depend on your application
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 25 / 26
Discrete and Continuous Mathematics
Suppose we are hiking in a valley defined byh(x , y) = x4 + y4, how can we find the minimum?In particular on a computer?What about using pencil and paper?
Background of Johann Bartels fromhttps://commons.wikimedia.org/wiki/File:Johann_Christian_Martin_Bartels.jpg
More at: https://doi.org/10.1006/hmat.1997.2123 and https://arxiv.org/abs/math/0509507v1 26 / 26