University of Tartu, Institute of Computer Science

Introduction to Scientific Computing

MTAT.08.025

eero.vainikko@ut.ee

Spring 2015

2 Practical information

Lectures: Liivi 2 - 403 WED 10:15Computer classes: Liivi 2 - 205, THU 10:15 Eero Vainikko3 eapLectures: 16h; Computer Classes: 16h; Independent work: 46hFinal grade forms from :

1. Active partitipation at lectures

2. Stand-up quiz(es)

3. Exam

4. Computer class activities

Course homepage (http://courses.cs.ut.ee/2015/isc)

3 Introduction 1.1 Syllabus

1 Introduction1.1 SyllabusLectures:

• Python for Scientific Computing NumPy, SciPy

• Scientific Computing - an Overview

• Floating point numbers, how to deal with roundoff errors

• Large problems in Linear Algebra, condition number

• Memory hierarchies and making use of it

• Numerical integration and differentiation

• Numerical solution of differential and integral equations

• Fast Fourier Transform

4 Introduction 1.1 Syllabus

Computer Classes (preliminary plan)

1. Python & Sage; Fibonacci numbers; Collatz conjecture

2. Discretization and round-off errors

3. NumPy arrays, matrices; LU-Factorization with Gauss Elimination Method(GEM)

4. UT HPC server; LU-Factorization and GEM on HPC cluster

5. Floating point numbers

6. Fractals

7. Fourier series and Fast Fourier Transform

8. Discrete-time models and ordinary differential equations (optional: Neural Net-work; Image denoising)

5 Introduction 1.2 Literature

1.2 Literature

General Scientific Computing:

1. RH Landau, A First Course in Scientific Computing. Symbolic, Graphic, andNumeric Modeling Using Maple, Java, Mathematica, and Fortran90. PrincentonUniversity Press, 2005.

2. LR Scott, T Clark, B Bagheri. Scientific Parallel Computing. Princenton Uni-versity Press, 2005.

3. MT Heath, Scientific Computing; ISBN: 007112229X, McGraw-Hill Compa-nies, 2001.

4. JW Demmel, Applied Numerical Linear Algebra; ISBN: 0898713897, Societyfor Industrial & Applied Mathematics, Paperback, 1997.

6 Introduction 1.2 Literature

Python:

1. Hans Petter Langetangen, A Primer on Scientific Programming withPython, Springer, 2009. Book webpage (http://vefur.simula.no/intro-programming/).

2. Hans Petter Langtangen, Python Scripting for Computational Science. ThirdEdition, Springer 2008. Web-site for the book (http://folk.uio.no/hpl/scripting/).

3. Neeme Kahusk, Sissejuhatus Pythonisse (http://www.cl.ut.ee/inimesed/nkahusk/sissejuhatus-pythonisse/).

4. Brad Dayley, Python Phrasebook, Sams 2007.

5. Travis E. Oliphant, Guide to NumPy (http://www.tramy.us), TrelgolPublishing 2006.

7 Introduction 1.3 Scripting vs programming

1.3 Scripting vs programming

1.3.1 What is a script?

• Very high-level, often short, programwritten in a high-level scripting language

• Scripting languages:

– Unix shells,

– Tcl,

– Perl,

– Python,

– Ruby,

– Scheme,

– Rexx,

– JavaScript,

– VisualBasic,

– ...

• This course: Python (and Sage)

8 Introduction 1.3 Scripting vs programming

1.3.2 Characteristics of a script

• Glue other programs together

• Extensive text processing

• File and directory manipulation

• Often special-purpose code

• Many small interacting scripts may yield a big system

• Perhaps a special-purpose GUI on top

• Portable across Unix, Windows, Mac

• Interpreted program (no compilation+linking)

9 Introduction 1.3 Scripting vs programming

1.3.3 Why not stick to Java, C/C++ or Fortran?

Features of Perl and Python compared with Java, C/C++ and Fortran:

• shorter, more high-level programs

• much faster software development

• more convenient programming

• you feel more productive

• no variable declarations ,but lots of consistency checks at run time

• lots of standardized libraries and tools

10 Introduction 1.4 Scripts yield short code

1.4 Scripts yield short code

Consider reading real numbers from a file, where each line can contain an arbitrarynumber of real numbers:� �1.1 9 5.2

1.762543E-02

0 0.01 0.001 9 3 7�Python solution:� �

F = open(filename, ’r’)

n = F.read().split()�Perl solution:� �

open F, $filename;

$s = join "", <F>;

@n = split ’ ’, $s;�

11 Introduction 1.5 Performance issues

Ruby solution:� �n = IO.readlines(filename).join.split�

...Doing this in C++ or Java requires at least a loop, and in Fortran and C quitesome code lines are necessary

1.5 Performance issues

1.5.1 Scripts can be slow

• Perl and Python scripts are first compiled to byte-code

• The byte-code is then interpreted

• Text processing is usually as fast as in C

• Loops over large data structures might be very slow

12 Introduction 1.5 Performance issues

� �for i in range(len(A)):

A[i] = ...�• Fortran, C and C++ compilers are good at optimizing such loops at compile time

and produce very efficient assembly code (e.g. 100 times faster)

• Fortunately, long loops in scripts can easily be migrated to Fortran or C (orspecial libraries like numpy!)

13 Introduction 1.5 Performance issues

1.5.2 Scripts may be fast enough

Read 100 000 (x,y) data from file and write (x,f(y)) out again

• Pure Python: 4s

• Pure Perl: 3s

• Pure Tcl: 11s

• Pure C (fscanf/fprintf): 1s

• Pure C++ (iostream): 3.6s

• Pure C++ (buffered streams): 2.5s

• Numerical Python modules: 2.2s (!)

• Remark: in practice, 100 000 data points are written and read in binary format,resulting in much smaller differences

14 Introduction 1.5 Performance issues

1.5.3 When scripting is convenient

• The application’s main task is to connect together existing components

• The application includes a graphical user interface

• The application performs extensive string/text manipulation

• The design of the application code is expected to change significantly

• CPU-time intensive parts can be migrated to C/C++ or Fortran

• The application can be made short if it operates heavily on list or hash structures

• The application is supposed to communicate with Web servers

• The application should run without modifications on Unix, Windows, and Mac-intosh computers, also when a GUI is included

15 Introduction 1.5 Performance issues

1.5.4 When to use C, C++, Java, Fortran

• Does the application implement complicated algorithms and data structures?

• Does the application manipulate large datasets so that execution speed iscritical?

• Are the application’s functions well-defined and changing slowly?

• Will type-safe languages be an advantage, e.g., in large development teams?

16 Python in SC 1.5 Performance issues

2 Python in Scientfic Computing

Slides we might use: Introduction to Scientific Computing with Python (http://www.physics.rutgers.edu/grad/509/python1.pdf)