introduction to python - michael-kraus.orgmichael-kraus.org/teaching/python_w11/python_ipp_1.pdf ·...

Introduction to Python

Part 1: The Python Programming Language and the Cython Extension

Michael Kraus ([email protected])

Max-Planck-Institut fur Plasmaphysik, Garching

27. February 2012

About Python

high-level, general-purpose, interpreted

multi-paradigm programming language (structural, object-oriented,functional, meta programming)

open source (interpreter, libraries, IDEs)

portable (available on Linux, Mac, Windows, ...)

powerful data structures and practical shortcuts

large and comprehensive standard library

lots of freely available third-party modules

extendable with C/C++, Fortran, Javaembed code (weave, f2py)wrap libraries (SWIG, ctypes)wrappers for common libraries (Qt, GTK, VTK, FFTW, MPI, ...)

Cython: extension of the Python languageprovides static typed functions and variablesgenerates efficient C code for fast computations

IDEs: IDLE, PyDev Plugin for Eclipse

Python and Scientific Computing

numpy: numerical array objects and routines to manipulate them

scipy: high-level data processing routines (regression, interpolation, ...)

matplotlib: 2D visualisation (LaTeX support, publication-ready plots)

mayavi: 3D visualisation

netcdf/hdf5: efficient data storage for scientific applications

cython: fast computations, shared-memory parallelisation

� glue between existing components

couple already-written codesadd new functionality to your codes (e.g. a GUI),using the plethora of available python modules

� powerful but simple reading and writing of data in different file formats

� analysis and visualisation of data (interactive, automated)

References

Practical Programming: An Introduction to ComputerScience Using Python (2009)Jennifer Campbell, Paul Gries, Jason Montojo, Greg Wilson

A Primer on Scientific Programming with Python (2011)Hans Petter Langtangen

Python Essential Reference (2009)David Beazley

References

Think Python: How to Think Like a Computer Scientisthttp://greenteapress.com/thinkpython/thinkpython.html

Python Scientific Lecture Noteshttp://scipy-lectures.github.com/

Software Carpentryhttp://www.software-carpentry.org/

Google’s Python Classhttp://code.google.com/edu/languages/

google-python-class/

Pythonhttp://www.python.org/

Cythonhttp://www.cython.org/

http://greenteapress.com/thinkpython/thinkpython.html

http://scipy-lectures.github.com/

http://www.software-carpentry.org/

http://code.google.com/edu/languages/google-python-class/

http://code.google.com/edu/languages/google-python-class/

http://www.python.org/

http://www.cython.org/

Python on the Linux Cluster

default: Python 2.6 (most packages outdated)

optional: Python 2.7 (regular updates, compiled with Intel)

module load python27/python

module load python27/nose

module load python27/numpy

module load python27/scipy

module load python27/cython

module load python27/ipython

module load python27/pyqt

module load python27/matplotlib

module load python27/h5py

module load python27/netcdf

prerequisites:

module load intel /12.1

module load mkl /10.3

Outline

Python

interpreter and shell

basic language features

data types and containers

control flow

functions

scripts and modules

input/output

exceptions

object oriented programming

Cython

compilation

static types

parallelisation

Python Basics: Interpreter and Shell

interpreted (as opposed to compiled) language

� Python code is not compiled prior to execution

interactive mode (advanced calculator)

� type in Python code directly into command line shell

� gives immediate feedback for each statement

> python

Python 2.7.2 (default , Dec 15 2011, 14:57:19)

[GCC Intel(R) C++ gcc 4.3 mode] on linux2

Type "help", "copyright", "credits" or "license" for ...

>>> print("Hello World!")

Hello World!

IPython: enhanced Interactive Python

� enhanced Python shell providing a rich toolkit

� improved interactive data visualisation (pylab)

> ipython -pylab

Python Basics: Interpreter and Shell

� quick help: help(function), help(module), dir(module), dir(variable)

>>> import math

>>> help(math)

help on module math:

[...]

DESCRIPTION

This module is always available. It provides access to

the mathematical functions defined by the C standard.

[...]

run a Python script in the interpreter:

> python script.py

run a Python script and enter interactive mode afterwards:

> python -i script.py

Python Basics: Indentation

blocks of code are delimited by indentation levels

� instruction under a flow-control keyword (if, else, while, for) or in thebody of a function must be indented

you can use as many spaces/tabs as you want (but consistently!)

guarantees some structure of the program and allows to visualise thisstructure quite easily

� correct:

>>> letters = [’a’,’b’,’c’]

>>> for s in letters:

>>> print(s)

� wrong (will throw an exception, IndentationError: unexpected indent):

>>> letters = [’a’,’b’,’c’]


>>> print(s)

>>> letters = [’a’,’b’,’c’]


>>> print(s)

Python Basics: Types

dynamic typing: an object’s type can change during runtime

>>> a = ’Hello ’

>>> type(a)

<type ’str’>

>>> a = 3

>>> type(a)

<type ’int’>

scalar data types: integer, float (64bit), complex (native!), boolean:

>>> b = 3.

>>> type(b)

<type ’float ’>

>>> c = 1. + 0.5j

>>> type(c)

<type ’complex ’>

>>> d = (3 > 4)

>>> type(d)

<type ’bool’>

>>> print(a)

3

>>> print(b)

3.0

>>> print(c.real)

1.0

>>> print(c.imag)

0.5

>>> print(d)

False

Python Basics: Containers

lists: ordered collection of objects, that may have different types

>>> l = [1, 2, 3, 4, 5]

>>> type(l)

<type ’list’>

� indexing: access individual objects contained in the list (starting at 0)

>>> l[2]

3

� counting from the end with negative indices:

>>> l[-1]

5

� slicing (sublists): l[start:stop:stride] (note: l[start:stop] containsthe elements with index i such that start <= i < stop)

>>> l[2:4]

[3, 4]

>>> l[3:]

[4, 5]

>>> l[::2]

[1, 3, 5]


� append elements:

>>> l.append (6)

>>> l

[1, 2, 3, 4, 5, 6]

� extent by another list:

>>> l.extend ([7 ,8])

>>> l

[1, 2, 3, 4, 5, 6, 7, 8]

� reverse order:

>>> l.reverse ()

>>> l

[8, 7, 6, 5, 4, 3, 2, 1]

� sort:

>>> l.sort()

>>> l

[1, 2, 3, 4, 5, 6, 7, 8]


� pop last element:

>>> l.pop()

8

>>> l

[1, 2, 3, 4, 5, 6, 7]

� remove some element:

>>> l.remove (4)

>>> l

[1, 2, 3, 5, 6, 7]

� number of elements:

>>> len(l)

7

� elements can be of different type:

>>> l.append(’s’)

>>> l

[1, 2, 3, 5, 6, 7, ’s’]


tuples: immutable lists (lets you create simple data structures)

� written between brackets, e.g. u = (x,y):

>>> u = (0,2)

>>> u

(0, 2)

or just separated by commas:

>>> t = 12345, 67890, ’hello ’, u

>>> t

(12345 , 67890 , ’hello’, (0, 2))

� tuples are unpacked using list syntax:

>>> u[1]

2

� elements of a tuple cannot be assigned independently:

>>> u[1] = 3

TypeError: ’tuple’ object does not support item assignment


sets: unordered collection with no duplicate elements

>>> basket = [’apple ’, ’orange ’, ’apple ’, \

’pear’, ’orange ’, ’banana ’]

>>> fruit = set(basket)

>>> fruit

set([’orange ’, ’pear’, ’apple’, ’banana ’])

� eliminates duplicate entries

� membership testing:

>>> ’orange ’ in fruit

True

>>> ’melon ’ in fruit

False


� create sets of characters:

>>> s = set(’abcd’)

>>> s

set([’a’, ’c’, ’b’, ’d’])

>>> t = set((’e’, ’f’, ’g’))

>>> t

set([’e’, ’g’, ’f’])

� add elements:

>>> s.add(’a’)

>>> s

set([’a’, ’c’, ’b’, ’d’])

>>> t.update ([’a’, ’b’])

>>> t

set([’a’, ’b’, ’e’, ’g’, ’f’])

� remove an element:

>>> t.remove(’g’)

>>> t

set([’a’, ’b’, ’e’, ’f’])


� support for mathematical operations like union, intersection, difference,and symmetric difference

� letters in s but not in t:

>>> s-t

set([’c’, ’d’])

� letters in either s or t:

>>> s | t

set([’a’, ’c’, ’b’, ’e’, ’d’, ’f’])

� letters in both s and t:

>>> s & t

set([’a’, ’b’])

� letters in t or s but not both:

>>> s ^ t

set([’c’, ’e’, ’d’, ’f’])


dictionaries: hash table that maps keys to values (unordered)

>>> tel = { ’peter ’: 5752, ’mary’: 5578 }

>>> tel[’jane’] = 5915

>>> tel

{’jane’: 5915, ’peter’: 5752, ’mary’: 5578}

>>> tel[’mary’]

5578

>>> tel.keys()

[’jane’, ’peter ’, ’mary’]

>>> tel.values ()

[5915, 5752, 5578]

>>> ’peter ’ in tel

True

� dictionaries can have keys and values with different types:

>>> d = { ’a’: 1, ’b’: 2, 3:’hello ’ }

>>> d

{’a’: 1, 3: ’hello’, ’b’: 2}

Python Basics: Control Flow

if/elif/else:

>>> if 2**2 == 4:

... print(’Obvious!’)

...

Obvious!

� no brackets enclosing the conditional statement

>>> a = 10

>>> if a == 1:

... print(’one’)

... elif a == 2:

... print(’two’)

... else:

... print(’a lot’)

...

a lot


for/range

>>> for i in range (3):

... print(i)

...

0

1

2

� for always loops through a sequence of elements� range([start,] stop[, step]) creates lists of arithmetic progressions� the arguments must be plain integers� the start argument defaults to 0, the step argument defaults to 1

while

>>> z = 1. + 1.j

>>> while abs(z) < 100:

... z = z**2 + 1.

...

>>> z

( -134+352j)


break

>>> z = 1. + 1.j

>>> while abs(z) < 100:

... if abs(z.imag) > 10:

... break

... z = z**2 + 1.

...

>>> z

(-11-16j)

continue

>>> a = [1, 0, 2]

>>> for element in a:

... if element == 0:

... continue

... print 1. / element

...

1.0

0.5


iterate over any sequence (list, set, dictionary, string, file, ...)

>>> for word in (’cool’, ’powerful ’, ’readable ’):

... print(’Python is %s’ % word)

...

Python is cool

Python is powerful

Python is readable

iterate over a dictionary:

>>> d = {’a’:1, ’b’: 2, ’c’: 1j }

>>> for key , val in d.iteritems ():

... print(’Key: %s has value: %s’ % (key ,val) )

...

Key: a has value: 1

Key: c has value: 1j

Key: b has value: 2

� for key in d: only gives you the keys, not the values(but of course, values can be accessed via d[key] within the loop)

Python Basics: Functions

defining functions with def:

>>> def test ():

... print(’This is a test!’)

...

>>> test()

This is a test!

passing arguments and returning a value:

>>> def disk_area(radius ):

... return 3.14 * radius * radius

...

>>> disk_area (1.5)

7.0649999999999995

� in the arguments, Python passes references to objects by value

� all values that the function should return... must be returned

� by default, functions return None


in Python everything is an object, variables hold references to objects� every variable assignment is an assignment of a reference

a = 2

h = [1,2,3]

� the value of a is a reference to an object of type int

� the value of h is a reference to an object of type list

Python passes references to objects by value� assigning a new instance to a variable (i.e. a reference to a new object)

within a function does not change the original variable outside thefunction

>>> def some_function(s):

... s = set(’xyz’)

...

>>> u = set(’abcd’)

>>> some_function(u)

>>> print(u)

set([’a’, ’c’, ’b’, ’d’])


immutable objects: numbers, strings, tuples, booleans

>>> def square(x):

... print(id(x))

... x *= x

.... print(id(x))

... return x

...

>>> a = 2

>>> b = square(a)

4312871488

4312871440

>>> print(a)

2

>>> print(b)

4

� altering immutable objects inside a function creates a new instance

� the original instance outside the function is not changed


mutable objects: lists, sets, dictionaries, classes

>>> def set_first_element_to_zero(l):

... l[0] = 0

...

>>> c = [1,2,3]

>>> print(c)

[1, 2, 3]

>>> set_first_element_to_zero(c)

>>> print(c)

[0, 2, 3]

� such changes inside a function are preserved when the function returns� the referenced object is changed (e.g. list entries, class variables), but

>>> def some_function(l):

... l = [4,5,6]

...

>>> b = [1,2,3]

>>> some_function(b)

>>> print(b)

[1, 2, 3]


multiple values can be returned as a sequence:

>>> def integer_divide(x,y):

... integer_part = x/y

... the_rest = x%y

... return integer_part , the_rest

...

>>> int , rest = integer_divide (10 ,3)

>>> print(int)

3

>>> print(rest)

1

map(f,s) applies the function f to the elements of s and returns a list:

>>> def square(x):

... return x*x;

>>> l = [1, 2, 3, 4]

>>> map(square ,l)

[1, 4, 9, 16]


optional parameters (keyword or named arguments):

>>> def say_my_name(name=’How should I know?’):

... return name

...

>>> say_my_name ()

’How should I know?’

>>> say_my_name(’Mike’)

’Mike’

� keyword arguments allow you to specify default values

� the order of keyword arguments does not matter:

>>> def slicer(seq , start=None , stop=None , step=None):

... return seq[start:stop:step]

...

>>> l = [1,2,3,4,5,6]

>>> slicer(l, start=1, stop =4)

[2, 3, 4]

>>> slicer(l, step=2, stop=5, start =1)

[2, 4]


functions are objects� functions can be assigned to a variable:

>>> func = test

>>> func()

This is a test!

� functions can be put into a list (or any collection):

>>> l = [1, ’a’, test]

>>> l

[1, ’a’, <function test at 0x10698e050 >]

>>> l[2]()

This is a test!

� functions can be passed as an argument to another function:

>>> def call_func(function ):

... function ()

...

>>> call_func(test)

This is a test!


docstrings: documentation about what the function does and it’sparameters

>>> def funcname(params ):

... """

... Concise one -line sentence describing the function.

...

... Extended summary , can contain multiple paragraphs.

... """

... # function body

... pass

...

>>> help(funcname)

funcname(params)

Concise one -line sentence describing the function.

Extended summary , can contain multiple paragraphs.

� the docstring conventions webpage documents the semantics andconventions associated with Python docstrings

http://www.python.org/dev/peps/pep-0257/

Python Basics: Scripts

you do not want to type everything into the shell, especially for longersets of instructions

� write a text file containing code using an editor with syntax highlighting

� execute by starting the interpreter with the filename as argument

script: file with extension .py containing a sequence of instructions thatare executed each time the script is called

test.py

message = "How are you?"

for word in message.split ():

print(word)

� execute the script:

> python test.py

How

are

you?

Python Basics: Modules

put definitions in a file and use them as a module in other scripts

div.py

def integer_divide(x,y):

integer_part = x/y

the_rest = x%y

return integer_part , the_rest

� importing the module gives access to its objects via module.object:

import div

int , rest = div.integer_divide (10 ,3)

� import objects from modules into the main namespace:

from div import integer_divide

int , rest = integer_divide (10 ,3)

� prescribe a shortcut for the module:

import div as d

int , rest = d.integer_divide (10 ,3)


create modules if you want to write larger and better organisedprograms

� define objects (variables, functions, classes) that you want to reuseseveral times in your own modules

demo.py

"""A demo module."""

def print_b ():

""" Prints b."""

print ’b’

def print_a ():

""" Prints a."""

print ’a’

print("Module demo got loaded!")

c = 2

d = 2


� import the demo module in another script and call its functions:

demo test.py

import demo

demo.print_a ()

demo.print_b ()

print(demo.c)

print(demo.d)

> python demo_test.py

Module demo got loaded!

a

b

2

2


� take a look at the documentation:

>>> help(demo)

Help on module demo:

NAME

demo - A demo module.

FILE

/Users/mkraus/Lehre/Python/code/demo.py

FUNCTIONS

print_a ()

Prints a.

print_b ()

Prints b.

DATA

c = 2

d = 2


you can define a __main__ function that is executed if the module isexecuted as a script but not when the module is imported:

demo2.py

""" Another demo module."""

def print_a ():

""" Prints a."""

print ’a’

if __name__ == ’__main__ ’:

print_a ()

� execute the module on the shell:

> python demo2.py

a

� load the module in interactive mode:

>>> import demo2

>>>


a directory that contains many modules is called a package� a package is a module with submodules (which can have submodules

themselves, etc.)� a special file called __init__.py (might be empty) tells Python that the

directory is a Python package, from which modules can be imported� a package might be organised as follows:

Graphics/

__init__.py

Primitive/

__init__.py

lines.py

text.py

...

Graph2D/

__init__.py

plot2d.py

...

Graph3D/

__init__.py

plot3d.py

...

....

Python Basics: Input/Output

reading and writing of strings from and to files

� write to a file:

>>> f = open(’tempfile ’, ’w’)

>>> type(f)

<type ’file’>

>>> f.write(’This is a test \nand another test’)

>>> f.close ()

� read from a file:

>>> f = open(’tempfile ’, ’r’)

>>> s = f.read()

>>> print(s)

This is a test

and another test

� numpy features more powerful reading/writing routines

Python Basics: Input/Output

� iterate over a file:

>>> f = open(’tempfile ’, ’r’)

>>> for line in f:

... print(line)

...

This is a test

and another test

� file modes:

r : read-onlyw : write-only (create a new file or overwrite existing file)a : appendr+: read and write

Python Basics: Exceptions

exceptions are raised by different kinds of errors arising when executingPython code and normally terminate program execution:

>>> 1/0

ZeroDivisionError: integer division or modulo by zero

>>> 1 + ’e’

TypeError: unsupported operand type(s) for +: ’int’ and ’str’

>>> d = { 1:1, 2:2 }

>>> d[3]

KeyError: 3

>>> l = [1,2,3]

>>> l[3]

IndexError: list index out of range

>>> l.foobar

AttributeError: ’list’ object has no attribute ’foobar ’

� you may (and often should) catch errors or define custom error types


catching exceptions: try/except

exceptions1.py

while True:

try:

x = int(raw_input(’Please enter a number: ’))

break

except ValueError:

print(’That was no valid number. Try again ...’)

> python exceptions1.py

Please enter a number: a

That was no valid number. Try again ...

Please enter a number: 1


catching exceptions: try/finally

exceptions2.py

try:


finally:

print(’Thank you for your input.’)



Thank you for your input.


Please enter a number: c

Thank you for your input.

ValueError: invalid literal for int() with base 10: ’c’

� important for resource management (e.g. closing a file)


raising exceptions to pass messages between parts of the code

exceptions3.py

def achilles_arrow(x):

if abs(x - 1) < 1e-3:

raise StopIteration

x = 1 - (1-x)/2.

return x

x = 0

while True:

try:

x = achilles_arrow(x)

except StopIteration:

print(’Iteration has stopped ’)

break

print (" Result: " + str(x))


Iteration has stopped

Result: 0.9990234375


reraise an exception:

exceptions4.py

while True:

try:


break

except ValueError , e:

print(’That was no valid number.’)

raise e


Please enter a number: a

That was no valid number.

ValueError: invalid literal for int() with base 10: ’a’



Python Basics: Object Oriented Programming (OOP)

python supports object oriented programming

� helps to organise your code

� simplifies reuse of code in similar contexts

objects consist of internal data and methods that perform various kindsof operations on that data

example: harmonic oscillator

� attributes: mass m, spring constant k, initial displacement x0

� methods: total_energy(), period(), frequency(), acceleration(x),velocity(x), kinetic_energy(x), ...


classes gather custom functions and variables

student.py

class Student:

def __init__(self , name):

self.name = name

def set_age(self , age):

self.age = age

def set_major(self , major):

self.major = major

def info(self):

print(’%s is a student in %s and %d years old.’ \

% (self.name , self.major , self.age) )

>>> from student import Student

>>> anna = Student(’anna’)

>>> anna.set_age (21)

>>> anna.set_major(’physics ’)

>>> anna.info()

anna is a student in physics and 21 years old.


inheritance allows to derive new classes from old ones that have thesame methods and attributes but might be extended by new ones

>>> class MasterStudent(Student ):

... internship = ’mandatory , from March to June’

...

>>> james = MasterStudent(’james ’)

>>> james.internship

’mandatory , from March to June’

� you can access all methods and attributes defined in Student as well:

>>> james.set_age (23)

>>> james.age

23

>>> james.set_major(’math’)

>>> james.info()

james is a student in math and 23 years old.

Cython

Cython is a programming language based on Python

� provides extra syntax allowing for static type declarations

(remember: Python is generally dynamically typed)

� the source code gets translated into optimised C/C++ code andcompiled as Python extension modules

� Cython can compile (most) regular Python code but the optional statictype declarations usually achieve major speed improvements

� allows for very fast program execution and tight integration withexternal C libraries

� combines the benefits of Python with the speed of C

References:http://www.cython.org/

http://docs.cython.org/

http://www.cython.org/

http://docs.cython.org/

Cython: Compilation

Cython code must, unlike Python, be compiled:

a .pyx source file is compiled by Cython to a .c file, containing the code ofa Python extension modulethe .c file is compiled by a C compiler to a .so (shared object library) filewhich can be imported directly into a Python session

� several ways to build Cython code:

use pyximport, importing Cython .pyx files as if they were .py files (usingdistutils to compile and build the background)write a distutils setup.pyrun the cython command-line utility manually to produce the .c file fromthe .pyx file, then manually compile the .c file into a shared object libraryuse the Sage notebook which allows Cython code inline

Cython: Compilation

imagine a simple “hello world” script:

hello.pyx

def say_hello_to(name):

print("Hello %s!" % name)

implicit compilation using pyximport:

>>> import pyximport

>>> pyximport.install ()

>>> import hello

>>> hello.say_hello_to("Mike")

Hello Mike!

� this allows to automatically run Cython on every .pyx that Python istrying to import

� use this for simple Cython builds where no extra C libraries and nospecial building setup is needed

Cython: Compilation

the following could be a corresponding setup.py script:

setup.py

from distutils.core import setup

from distutils.extension import Extension

from Cython.Distutils import build_ext

ext_modules = [Extension("hello", ["hello.pyx"])]

setup(

name = ’Hello World App’,

cmdclass = {’build_ext ’: build_ext},

ext_modules = ext_modules

)

� you can specify additional build settings (include dirs, linker flags, ...)

ext_modules = [Extension("hello", ["hello.pyx"],

include_dirs = [numpy.get_include ()],

extra_compile_args = [’-fopenmp ’],

extra_link_args = [’-fopenmp ’] )]

Cython: Compilation

the script can be called with:

> python setup.py build_ext --inplace

after compilation, Cython modules can be imported like every otherPython module:

>>> import hello

>>> hello.say_hello_to("Mike")

Hello Mike!

Cython: Static Types

Cython can compile normal Python code without changes (with a fewexceptions of some as-yet unsupported language features), but forperformance critical code, it is often helpful to add static typedeclarations

� they allow Cython to step out of the dynamic nature of the Pythoncode and generate simpler and faster C code (sometimes faster byorders of magnitude)

� however, type declarations can make the source code more verbose andthus less readable

� it is discouraged to use them without good reason, i.e. only inperformance critical sections where they really make the codesubstantially faster


consider the following pure Python code:

def f(x):

return x**2-x

def integrate_f(a, b, N):

s = 0

dx = (b-a)/N

for i in range(N):

s += f(a+i*dx)

return s * dx

� simply compiling this in Cython gives a 35% speedup

� adding some static types can make a much larger difference


with additional type declarations, the former example might look like:

def f(double x):

return x**2-x

def integrate_f(double a, double b, int N):

cdef int i

cdef double s, dx

s = 0

dx = (b-a)/N

for i in range(N):

s += f(a+i*dx)

return s * dx

� typing the iterator variable i with C semantics, tells Cython to compilethe for-loop to pure C code

� typing a, s and dx is important as they are involved in arithmetic withinthe for-loop

� this results in a 4x speedup over the pure Python version


Python function calls can be expensive: one needs to convert to andfrom Python objects to do the call

� in our example, the argument is assumed to be a C double both insidef() and in the call to it, yet a Python float object must be constructedaround the argument in order to pass it

� Cython provides the cdef keyword for declaring a C-style function:

cdef double f(double x):

return x**2-x

� speedup: 150x over pure Python

� but: now the function is no longer available from Python-space, asPython wouldn’t know how to call it

� using the cpdef keyword instead, a Python wrapper is created as well

Cython: Static Types and NumPy Arrays

you can use NumPy from Cython exactly the same as in regularPython, but by doing so you are loosing potentially high speedups

� Cython has support for fast access to NumPy arrays:

import numpy as np

cimport numpy as np

def update(np.ndarray[double , ndim =2] f):

cdef unsigned int i, j

cdef np.ndarray[double , ndim =2] h = \

np.zeros ([xmax , ymax], dtype=double)

for i in range(0, f.shape [0]):

for j in range(0, f.shape [1]):

...

� typing the contents of ndarray by specifying the datatype and thenumber of dimensions enables access to the data buffer directly at Cspeed, otherwise the [] operator still uses full Python operations

Cython: Faster Code

if you’re confident you’ve done everything right, there are a fewdirectives that can further speed up your code

� boundscheck=False disables array boundary checks

� wraparound=False disables negative indexing of NumPy arrays

there are several ways of setting these compiler directives

� globally, with a special header comment at the top of the file:

#cython: boundscheck=False

#cython: wraparound=False

� locally, as function decorators or in a with statement:

cimport cython

@cython.boundscheck(False)

def update(np.ndarray[double , ndim =2] f):

...

with cython.boundscheck(True):

...

Cython: Parallelisation

Cython supports native parallelism via OpenMP (more backends mightbe supported in the future)

by default, Python’s Global Interpreter Lock (GIL) prevents thatseveral threads use the Python interpreter simultaneously

� to use this kind of parallelism, the GIL must be released

� parallel loops can be created by usingprange([start], stop[, step], nogil=False, schedule=None)

� OpenMP automatically starts a thread pool and distributes the workaccording to the schedule used

static: approx. equally sized chunks, one for each threaddynamic:iterations are distributed as threads request them, chunk size of 1guided: iterations are distributed as threads request them, the chunk sizeis proportional to the number of unassigned iterations divided by thenumber of threads, decreasing to 1auto: the decision regarding scheduling is delegated to the compiler orruntime system

� you have to add the NumPy includes and OpenMP flags to setup.py

Cython: Parallelisation

thread-locality and reductions are automatically inferred for variablesexample with reduction: the values from the thread-local copies of thevariable will be reduced with the operator and assigned to the originalvariable after the loop

from cython.parallel import prange

cdef int i

cdef int sum = 0

for i in prange(n, nogil=True):

sum += i

example with a shared NumPy array:

from cython.parallel import prange

def func(np.ndarray[double] x, double alpha):

cdef unsigned int i

for i in prange(x.shape[0], nogil=True):

x[i] = alpha * x[i]

Outlook: Scientific Computing and Plotting

NumPy: fundamental package for scientific computing with Python

powerful N-dimensional array object and various derived objects (matrices)routines for fast operations on arrays (selecting, sorting, shapemanipulation, mathematical, I/O, ...)modules for linear algebra, statistics, Fourier transforms, random numbersit’s fast: most routines are optimised, pre-compiled C code

SciPy: numerical routines, data manipulation and analysis

Matplotlib: 2D plotting

allows generation of plots, histograms, power spectra, bar charts, errorcharts, scatter plots, etc., with just a few lines of codefull control of line styles, font properties, axes properties, etc., via anobject oriented interface or via a set of functions similar to MATLAB

MayaVi: 3D plotting and interactive scientific data visualisation

HDF5/NetCDF: self-describing and efficient storage of scientific data

Outlook: Advanced Topics

calling and embedding C and Fortran code

weave: embed C code within your .py filesSWIG: automatic wrapper generation for C/C++ codectypes, cython: wrap librariesf2py: wrap and compile Fortran routines to behave like Python modules

GUI programming with PyQt and PySide

overview and simple examplesbasic widgetssignals and slotsmenus, toolbars, statusbarlayoutdialogsembedding matplotlib plotscreating UIs with Qt Designer

Symbolic Calculations with Sage

introduction to python - michael-kraus.orgmichael-kraus.org/teaching/python_w11/python_ipp_1.pdf ·...

Documents