lecture#4: advanced python
Post on 14-Apr-2018
237 Views
Preview:
TRANSCRIPT
-
7/29/2019 Lecture#4: Advanced Python
1/35
High performance computing
for non-programmers
Lecture #4:Python programminglanguage: Advanced
Glib Ivashkevych
junior researcher, NSC KIPT
-
7/29/2019 Lecture#4: Advanced Python
2/35
List comprehensions
rationale
newlist = []
for elem in oldlist:
newlist.append(somefunc(elem))
Very common pattern
Special syntax for this?
-
7/29/2019 Lecture#4: Advanced Python
3/35
List comprehensions
syntax
newlist = []
for elem in oldlist:
newlist.append(somefunc(elem))
is equivalent to:
newlist = [somefunc(elem) for elem in oldlist]
-
7/29/2019 Lecture#4: Advanced Python
4/35
List comprehensions
syntax
zlist = []
for x in xlist:
for y in ylist: zlist.append(func(x, y))
is equivalent to:
zlist = [func(x,y) for x in xlist for y in ylist]
-
7/29/2019 Lecture#4: Advanced Python
5/35
List comprehensions
syntax
zlist = []
for x in xlist:
for y in ylist:if cond(x,y):
zlist.append(f(x, y))
is equivalent to:
zlist = [f(x,y) for x in xlist for y in ylist
if cond(x,y)]
-
7/29/2019 Lecture#4: Advanced Python
6/35
Generators
rationale
for elem in somelist:
do_something(elem)
What ifsomelist is huge?
Large memory footprint for storing somelist
-
7/29/2019 Lecture#4: Advanced Python
7/35
Generators
yield instruction
def huge_generator(num):
for i in xrange(num):
yield someexpr(i)
some_generator = huge_generator(huge_number)
for elem in some_generator:
do_something(elem)
A lot better now. Generators know how to get
next element (some_generator.next())
-
7/29/2019 Lecture#4: Advanced Python
8/35
Generators
expression syntax
def huge_generator(num):
for i in xrange(num):
yield f(i)
sgen = huge_generator(huge_number)
is equivalent to:
sgen = (f(i) for i in xrange(huge_number))
-
7/29/2019 Lecture#4: Advanced Python
9/35
Iterators
behind the scenes
for elem in somelist:
...for elem in sometuple:
...
for elem in someiterable:
...
Some pattern exists here, yeap?
-
7/29/2019 Lecture#4: Advanced Python
10/35
Iterators
behind the scenes
for calls __iter()__ on container object
__iter()__ returns iterator object
iterator knows how to get next element
and where to stop(raises StopIteration exception, when there are no more elements)
-
7/29/2019 Lecture#4: Advanced Python
11/35
Iterators
how to createclass CustomIterable(object):
def __init__(self, *args, **kwargs):
#initialize your object heredef __iter__(self):
return self
def next(self):
#return some value or raise StopIteration
CustomIterablecan now be used in loops
-
7/29/2019 Lecture#4: Advanced Python
12/35
Magic methods
what about this __ methods?Reserved methods with special meaning
for el in lst:
uses
LstClass.__iter__(lst)
obj = SomeClass(args)
uses
SomeClass.__new__(SomeClass, args) and
SomeClass.__init__(obj, args)
-
7/29/2019 Lecture#4: Advanced Python
13/35
Magic methods
dirty secrets of constructors__init__is not a constructor (its initializer)
although its usually to treat it as suchexcept when subclassing immutables
__new__is an actual constructor
you need it when subclassing immutables
-
7/29/2019 Lecture#4: Advanced Python
14/35
Magic methods
be polite__str__is used to emulate str() built-in
function
__unicode__is used to emulate unicode()
built-in function
__repr__is used to get a string representation
of object (also `obj` and repr() built-in)
-
7/29/2019 Lecture#4: Advanced Python
15/35
Magic methods
be a container__len__is used to emulate len()
len(cont) -> cont.__len__()
or ContClass.__len__(cont)
__getitem__is used to emulate obj[key]
obj[key] -> obj.__getitem__(key)
or ObjClass.__getitem__(obj, key)
-
7/29/2019 Lecture#4: Advanced Python
16/35
Magic methods
be a container__setitem__is used to emulate obj[key]
=value
obj[key] = value ->
obj.__setitem__(key, value)
or ObjClass.__getitem__(obj, key, value)
-
7/29/2019 Lecture#4: Advanced Python
17/35
Magic methods
be a container
__delitem__(self, key): del obj[key]__iter__(self): iter(obj) or in loops
__reversed__(self): reversed(obj)
__contains__(self, item): item in obj
-
7/29/2019 Lecture#4: Advanced Python
18/35
Magic methods
be smart__setattr__(self, name, value)
Intercept assignment to attribute nameBe aware of infinite recursion!
__getattribute__(self, name)
__getattr__(self, name)
Intercept access to attribute name
-
7/29/2019 Lecture#4: Advanced Python
19/35
Magic methods
be callable__call__(self, ...)
Intercept call on instanceclass SomeClass(object):
def__call__(self, *args, **kwargs):
do_something
s = SomeClass()
s(1,2,3)
-
7/29/2019 Lecture#4: Advanced Python
20/35
Magic methods
all together
Magic methods in Python allow you to create classes,
which look and behave like built-in types
See this blog post for general overview
http://www.rafekettler.com/magicmethods.html
http://www.rafekettler.com/magicmethods.htmlhttp://www.rafekettler.com/magicmethods.html -
7/29/2019 Lecture#4: Advanced Python
21/35
Decorators
rationale
Add functionality without changing the code
Examples:
logging, timing, tests
-
7/29/2019 Lecture#4: Advanced Python
22/35
Decorators
rationaledef wrapper(f):
def inner(*args):print Gonna call %s % f.func_name
f(*args)
return inner
def func(*args):
do_something
func = wrapper(func) #additional functionality
-
7/29/2019 Lecture#4: Advanced Python
23/35
Decorators
rationaledef wrapper(f):
def inner(*args):print Gonna call %s % f.func_name
f(*args)
return inner
@wrapper
def func(*args):
do_something
-
7/29/2019 Lecture#4: Advanced Python
24/35
Decorators
rationaleAnd now magic happens: wrapper can be used to decorateany function. Code reuse, good design - PROFIT!
...
@wrapper
def f1(*args):
do_something
@wrapper
def f2(*args):
do_something_else
...
-
7/29/2019 Lecture#4: Advanced Python
25/35
Decorators
parametrizedWhat if your decorator depends on some value? Thats ok.
def dec(a):
def wrapper(f): This is actual decorator
def inner(*args):
print Func is %s, param is %s % (f.func_name,str(a))
f(*args)
return inner
return wrapper
@dec(Some string parameter)
def func(*args):
...
-
7/29/2019 Lecture#4: Advanced Python
26/35
Decorators
leveraging __call__class MyDecorator(object):
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
print Starting %s call %s self.f.func_name
self.f(*args, **kwargs)
print Finished %s %s self.f.func_name@MyDecorator
def func(*args, **kwargs):
...
Cool. Really.
-
7/29/2019 Lecture#4: Advanced Python
27/35
Metaprogramming
code that creates codeclass MyClass(object):
def __init__(self, data):
self.data = data Something interesting
...
...
Classes and instances can beextended at runtime
-
7/29/2019 Lecture#4: Advanced Python
28/35
Metaprogramming
code that creates code>>> MyClass.new_method = lambda self: self.data * 2
>>> m = MyClass(5.)
>>> m.new_method()
10.
Classes and instances can beextended at runtime
-
7/29/2019 Lecture#4: Advanced Python
29/35
Metaprogramming
rationaleclass IsothermalIdealGas(object):
pressure = 1.
temp = 1.
volume = 1.
quant = 1.
Not cool. Really. PV = nT? Never mind.(forget about R, this is computer science class)
Developer ofIsothermalIdealGasnever heard
about equation of state. Lets patch it.
-
7/29/2019 Lecture#4: Advanced Python
30/35
Metaprogramming
templating:jinja2from jinja2 import Template
tpl = Template(def custom_setattr(self, name, value):
if name == 'P':
object.__setattr__(self, name, value)
object.__setattr__(self, 'V', {{V_eq}})
elif name == 'V':
object.__setattr__(self, name, value)
object.__setattr__(self, 'P', {{P_eq}}))v_eq = Template(self.{{T}}*self.{{Q}} / self.{{V}}**{{k}})
p_eq = Template((self.{{T}}*self.{{Q}} / self.{{P}})**(1./
{{k}}))
keys = {T:T, P:P, V:V, Q:Q, k:2}
source = tpl.render(V_eq=v_eq.render(...), P_eq=p_eq.render(...))
-
7/29/2019 Lecture#4: Advanced Python
31/35
Metaprogramming
templating:jinja2Template variables
{{var}},{{var.someattr}}, {{var[key]}}
for block{% for item in iterable %}
...
{% endfor %}
if block{% if var%}
...
{% endif %}
-
7/29/2019 Lecture#4: Advanced Python
32/35
Metaprogramming
templating:jinja2from jinja2 import Template
tpl = Template({{foo}} - {{bar}})rtpl = tpl.render(foo=First,bar=Second)
or
rtpl = tpl.render({foo:First,bar:Second})
Ok, now we can generate code at runtime.
Usually, for C/CUDA code.
-
7/29/2019 Lecture#4: Advanced Python
33/35
Metaprogramming
warningNever,
ever-trust other peoples data when using
it to generate executable code.Always assume the worst. They all
want to crash your code. Always.
(google it, yeah. something like python eval dangerous)
-
7/29/2019 Lecture#4: Advanced Python
34/35
Python
With great power comes greatresponsibility.- Uncle Ben from SpiderMan
or, for those who prefer classic:
Un grand pouvoir implique degrandes responsabilits.
- Franois-Marie Arouet aka Voltaire
http://conjugaison.lemonde.fr/conjugaison/troisieme-groupe/pouvoirhttp://conjugaison.lemonde.fr/conjugaison/troisieme-groupe/pouvoir -
7/29/2019 Lecture#4: Advanced Python
35/35
Questions?
top related