caching techniques in python, europython2010

Caching techinques in python

Michael Domanskieuropython 2010

czwartek, 22 lipca 2010

who I am

• python developer, professionally for a few years now

• experienced also in c and objective-c

• currently working for 10clouds.com

Interesting intro

• a bit of theory

• common patterns

• common problems

• common solutions

How I think about cache

• imagine a giant dict storing all your data

• you have to manage all data manually

• or provide some automated behaviour

similar to....

• manual memory managment in c

• cache is memory

• and you have to controll it manually

profits

• improved performance

• ...?

problems

• managing any type of memory is hard

• automation often have to be done custom each time

common patterns

memoization

• very old pattern (circa 1968)

• we own the name to Donald Mitchie

• we assosciate input with output, and store in somewhere

• based on the assumption that for a given input, output is always the same

how it works

code example

CACHE_DICT = {}

def cached(key): def func_wrapper(func): def arg_wrapper(*args, **kwargs): if not key in CACHE_DICT: value = func(*args, **kwargs) CACHE_DICT[key] = value return CACHE_DICT[key] return arg_wrapper return func_wrapper

what if output can change?

• our pattern is still usefull

• we simply need to add something

cache invalidation

There are only two hard problems in Computer Science: cache invalidation and naming things

Phil Karlton

• basically, we update data in cache

• we need to know when and what to change

• the more granular you want to be, the harder it gets

def invalidate(key): try: del CACHE_DICT[key] except KeyError: print "someone tried to invalidate not present key: %s" %key

code example

common problems

invalidating too much/not enough

• flushing all data any time something changes

• not flushing cache at all

• tragic effects

@cached('key1')def simple_function1(): return db_get(id=1)

# SUPPOSE THIS IS IN ANOTHER MODULE

@cached('big_key1')def some_bigger_function(): """ this function depends on big_key1, key1 and key2 """ def inner_workings(): db_set(1, 'something totally new') ####### ## imagine 100 lines of code here :) ###### inner_workings()

return [simple_function1(),simple_function2()]

if __name__ == '__main__': simple_function1() simple_function2() a,b = some_bigger_function() assert a == db_get(id=1), "this fails because we didn't invalidated cache properly"

invalidating too soon/too late

• your cache have to be synchronised to you db

• sometimes very hard to spot

• leads to tragic mistakes

def some_bigger_function(): db_set(1, 'something') value = simple_function1() db_set(2, 'something else') #### now we know we used 2 cached functions so.... invalidate('key1') invalidate('key2') #### now we know we are safe, but for a price return simple_function2()

if __name__ == '__main__': some_bigger_function()

superposition of dependancy

• somehow less obvious problem

• eventually you will start caching effects of computation

• you have to know very preciselly of what your data is dependant

@cached('key')def some_bigger_function():

return { '1': simple_function1(), '2': simple_function2(), '3': db_get(id=3) }

if __name__ == '__main__': simple_function1() # somewhere else db_set(1, 'foobar') # and again db_set(3, 'bazbar') invalidate('key') # ooops, we forgot something data = some_bigger_function() assert data['1'] == db_get(id=1), "this fails because we didn't manage to invalidate all the keys"

summing up

• know your data....

• be aware what and when you cache

• take care when using cached data in computation

common solutions

process level cache

• very fast access

• simple to implement

• very effective as long as you’re using single process

clever tricks with dicts

code example

CACHE_DICT = {}

def cached(key): def func_wrapper(func): def arg_wrapper(*args, **kwargs): if not key in CACHE_DICT: value = func(*args, **kwargs) CACHE_DICT[key] = value return CACHE_DICT[key] return arg_wrapper return func_wrapper

invalidation

def invalidate(key): try: del CACHE_DICT[key] except KeyError: print "someone tried to invalidate not present key: %s" %key

code example

application level cache

memcache

• battle tested

• scales

• fast

• supports a few cool features

• behaves a lot like dict

• supports time-based expiration

• python-memcache

• python-libmemcache

• python-cmemcache

• pylibmc

libraries?

why no benchmarks

• not the point of this talk :)

• benchmarks are generic, caching is specific

• pick your flavour, think for yourself

cache = memcache.Client(['localhost:11211'])

def memcached(key): def func_wrapper(func): def arg_wrapper(*args, **kwargs): value = cache.get(str(key)) if not value: value = func(*args, **kwargs) cache.set(str(key), value) return value return arg_wrapper return func_wrapper

code example

invalidation

def mem_invalidate(key): cache.set(str(key), None)

code example

batch key managment

• what if I don’t want to expire each key manually

• that’s a lot to remember

• and we have to be carefull :(

groups?

• group keys into sets

• which are tied to one key per set

• expire one key, instead of twenty

how to get there?

• store some extra data

• you can store dicts in cache

• and cache behaves like dict

• so it’s a case of comparing keys and values

#we start with specified key and groupkey='some_key'group='some_group'

# now retrieve some data from memcacheddata=memcached_client.get_multi(key, group)# now data is a dict that should look like #{'some_key' :{'group_key' : '1234',# 'value' : 'some_value' },# 'some_group' : '1234'}#if data and (key in data) and (group in data): if data[key]['group_key']==data[group]: return data[key]['value']

def cached(key, group_key='', exp_time=0 ):

# we don't want to mix time based and event based expiration models if group_key : assert exp_time==0, "can't set expiration time for grouped keys" def f_wrapper(func): def arg_wrapper(*args, **kwargs): value = None if group_key: data = cache.get_multi([tools.make_key(group_key)]+[tools.make_key(key)]) data_dict = data.get(tools.make_key(key)) if data_dict: value = data_dict['value'] group_value = data_dict['group_value'] if group_value != data[tools.make_key(group_key)]: value = None else: value = cache.get(key) if not value: value = func(*args, **kwargs) if exp_time: cache.set(tools.make_key(key), value, exp_time) elif not group_key: cache.set(tools.make_key(key), value) else: # exp_time not set and we have group_keys group_value = make_group_value(group_key) data_dict = { 'value':value, 'group_value': group_value} cache.set_multi({ tools.make_key(key):data_dict, tools.make_key(group_key):group_value }) return value arg_wrapper.__name__ = func.__name__ return arg_wrapper return f_wrapper

questions?

code samples @http://github.com/

mdomans/europython2010

follow me

twitter: mdomansblog: blog.mdomans.com

caching techniques in python, europython2010

function czwartek

wrapper czwartek

key czwartek

time czwartek

computation czwartek

thekeys czwartek

kwargs cache

c cache

Technology

machine learning techniques in python dissertation -...

applying machine learning techniques for caching in next

comparison of different caching techniques for high … ·...

advanced python techniques: decorators

caching techniques for high performance web map services

advanced caching techniques · winter 2006 cse 548 -...

ece 7995 caching and prefetching techniques. locality in...

caching techniques for high performance web map services ·...

accelerating dynamic web content delivery using...

io-lite: a unified buffering and caching system by pai,...

caching techniques for rapid provisioning of virtual...

special forces caching techniques tc 31 29 a

an automatic prefetching and caching...

ece7995 caching and prefetching techniques in computer...

when dynamic becomes static : the next step in web caching...

python programming techniques eliot feibush

working with data in asp.net 2.0 :: caching data with the...

caching techniques of the us army - steve … techniques of...

architecture of ml systems 01 introduction and overview ·...

a proposed model for web proxy caching techniques to