python-csp: bringing occam to python
DESCRIPTION
Bringing OCAAMTRANSCRIPT
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
python-csp: bringing OCCAM to Python
Sarah Mount
Europython 2010
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
1 Ancient history
2 python-csp: features and idiomsProcess creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
3 Using built-in processes
4 Future challenges
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
This talk. . .
is all about the python-csp project. There is a similar projectcalled PyCSP, these will merge over the next few months. Currentcode base is here: http://code.google.com/p/python-csp
Please do contribute bug fixes / issues / code / docs / rants / . . .
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
INMOS B0042 Board c© David May
The Transputer was the original “multicore” machine and was builtto run the OCCAM language – a realisation of CSP.http://www.cs.bris.ac.uk/~dave/transputer.html
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Tony Hoare
Invented CSP, OCCAM, Quicksort and other baddass stuff. Youmay remember him from such talks as his Europython 2009Keynote.
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
OCCAM
OCCAM lives on in it’s current implementation as OCCAM-πhttp://www.cs.kent.ac.uk/projects/ofa/kroc/
It can run on Lego bricks, Arduino boards and other things. LikePython it uses indentation to demark scope. Unlike PythonOCCAM is MOSTLY IN UPPERCASE.
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
CSP and message passingThis next bit will be revision for many of you!
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Most of us think of programs like this
http://xkcd.com/205/
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
This is the old “standard” model!
Multicore will soon take over the world (if it hasn’t already).
Shared memory concurrency / parallelism is hard:
Race hazardsDeadlockLivelockTracking which lock goes with which data and what orderlocks should be used.Operating Systems are old news!
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Message passing
Definition
In message passing concurrency (or parallelism) “everything” is aprocess and no data is shared between processes. Instead, data is“passed” between channels which connect processes together.Think: OS processes and UNIX pipes.
The actor model (similar to Kamaelia) uses asynchronouschannels. Processes write data to a channel and continueexecution.
CSP (and π-calculus) use synchronous channels, orrendezvous, where a process writing to a channel has to waituntil the value has been read by another process beforeproceeding.
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process diagram
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Why synchronous channels?
Leslie Lamport (1978). Time, clocks, and the ordering of events ina distributed system”. Communications of the ACM 21 (7)
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
Hello World!
from csp . c s p p r o c e s s i m p o r t ∗
@ p r o c e s sd e f h e l l o ( ) :
p r i n t ’ H e l l o CSP w o r l d ! ’
h e l l o ( ) . s t a r t ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
Example of process creation: web server
@ p r o c e s sd e f s e r v e r ( host , p o r t ) :
sock = s o c k e t . s o c k e t ( . . . )# Set up s o c k e t sw h i l e True :
conn , addr = sock . a c c e p t ( )r e q u e s t = conn . r e c v (4096) . s t r i p ( )i f r e q u e s t . s t a r t s w i t h ( ’GET ’ ) :
ok ( r e q u e s t , conn ) . s t a r t ( )e l s e :
n o t f o u n d ( r e q u e s t , conn ) . s t a r t ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
Example of process creation: web server
@ p r o c e s sd e f n o t f o u n d ( r e q u e s t , conn ) :
page = ’<h1>F i l e not found</h1> ’conn . send ( r e s p o n s e (404 ,
’ Not Found ’ ,page ) )
conn . shutdown ( s o c k e t .SHUT RDWR)conn . c l o s e ( )r e t u r n
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
Combining processes: in parallel
# With a Par o b j e c t :Par ( p1 , p2 , p3 , . . . pn ) . s t a r t ( )# With s y n t a c t i c s u g a r :p1 // ( p2 , p3 , . . . pn )# RHS must be a s e q u e n c e t y p e .
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
Combining processes: repeating a process sequentially
@ p r o c e s sd e f h e l l o ( ) :
p r i n t ’ H e l l o CSP w o r l d ! ’i f n a m e == ’ m a i n ’ :
h e l l o ( ) ∗ 32 ∗ h e l l o ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
Channels
@ p r o c e s sd e f c l i e n t ( chan ) :
p r i n t chan . r e a d ( )@ p r o c e s sd e f s e r v e r ( chan ) :
chan . w r i t e ( ’ H e l l o CSP wo r l d ! ’ )
i f n a m e == ’ m a i n ’ :ch = Channel ( )Par ( c l i e n t ( ch ) , s e r v e r ( ch ) ) . s t a r t ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
What you need to know about channels
Channels are currently UNIX pipes
This will likely change
Channel rendezvous is “slow” compared with JCSP:200µ s vs 16µ s
This will be fixed by having channels written in C
Because channels are, at the OS level, open “files”, there canonly be a limited number of them (around 300 on mymachine)
This makes me sad :-(
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
Message passing an responsiveness
This isn’t about speed.
Sometimes, you have several channels and reading from themall round-robin may reduce responsiveness if one channel readblocks for a long time.
This is BAD:
@ p r o c e s sd e f f o o ( c h a n n e l s ) :
f o r chan i n c h a n n e l s :p r i n t chan . r e a d ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
ALTing and choice
Definition
ALTing, or non-deterministic choice, selects a channel to read fromfrom those channels which are ready to read from.
You can do this with (only) two channels:
@ p r o c e s sd e f f o o ( c1 , c2 ) :
p r i n t c1 | c2p r i n t c1 | c2
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
ALTing proper (many channels)
@ p r o c e s sd e f f o o ( c1 , c2 , c3 , c4 , c5 ) :
a l t = A l t ( c1 , c2 , c3 , c4 , c5 )# Choose random a v a i l a b l e o f f e rp r i n t a l t . s e l e c t ( )
# Avoid l a s t s e l e c t e d c h a n n e lp r i n t a l t . f a i r s e l e c t ( )
# Choose c h a n n e l w i t h l o w e s t i n d e xp r i n t a l t . p r i s e l e c t ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
Syntactic sugar for ALTing
@ p r o c e s sd e f f o o ( c1 , c2 , c3 , c4 , c5 ) :
gen = A l t ( c1 , c2 , c3 , c4 , c5 )p r i n t gen . n e x t ( )p r i n t gen . n e x t ( )p r i n t gen . n e x t ( )p r i n t gen . n e x t ( )p r i n t gen . n e x t ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
ALTing: when you really, really must return right now
@ p r o c e s sd e f f o o ( c1 , c2 , c3 ) :
# S k i p ( ) w i l l / a l w a y s / complete# a r e a d i m m e d i a t e l ygen = A l t ( c1 , c2 , c3 , S k i p ( ) )p r i n t gen . n e x t ( ). . .
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
Channel poisoning
Definition
Idiomatically in this style of programming most processes containinfinite loops. Poisoning a channel asks all processes connected tothat channel to shut down automatically. Each process which hasa poisoned channel will automatically poison any other channels itis connected to. This way, a whole graph of connected processescan be terminated, avoiding race conditions.
@ p r o c e s sd e f f o o ( c h a n n e l ) :
. . .c h a n n e l . p o i s o n ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
A bigger example: Mandelbrot generator
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
@ p r o c e s sd e f main ( IMSIZE ) :
c h a n n e l s = [ ]# One p r o d u c e r + c h a n n e l f o r each
image column .f o r x i n r an ge ( IMSIZE [ 0 ] ) :
c h a n n e l s . append ( Channel ( ) )p r o c e s s e s . append ( mande lbrot ( x ,
. . . c h a n n e l s [ x ] ) )p r o c e s s e s . i n s e r t ( 0 , consume ( IMSIZE ,
c h a n n e l s ) )mandel = Par (∗ p r o c e s s e s )mandel . s t a r t ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
@ p r o c e s sd e f mande lbrot ( xcoord , cout ) :
# Do some maths h e r ecout . w r i t e ( xcoord , co lumn data )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process creationParallel, sequential and repeated processesCommunication via channel read / writesMore channels: ALTing and choiceMore channels: poisoningProducer / consumer or worker / farmer models
@ p r o c e s sd e f consumer ( c i n s ) :
# I n i t i a l i s e PyGame . . .gen = l e n ( c i n s ) ∗ A l t (∗ c i n s )f o r i i n ran ge ( l e n ( c i n s ) ) :
xcoord , column = gen . n e x t ( )# B l i t t h a t column to s c r e e n
f o r e v e n t i n pygame . e v e n t . g e t ( ) :i f e v e n t . t y p e == pygame . QUIT :
f o r c h a n n e l i n c i n s :c h a n n e l . p o i s o n ( )
pygame . q u i t ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Oscilloscope
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
@ p r o c e s sd e f O s c i l l o s c o p e ( i n c h a n ) :
# I n i t i a l i s e PyGameydata = [ ]w h i l e not QUIT :
ydata . append ( i n c h a n . r e a d ( ) )ydata . pop ( 0 )# B l i t data to s c r e e n . . .
# Deal w i t h e v e n t s
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
from csp . b u i l t i n s i m p o r t S i nd e f t r a c e s i n ( ) :
chans = Channel ( ) , Channel ( )Par ( G e n e r a t e F l o a t s ( chans [ 0 ] ) ,
S i n ( chans [ 0 ] , chans [ 1 ] ) ,O s c i l l o s c o p e ( chans [ 1 ] ) ) . s t a r t ( )
r e t u r n
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Process diagram
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
d e f t r ac e m ux ( ) :chans = [ Channel ( ) f o r i i n ra ng e ( 6 ) ]par = Par ( G e n e r a t e F l o a t s ( chans [ 0 ] ) ,
D e l t a 2 ( chans [ 0 ] , chans [ 1 ] ,chans [ 2 ] ) ,
Cos ( chans [ 1 ] , chans [ 3 ] ) ,S i n ( chans [ 2 ] , chans [ 4 ] ) ,Mux2( chans [ 3 ] , chans [ 4 ] ,
chans [ 5 ] ) ,O s c i l l o s c o p e ( chans [ 5 ] ) )
par . s t a r t ( )
Sarah Mount python-csp: bringing OCCAM to Python
Ancient historypython-csp: features and idioms
Using built-in processesFuture challenges
Wiring
Sarah Mount python-csp: bringing OCCAM to Python