semantics of minimally synchronous parallel ml myrto arapini, frdric loulergue, frdric gava and...
DESCRIPTION
IntroductionTRANSCRIPT
![Page 1: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/1.jpg)
Semantics of Minimally Synchronous Parallel ML
Myrto Arapini, Frédéric Loulergue, Frédéric Gava and
Frédéric DabrowskiLACL, Paris, France
![Page 2: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/2.jpg)
F. Gava SNPD 2003 2
Outline Introduction Pure functional minimally
synchronous parallel programming High level semantics Distributed evaluation Conclusions and future work
![Page 3: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/3.jpg)
Introduction
![Page 4: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/4.jpg)
F. Gava SNPD 2003 4
Our previous work:Bulk Synchronous Parallelism + Functional Programming = BSML Bulk Synchronous Parallelism :
Scalability Portability Simple cost model
Functional Programming : High level features (higher order
functions, pattern matching, concrete types, etc.)
Programs proofs Safety of the environment
![Page 5: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/5.jpg)
F. Gava SNPD 2003 5
What is MSPML ?Why MSPML ? = Minimally Synchronous Parallel ML BSML without synchronization barriers
Comparison of efficiency BSML/MSPML Further extensions restricted or impossible
in BSML (divide-and-conquer, nesting of parallel values)
Basis for an additional level to BSML for Grid computing: BSML on each nodes (clusters) + MSPML for coordination
![Page 6: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/6.jpg)
F. Gava SNPD 2003 7
The MSPML Library Parallel ML library for the
Objective Caml language operations on a parallel data
structureAbtract type: par
access to the machine parameters:p: unit int
p() = number of processes
![Page 7: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/7.jpg)
F. Gava SNPD 2003 8
Creation of Parallel Vectors mkpar: (int )par
(mkpar f )f (p-1)…(f 1)(f 0)
![Page 8: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/8.jpg)
F. Gava SNPD 2003 9
Pointwise Parallel Application apply: ( ) par par par
apply
=
fp-1…f1f0
vp-1…v1v0
fp-1 vp-1…f1 v1f0 v0
![Page 9: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/9.jpg)
F. Gava SNPD 2003 10
Example (1) let replicate x = mkpar(fun pid->x)replicate: ’a -> ’a par
(replicate 5)
5…55
![Page 10: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/10.jpg)
F. Gava SNPD 2003 11
Example (2) let parfun f v = apply (replicate f) vparfun: (’a->’b)->’a par->’b par
parfun (fun x->x+1) parallel_vectorf
vp-1…v1v0
f…ffapply
= (f vp-1 )…(f v1 )(f v0 )
![Page 11: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/11.jpg)
F. Gava SNPD 2003 12
Communication Operation: get
get: parint par par get
vp-1…v1v0
ip-1…i1i0(=1)
vip-1…vi1
vi0
=
![Page 12: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/12.jpg)
F. Gava SNPD 2003 13
Example (3)let bcast_direct root parallel_vector =if not(within_bounds root)then raise Bcast_Error else get parallel_vector
(replicate root)bcast_direct: int->’a par->’a par
vp-1…v1v0
vroot…vrootvroot
(bcast_direct root
=
)
![Page 13: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/13.jpg)
F. Gava SNPD 2003 14
Global Conditionalif parallel_vector at n then … else
…if
at n then E1 else E2= E1
… true fp-1…b1b0
n
![Page 14: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/14.jpg)
F. Gava SNPD 2003 15
Implementation of MSPML MSPML v 0.05 :
F. Loulergue Library for Ocaml (uses threads and
TCP/IP) October 2003 http://mspml.free.fr
![Page 15: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/15.jpg)
High level semantics
![Page 16: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/16.jpg)
F. Gava SNPD 2003 17
Terms Functional semantics:
can be evaluated sequentially or in parallel Terms :
![Page 17: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/17.jpg)
F. Gava SNPD 2003 18
Values and judgements Values:
Judgement:e v
Term « e » evaluates to value « v »
![Page 18: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/18.jpg)
F. Gava SNPD 2003 19
Some rules
![Page 19: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/19.jpg)
Distributed evaluation
![Page 20: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/20.jpg)
F. Gava SNPD 2003 21
Informal presentation (1) MSPML programs seen as SPMD programs The parallel machine runs p copies of the
same MSPML program Rules for local reduction + rule for
communication For example at processor i the expression
(mkpar f) is evaluated to (f i)
![Page 21: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/21.jpg)
F. Gava SNPD 2003 22
Informal presentation (2)Proc. 0 1 2
empty 0,v’
Local computation
Communication environment
get v 1
0,v
request 0 1v’A BIT LATER0,v’’
![Page 22: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/22.jpg)
F. Gava SNPD 2003 23
Informal presentation (3)Proc. 0 1 2
empty 0,v’0,v’’1,w’2,w’’
Local computation
Communication environment
0,v0,v’1,w
request 2 0Not Ready !
!
![Page 23: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/23.jpg)
F. Gava SNPD 2003 24
Terms and judgments New term: request i j
the processor asks the value stored at processor j during the ith step
Step = each step is ended by a call to get Judgment:
At process i, the term ed with communication environment Ec is evaluate to e’d with new communication environment E’c
![Page 24: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/24.jpg)
F. Gava SNPD 2003 25
Local computation rules
![Page 25: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/25.jpg)
F. Gava SNPD 2003 26
Communication rule
![Page 26: Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frdric Loulergue, Frdric Gava and Frdric Dabrowski LACL, Paris, France](https://reader033.vdocuments.us/reader033/viewer/2022051104/5a4d1b6e7f8b9ab0599b42e1/html5/thumbnails/26.jpg)
F. Gava SNPD 2003 27
Conclusion and Future Work Conclusion
Minimally Synchronous Parallel ML:Functional semantics & Deadlock free
Two semantics: High level semantics (programmer’s view) Distributed evaluation (implementation’s view)
Implementation Future Work
MPI Implementation Comparison with BSMLlib Extensions: parallel composition, etc.