development of correct-by-construction functional parallel ... ·...

Development of Correct-by-ConstructionFunctional Parallel Programs

ACMSAC 2016 Tutorial

Frédéric Loulergue

Univ Orléans, LIFO, Research Group LMV, Orléans, France

http://frederic.loulergue.eu/sac2016

April 4, 2016


Our Goal

To ease the development of correctand verified parallel programswith predictable performances

using theories and tools to allowa user to develop an applicationby using building blocks and

implementing short programs satisfyingconditions easily or automatically proved

F. Loulergue Development of Correct-by-Construction Functional Parallel Programs April 4, 2016 2 / 89

Parallel Programming

AutomaticParallelization

Structured Parallelism

I Algorithmic Skeletons

I BridgingModels

I Declarative Parallel Programming

I . . .

Concurrent &DistributedProgramming


Parallel Programming – BridgingModels I

BridgingModelI Leslie Valiant in his 1990 CACMpaper

‘‘A BridgingModel for Parallel Computation”http://dx.doi.org/10.1145/79173.79181The von Neumannmodel is the connecting bridge that enables programsfrom the diverse and chaotic world of software to run efficientby onmachines from the diverse and chaotic world of hardware

I Valiant’s proposal: Bulk Synchronous Parallelism (BSP)I Othermodels: LogP and variants, BSP variants, . . .


http://dx.doi.org/10.1145/79173.79181

Parallel Programming – BridgingModels II

Research on BSP90’ by Valiant &McColl

ThreemodelsI abstract architectureI executionmodelI cost model

BSP computerI p processor / memory pairs

(of speed r)I a communication network (of

speed g)I a global synchronisation unit

(of speed L)

Executionmodel

Cost modelT(s) = max0i<p wi + h ⇥ g + Lwhere h = max0i<p{h+

i , h�i }


Parallel Programming – BridgingModels III

ApplicationsI scientific computation [5]I genetic algorithms [6]I genetic programming [10]I neural networks [21]I parallel databases [1]I parallel constraints solvers [14]I . . .


Parallel Programming – BridgingModels IV

Programming

I BSPlib for C and variantsI Bulk Synchronous Parallel ML (BSML)

I global view of programsI parallel data structure: parallel vectorI 4 parallel operationsI pure functional semanticsI library for OCaml (+MPI)


Parallel Programming – Algorithmic Skeletons

Algorithmic SkeletonsI Coined byMurray Cole in

Algorithmic Skeletons: StructuredManagementof Parallel Computation, MIT Press, 1989http://homepages.inf.ed.ac.uk/mic/Pubs/skeletonbook.ps.gz

I Popular skeletons: Google’sMapReduce

Skeletal ParallelismI Skeleton = pattern of a parallel algorithm

familiar sequential semanticsI Program = composition of skeletons


http://homepages.inf.ed.ac.uk/mic/Pubs/skeletonbook.ps.gz


Algorithmic SkeletonsI Coined byMurray Cole in

Algorithmic Skeletons: StructuredManagementof Parallel Computation, MIT Press, 1989http://homepages.inf.ed.ac.uk/mic/Pubs/skeletonbook.ps.gz

I Popular skeletons: Google’sMapReduce

Skeletal ParallelismI Skeleton = higher-order function implemented in parallel

familiar sequential semanticsI Program = composition of skeletons


http://homepages.inf.ed.ac.uk/mic/Pubs/skeletonbook.ps.gz


Libraries of Algorithmic Skeletons

I For C++: SkeToa, OSLb, Muesli, QUAFF, . . .I For C: eSkel, SKElibI For Java: Lithium,Muskel, Calcium, . . .I For functional languages:

I OCaml: OCamlP3L, ParmapI Erlang: SkelI Haskell: HaskSkel, Edenskeletons

ahttp://sketo.ipl-lab.orgbhttp://traclifo.univ-orleans.fr/OSL

Algorithmic Skeletons TheoryI List homomorphisms for parallel programming (Cole 1993)I Many further developments in particular in Tokyo


http://sketo.ipl-lab.org

http://traclifo.univ-orleans.fr/OSL


An Example: Variance

1n

n�1X

k=0

(xk � 1n

n�1X

k=0

xk)2

Variance as anOSL Program

double avg = reduce(plus<double>(), x) / x.getSize();

double variance =

reduce(plus<double>(),

map(bind(minus<double>(),avg, _2), x));


ProgramCorrectness

A posteriori verificationWrite the program then try to prove its correctness:

I verification condition generator + proversI interactive theorem proversI softwaremodel checkingI . . .

Correctness by contruction

I BmethodI BirdMeertens Formalism

(theory of lists, . . . )I . . .

Specificationor naive implementation

Efficient implementation

Program transformationbased on an equational theory


Increasing Confidence

Usual UsageI Pen-and-paper program transformationI From the last form: hand-written C++ code for a skeleton library

(mostly in C, C++, Java)

Potential ProblemsI Pen-and-paper transformationmay be erroneousI The C++ codemay not be equivalent to the last formI The skeleton librarymay contain bugs

) Software assistance and verification needed


Overview of our approach

Programs correctby construction

I Write naive correctprograms (specification)

I Apply programtransformationtechniques

I To obtain equivalentefficient programs

I That are automaticallyparallelised

ProgramSpecificationSpecification

Building Blocks

ImplementationImplementation

SpecificationSpecification

TheoryTheory


derivation

Host Language & Parallel Library


Native CodeNative Code

compilation

Operating SystemOperating System Native CodeNative CodeNative CodeNative Code Compiled LibrariesCompiled Libraries


Usage of the Coq proof assistant

Coq proof assistant OCamlThe Bulk SynchronousParallel ML library

SpecificationsTransformations

Automaticparallelisation

Extraction toOCaml + BSML


The Coq Proof Assistant I

ACMSIGPLAN Software Award 2013The Coq proof assistant provides a rich environment forinteractive development of machine-checked formal reasoning.Coq is having a profound impact on research on programminglanguages and systems [. . . ] It has beenwidely adopted as aresearch tool by the programming language researchcommunity [. . . ] Last but not least, these successes have helpedto spark a wave of widespread interest in dependent typetheory, the richly expressive core logic onwhich Coq is based.

[. . . ] The Coq team continues to develop the system, bringingsignificant improvements in expressiveness and usability witheach new release.

In short, Coq is playing an essential role in our transition to anew era of formal assurance inmathematics, semantics, andprogram verification.


The Coq Proof Assistant II

FoundationsI Calculus of inductive constructionsI Curry-Howard correspondance


Curry-Howard Correspondance

Natural Deduction

(v)A 2 �

� ` A

(i)�,A ` B

� ` A ! B

(a)� ` A ! B � ` A

� ` B

Simply Typed �-Calculus

(V)x : A 2 �

� ` x : A

(L)�, x : A ` e : B

� ` (�x:A.e) : A ! B

(A)� ` e : A ! B � ` e0 : A

� ` (e e0) : B



Natural Deduction – Example 1

(i)

(i)

(i)

(i)

(a)

(v)A ! C 2 �

� ` A ! C

(v)A 2 �

� ` A

� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! C

A, B ` (A ! C) ! (B ! C) ! C

A ` B ! (A ! C) ! (B ! C) ! C

` A ! B ! (A ! C) ! (B ! C) ! C




(i)

(i)

(i)

(i)

(a)

(v)A ! C 2 �

� ` A ! C

(v)A 2 �

� ` A

� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! C

A, B ` (A ! C) ! (B ! C) ! C

A ` B ! (A ! C) ! (B ! C) ! C` A ! B ! (A ! C) ! (B ! C) ! C




(i)(i)

(i)

(i)

(a)

(v)A ! C 2 �

� ` A ! C

(v)A 2 �

� ` A

� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! C

A, B ` (A ! C) ! (B ! C) ! CA ` B ! (A ! C) ! (B ! C) ! C

` A ! B ! (A ! C) ! (B ! C) ! C




(i)(i)

(i)

(i)

(a)

(v)A ! C 2 �

� ` A ! C

(v)A 2 �

� ` A

� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! CA, B ` (A ! C) ! (B ! C) ! C

A ` B ! (A ! C) ! (B ! C) ! C` A ! B ! (A ! C) ! (B ! C) ! C




(i)(i)

(i)(i)

(a)

(v)A ! C 2 �

� ` A ! C

(v)A 2 �

� ` A

� ⌘ A, B, A ! C, B ! C ` CA, B, A ! C ` (B ! C) ! C

A, B ` (A ! C) ! (B ! C) ! CA ` B ! (A ! C) ! (B ! C) ! C

` A ! B ! (A ! C) ! (B ! C) ! C




(i)(i)

(i)(i)

(a)

(v)A ! C 2 �

� ` A ! C

(v)A 2 �

� ` A� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! CA, B ` (A ! C) ! (B ! C) ! C

A ` B ! (A ! C) ! (B ! C) ! C` A ! B ! (A ! C) ! (B ! C) ! C




(i)(i)

(i)(i)

(a)(v)

A ! C 2 �

� ` A ! C(v)

A 2 �

� ` A� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! CA, B ` (A ! C) ! (B ! C) ! C

A ` B ! (A ! C) ! (B ! C) ! C` A ! B ! (A ! C) ! (B ! C) ! C




(i)

(i)

(i)

(i)

(a)

(v)B ! C 2 �

� ` B ! C

(v)B 2 �

� ` B

� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! C

A, B ` (A ! C) ! (B ! C) ! C

A ` B ! (A ! C) ! (B ! C) ! C

` A ! B ! (A ! C) ! (B ! C) ! C




(i)

(i)

(i)

(i)

(a)

(v)B ! C 2 �

� ` B ! C

(v)B 2 �

� ` B

� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! C

A, B ` (A ! C) ! (B ! C) ! C

A ` B ! (A ! C) ! (B ! C) ! C` A ! B ! (A ! C) ! (B ! C) ! C




(i)(i)

(i)

(i)

(a)

(v)B ! C 2 �

� ` B ! C

(v)B 2 �

� ` B

� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! C

A, B ` (A ! C) ! (B ! C) ! CA ` B ! (A ! C) ! (B ! C) ! C

` A ! B ! (A ! C) ! (B ! C) ! C




(i)(i)

(i)

(i)

(a)

(v)B ! C 2 �

� ` B ! C

(v)B 2 �

� ` B

� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! CA, B ` (A ! C) ! (B ! C) ! C

A ` B ! (A ! C) ! (B ! C) ! C` A ! B ! (A ! C) ! (B ! C) ! C




(i)(i)

(i)(i)

(a)

(v)B ! C 2 �

� ` B ! C

(v)B 2 �

� ` B

� ⌘ A, B, A ! C, B ! C ` CA, B, A ! C ` (B ! C) ! C

A, B ` (A ! C) ! (B ! C) ! CA ` B ! (A ! C) ! (B ! C) ! C

` A ! B ! (A ! C) ! (B ! C) ! C




(i)(i)

(i)(i)

(a)

(v)B ! C 2 �

� ` B ! C

(v)B 2 �

� ` B� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! CA, B ` (A ! C) ! (B ! C) ! C

A ` B ! (A ! C) ! (B ! C) ! C` A ! B ! (A ! C) ! (B ! C) ! C




(i)(i)

(i)(i)

(a)(v)

B ! C 2 �

� ` B ! C(v)

B 2 �

� ` B� ⌘ A, B, A ! C, B ! C ` C

A, B, A ! C ` (B ! C) ! CA, B ` (A ! C) ! (B ! C) ! C

A ` B ! (A ! C) ! (B ! C) ! C` A ! B ! (A ! C) ! (B ! C) ! C



�-calculus: find a termwith the given type

(L)

(L)

(L)

(L)

(A)

(V)f:A ! C 2 �

� ` f:A ! C

(V)x:A 2 �

� ` x:A

� ⌘ x:A, y:B, f:A ! C, g:B ! C ` (f x) : C

x:A, y:B, f:A ! C ` �g:B!C.(f x) : (B ! C) ! C

x:A, y:B ` �f:A!C.�g:B!C.(f x) : (A ! c) ! (B ! C) ! C

x:A ` �y:B.�f:A!C.�g:B!C.(f x) : B!(A!C)!(B!C)!C

` ? : A!B!(A!C)!(B!C)!C

�x:A.�y:B.�f:A!C.�g:B!C.(f x)is a way to encode the proof tree of

A!B!(A!C)!(B!C)!C




(L)

(L)

(L)

(L)

(A)

(V)f:A ! C 2 �

� ` f:A ! C

(V)x:A 2 �

� ` x:A

� ⌘ x:A, y:B, f:A ! C, g:B ! C ` (f x) : C


x:A, y:B ` �f:A!C.�g:B!C.(f x) : (A ! c) ! (B ! C) ! C

x:A ` ? : B!(A!C)!(B!C)!C` �x:A. ? : A!B!(A!C)!(B!C)!C


A!B!(A!C)!(B!C)!C




(L)(L)

(L)

(L)

(A)

(V)f:A ! C 2 �

� ` f:A ! C

(V)x:A 2 �

� ` x:A

� ⌘ x:A, y:B, f:A ! C, g:B ! C ` (f x) : C


x:A, y:B ` ? : (A ! c) ! (B ! C) ! Cx:A ` �y:B.? : B!(A!C)!(B!C)!C

` �x:A.�y:B. ? : A!B!(A!C)!(B!C)!C


A!B!(A!C)!(B!C)!C




(L)(L)

(L)

(L)

(A)

(V)f:A ! C 2 �

� ` f:A ! C

(V)x:A 2 �

� ` x:A

� ⌘ x:A, y:B, f:A ! C, g:B ! C ` (f x) : C

x:A, y:B, f:A ! C ` ? : (B ! C) ! Cx:A, y:B ` �f:A!C. ? : (A ! c) ! (B ! C) ! Cx:A ` �y:B.�f:A!C. ? : B!(A!C)!(B!C)!C

` �x:A.�y:B.�f:A!C. ? : A!B!(A!C)!(B!C)!C


A!B!(A!C)!(B!C)!C




(L)(L)

(L)(L)

(A)

(V)f:A ! C 2 �

� ` f:A ! C

(V)x:A 2 �

� ` x:A

� ⌘ x:A, y:B, f:A ! C, g:B ! C ` ? : Cx:A, y:B, f:A ! C ` �g:B!C. ? : (B ! C) ! C

x:A, y:B ` �f:A!C.�g:B!C. ? : (A ! c) ! (B ! C) ! Cx:A ` �y:B.�f:A!C.�g:B!C. ? : B!(A!C)!(B!C)!C

` �x:A.�y:B.�f:A!C.�g:B!C. ? : A!B!(A!C)!(B!C)!C


A!B!(A!C)!(B!C)!C




(L)(L)

(L)(L)

(A)

(V)f:A ! C 2 �

� ` f:A ! C

(V)x:A 2 �

� ` x:A� ⌘ x:A, y:B, f:A ! C, g:B ! C ` (f x) : C

x:A, y:B, f:A ! C ` �g:B!C.(f x) : (B ! C) ! Cx:A, y:B ` �f:A!C.�g:B!C.(f x) : (A ! c) ! (B ! C) ! Cx:A ` �y:B.�f:A!C.�g:B!C.(f x) : B!(A!C)!(B!C)!C

` �x:A.�y:B.�f:A!C.�g:B!C.(f x) : A!B!(A!C)!(B!C)!C


A!B!(A!C)!(B!C)!C




(L)(L)

(L)(L)

(A)(V)

f:A ! C 2 �

� ` f:A ! C(V)

x:A 2 �

� ` x:A� ⌘ x:A, y:B, f:A ! C, g:B ! C ` (f x) : C

x:A, y:B, f:A ! C ` �g:B!C.(f x) : (B ! C) ! Cx:A, y:B ` �f:A!C.�g:B!C.(f x) : (A ! c) ! (B ! C) ! Cx:A ` �y:B.�f:A!C.�g:B!C.(f x) : B!(A!C)!(B!C)!C

` �x:A.�y:B.�f:A!C.�g:B!C.(f x) : A!B!(A!C)!(B!C)!C


A!B!(A!C)!(B!C)!C


Curry-Howard Isomomorphism

For all formula there exists a proof of this formula in natural deduction ifand only if there exists a �-term that has this formula as type.

I Theorem statement, TypeI Proof, Program


Coq in practice

I Functional programming languageI Rich type system: allow to express logical propertiesI Language for building proofs (ie proof terms)I Program extraction


Previous examples in Coq

The Proof General mode for Emacs . . .



. . .or the CoqIDE



We open the file Introduction.v1:

1available at http://frederic.loulergue.eu/sac2016F. Loulergue Development of Correct-by-Construction Functional Parallel Programs April 4, 2016 25 / 89



We start to feed Coqwith the commands:



We state a lemma and enter the interactive proof mode:



The tactic intro “apply” the (i) rule:



The context is now similar to �:



We apply rule (a) by naming the implication part:

and so nowwe have only to deal with A . . .



. . . that is an assumption, we use rule (v):

“No more subgoals”⌘ proof done⌘ �-term built



Qed typechecks the term against the lemma statement:



Second version, we domultiple intro:



and apply HBC instead of apply HAC:



Print t. prints the term t:

It is the �-termwe constructed “by hand”



The �-term for the second proof is:



We could give directly the proof as a �-term:



. . .or use Coqmore powerful tactics:


Interactive Sessions

Remaining of the TutorialI Demo-like using an interactive session of CoqI The VMprovides all the tools:

I to have the same interactive session on yourmachine,I to do the proposed exercises.

The VirtualMachineI http://frederic.loulergue.eu/sac2016I USBKey



Sommaire

1 Functional Programming in Coq

2 Proofs in Coq

3 Theory of Lists and Parallelisation in CoqAnOverview of BirdMeertens FormalismTheory of Lists in CoqMaximumPrefix SumAutomatic Parallelization

4 Conclusion

5 Bibliography


Functional Programming in Coq

Data Structures, Values and FunctionsI Inductive TypesI PatternMatchingI PolymorphismI Recursive FunctionsI How toDeal with Partial Functions

ModularityI Modules, ParametricModulesI Type Classes

Extraction

Interactive SessionI Programming.v


Sommaire


2 Proofs in Coq


4 Conclusion

5 Bibliography


Proofs in Coq

StatementsI Quantification : forall, existsI Inductive Predicates

TacticsI intros

I intros (with introductionpatterns)

I apply

I rewrite

I autowith . . .using . . .I induction as

I destruct as

I omega

TacticalsI composition ;

I now

Dependant Pairs andProgram

Interactive SessionProofs.v


Sommaire


2 Proofs in Coq


4 Conclusion

5 Bibliography


BirdMeertens Formalism

John Backus Turing Award 1977Associated with the functional style of programming is an algebra of programs whosevariables range over programs and whose operations are combining forms. Thisalgebra can be used to transform programs and to solve equations whose “unknowns”are programs in much the same way one transforms equations in high school algebra.

Bird andMeertens [4, 18]I put this approach into practice with the formalism that took their

namesI also called Squiggol


Join-lists

Finite sequence of values having the same typeI empty list [],I singleton list [a] (for an element a),I concatenation x++ y of two lists x and y.

Properties of ‘constructors’I [] unit for++,I ++ associative operation.

map

map f [ ] = [ ]map f [a] = [f a]map f (x++ y) = (map f x)

++ (map f y)

reducereduce � [ ] = i�reduce � [a] = [a]reduce � (x++ y) = (reduce � x)

�(reduce � y)


Lists and Parallelism (distributedmemory)

Usual List

[x0; . . . ; xl1�1; . . . ; xli�1; . . . ; xli�1; . . . xlp�1

; . . . ; xlp�1]

Distributed ListProcessor 1 . . . Processor i . . . Processor p

[a0; . . . ; an1�1] ++ . . .++ [ani�1 ; . . . ; ani�1] ++ . . .++ [anp�1 ; . . . ; anp�1]


Homomorphic Functions

A function h is�-homomorphicif for all lists x and y,

h (x++ y) = (h x) � (h y) (1)

for a binary operation�.

(img(h), �, h []) is a monoid

a = h x = h([] ++ x) = (h []) � (h x) = i� � a

a = h x = h(x++ []) = (h x) � (h []) = a � i�� is associative . . .

Variant with an additional condition

h : A ! B(B,�, i�) est unmonoïde

Notationh = L�, fM


Homomorphisms andData Parallelism

Theorem (First homomorphism theorem)If h is�-homomorphic then h = (reduce�) � (map f).

Processeur1 . . . Processeuri . . . Processeurp

h�

[a0; . . . ; an1�1] ++ . . . ++ [ani�1 ; . . . ; ani�1] ++ . . . ++ [anp�1 ; . . . ; anp�1]�

= { map phase }

reduce��

[f a0; . . . ; f an1�1] ++ . . . ++ [f ani�1 ; . . . ; f ani�1] ++ . . . ++ [f anp�1 ; . . . ; f anp�1]�

= { local reduce phase }

L n1�1k=0 f ak � . . .

L ni�1k=ni�1

f ak � . . . �L np�1

k=np�1f ak

= { global reduce phase }

L np�1k=0 f ak


Third Homomorphism Theorem (1)

Definition (�-leftwards and�-rightwards functions)A function h is�-leftwards for an operation�, if for every list x and everyelement a,

h ([a] ++ x) = a � h x.

A function h is�-rightwards for an operation�, if for every list x andevery element a, h (x++ [a]) = (h x) � a.

foldr and foldlThe unique function h�-leftwards (resp. �-rightwards), is usuallywritten foldr� e (resp. foldl� e) where e = h [].

Properties of foldr and foldlfoldr � e (x++ y) = foldr � (foldr � e y) x (fr)foldl � e (x++ y) = foldl � (foldl � e x) y (fl)



Theorem (Third homomorphism theorem)Let be h a function,� and⌦ binary operations. If h is�-leftwards and⌦-rightwards, then h is�-homomorphic.

Approach to parallelisationI Third theorem) homomorphicI First theorem) reduce �map

I Then replace by parallel versions of map and reduce

ProblemThe third homomorphism theorem is not constructive



Definition (Weak right inverse)Let be h a function on lists.h0 is aweak right inverse of h iff for every list x,

h x = h(h0(h x)).

Lemma (Existence of a weak right inverse)

For a computable function h whose domain is countable, there exists afunction h0 such that : for all x, h(h0(h x)) = h x.

Proof.h0 may be partial. For compute h0 a, let’s enumerate the elements of thedomain of h and stopwhenwemeet a x such that h x = a and return x.This process terminates for all elements of the image of h, by may notterminate otherwise.



Theorem (Weak third homomorphism theorem)

Let be h a function, h’ a weak right inverse of h,� and⌦ binary operations.If h is�-leftwards and⌦-rightwards, then h is�-homomorphic wherea � b = h((h0 a) ++ (h0 b)).


Sommaire


2 Proofs in Coq


4 Conclusion

5 Bibliography


Type Classes (1)

MonoidClass LeftNeutral ‘(op: B!A!A) (e : B) :={ left_neutral : 8 a, op e a = a }.

ClassRightNeutral ‘(op: A!B!A) (e : B) :={ right_neutral : 8 a, op a e = a }.

ClassNeutral ‘(op: A!A!A) (e : A) :={ neutral_left_neutral :> LeftNeutral op e;neutral_right_neutral :> RightNeutral op e }.

ClassAssociative ‘(op:A!A!A) :={ associative : 8 (x y z: A), op (op x y) z = op x (op y z) }.

ClassMonoid ‘(op : A!A!A) (e : A) :={ monoid_assoc :> Associative op;monoid_neutral :> Neutral op e }.


Type Classes (2)

InstancesProgram Instance plus_O_monoid :Monoid plus 0.NextObligation.constructor. intros. now rewrite plus_assoc.Qed.NextObligation.constructor.� constructor. trivial.� constructor. intros. now rewrite plus_n_O.Qed.

Instance app_nil_monoid (A:Type) : Monoid (@app A) [].Admitted.


Type Classes (3)

Instance ResolutionDefinition reduce ‘(op:A!A!A) ‘{Monoid A op e} (l:list A) : A :=fold_left op l e.

Definition result1 := reduce plus [0;1;2].Eval compute in result1. (⇤ = 3 : nat ⇤)

Definition result2 := reduce (@app Set) [[nat];[bool];[list nat]].Eval compute in result2. (⇤ = [nat; bool; list nat] : list Set ⇤)

Require Import ZArith.

FailDefinition resulte := reduce Zplus ([�1; 0; 1 ])%Z.(⇤ Error: Cannot infer the implicit parameter e of reduce. ⇤)


Type Classes (4)

Available Instances

Print InstancesMonoid.

app_nil_monoid : forall A : Type, Monoid (app (A:=A)) []plus_O_monoid : Monoid plus 0


Homomorphic Function in Coq

A fonction h is�-homomorphicif for all lists x and y,

h (x++ y) = (h x) � (h y) (2)

for a binary operation�.

Coq

ClassHomomorphic ‘(h:list A!B) ‘(op:B!B!B) :={ homomorphic : 8 x y, h (x++y) = op (h x) (h y) }.

List Data StructureI Usual definition of Coq standard libraryI foldr and foldl are List.fold_left and List.fold_right


Homomorphisms in Coq

Modelled as Type ClassesI One class per equation

ClassHomomorphism_f ‘(h : list A!B) ‘(f: A!B) :={homomorphism_f : 8 (a:A), h [a] = f a}.

ClassHomomorphism ‘(h:list A!B) ‘(op: B!B!B) ‘(f:A!B)‘{LMonoid B op e} ‘{Homomorphic A B h op} ‘{Homomorphism_f A B h f} :={homomorphism_nil : h [] = e}.


Homomorphic)monoid

(img(h), �, h []) is a monoid

If a function h:list A!B is�-homomorphic then (img h,�,h[]) is a monoid.

Image of h in CoqI Definition img ‘(h:list A!B) := { b:B | 9 l, h l = b }.

I For P:A!Prop, expression { a:A | P a } is a notation of sig P.I Inductive sig (A:Type) (P:A!Prop) : Type := exist : 8 x : A, P x! sig P.

I . . . but we then prove equality of some proof terms.


First Homomorphism Theorem in Coq

First homomorphism theorem

Definition hom_to_map_reduce {A B:Type}(h:list A!B)‘{H:Homomorphic A B h op} : list A! img h :=(reduce op) � (List.map (fun x)h[x])).

Theorem first_homomorphism_theorem ‘{H:Homomorphic A B h op} :8 l, h l = of_img (hom_to_map_reduce h l).

reduce

Definition reduce ‘(op:A!A!A) ‘{m:Monoid A op e} :=fun l) fold_left op l e.


Third Homomorphism Theorem in Coq

ProblemI In Coq only terminating functions,I Lemma 5 cannot be proved.

Weak third homomorphism theorem

Instance third_homomorphism_theorem_right_inverse ‘{h:list A!B}‘{inv:Right_inverse A B h h’}‘{Hl:Leftwards A B h opl e} ‘{Hr:Rightwards A B h opr e} :Homomorphic h (fun l r)h( (h’ l)++(h’ r))).

with

ClassRight_inverse ‘(h:list A!B)(h’:B!list A) := {right_inverse: 8 l, h l = h(h’(h l))}.


Sommaire


2 Proofs in Coq


4 Conclusion

5 Bibliography


AnApplication: MaximumPrefix Sum (1)

mpsmps [1;2;�1;2;�1;�1;3;�4] = 5

First versionmps = maximum � (map sum) � prefix.

Third homomorphism theorem?I mps([a] ++ x) = 0 " (a+mps x)I mps(x++ [a]) = ?



Tupling(f4g) x = (f x, g x)

ms = mps4sumI �-leftwards with a � (bm, bs) = (0 " (a+ bm), a+ bs)I ⌦-rightwards with (am, as) ⌦ b = (am " (as + b), as + b)

Right inverse

ms0(m, s) = [m; s � m]

,ms(ms0(ms x)) = ms(ms0(mps x, sum x))

= (mps4sum) [mps x; (sum x) � (mps x)]= (mps [mps x; (sum x) � (mps x)], sum x)= (mps x, sum x) as, by construction, sum x mps x



Tupling(f4g) x = (f x, g x)

ms = mps4sumI �-leftwards with a � (bm, bs) = (0 " (a+ bm), a+ bs)I ⌦-rightwards with (am, as) ⌦ b = (am " (as + b), as + b)

Right inverse

ms0(m, s) = [m; s � m] ,ms(ms0(ms x)) = ms(ms0(mps x, sum x))

= (mps4sum) [mps x; (sum x) � (mps x)]= (mps [mps x; (sum x) � (mps x)], sum x)= (mps x, sum x) as, by construction, sum x mps x



By the weak third homomorphism theorem,ms is L�, fMwith :8>>>>>><

>>>>>>:

f a = (mps [a], sum [a])

= (0 " (a+mps []), a)= (0 " a, a)

(am, as) � (bm, bs) = ms(ms0 (am, as) ++ms0 (bm, bs))= ms [am; as � am; bm; bs � bm])= (mps [am; as � am; bm; bs � bm], as + bs)= (0 " am " (as + bm), as + bs)




>>>>>>:

f a = (mps [a], sum [a]) = (0 " (a+mps []), a)

= (0 " a, a)(am, as) � (bm, bs) = ms(ms0 (am, as) ++ms0 (bm, bs))

= ms [am; as � am; bm; bs � bm])= (mps [am; as � am; bm; bs � bm], as + bs)= (0 " am " (as + bm), as + bs)




>>>>>>:

f a = (mps [a], sum [a]) = (0 " (a+mps []), a)= (0 " a, a)





>>>>>>:


(am, as) � (bm, bs) = ms(ms0 (am, as) ++ms0 (bm, bs))

= ms [am; as � am; bm; bs � bm])= (mps [am; as � am; bm; bs � bm], as + bs)= (0 " am " (as + bm), as + bs)




>>>>>>:


(am, as) � (bm, bs) = ms(ms0 (am, as) ++ms0 (bm, bs))= ms [am; as � am; bm; bs � bm])

= (mps [am; as � am; bm; bs � bm], as + bs)= (0 " am " (as + bm), as + bs)




>>>>>>:


(am, as) � (bm, bs) = ms(ms0 (am, as) ++ms0 (bm, bs))= ms [am; as � am; bm; bs � bm])= (mps [am; as � am; bm; bs � bm], as + bs)

= (0 " am " (as + bm), as + bs)




>>>>>>:




MaximumPrefix Sum in Coq (1)

First definitionDefinitionmps_spec : list t! t := maximum �’ (map sum) �’’ prefix.

prefix, sum andmaximum

Fixpoint prefix ‘(xs:list A) :=match xswith| []) [[]]| x::xs) []::(map (cons x) (prefix xs))end.Definition sum : list t! t := reduce addDefinitionmaximum : 8 (l:list t), NonEmpty l! t := NE.reducemax.

Type class for non-empty listsClassNonEmpty {A:Type} (l:list A) := { non_emptiness : l<> [] }.



ms_spec

Definition tupling ‘(f:A!B)‘(g:A!C) := fun x) (f x, g x).Definitionms_spec := tuplingmps_spec sum.

ms_spec is opl-leftwards and opr-rightwards

Definition opl (a:t) (b:t⇤t) : t⇤t := ( max 0 (a+fst b), a+(snd b) ).Definition opr (a:t⇤t) (b:t) : t⇤t := ( max (fst a) ((snd a)+b), (snd a)+b).Instancems_lefttwards : Leftwardsms_spec opl (0,0). Proof. (⇤ omitted ⇤)Qed.Instancems_rightwards : Rightwardsms_spec opr (0,0). Proof. (⇤ omitted ⇤)Qed.

ms’ is a weak right inverse of ms_spec

Definitionms’ (p:t⇤t) := let (m,s) := p in [ m; s +�m].Program Instancems_right_inverse : Right_inversems_specms’.Proof. (⇤ omis ⇤)Qed.



The third homomorphism theorem can be applied

Instancems_homomorphic :Homomorphic ms_spec (fun l r)ms_spec(ms’ l ++ms’ r)) :=third_homomorphism_theorem_right_inverse.

Simplifications

ClassOptimised_op ‘(h:list A!B)‘{H:Homomorphic A B h op} :={optimised_op_sig: { op’:(img h)!(img h)!B |8 a b, op’ a b = op (‘ a) (‘ b) }

}.

+ taken into account in the first homomorphism theorem


Sommaire


2 Proofs in Coq


4 Conclusion

5 Bibliography


Automatic Parallelization

ModuleMPS_Parallel(Bsml : PRIMITIVES).

(⇤Modules Omitted ⇤)

Definition par_ms := Eval simpl inParallel.left_parallel (f:=first_hom_thm_fun optimised_ms).

Definition par_mps :=fst � (@proj1_sig (N.t ⇤N.t) _ ) � par_ms.

EndMPS_Parallel.

Print par_ms

par_ms =fun plst : Bsml.par (list t))Map_reduce.mapReducePar f odot plst: Bsml.par (list t)! imgms_spec


Extraction and ExecutionPlongement de BSML dans Coq

module Primitives : PRIMITIVESwith type ↵ par = ↵ Bsml.par =

struct

let bsp p = nat of int (Bsml.bsp p)type ↵ par = ↵ Bsml.parlet mkpar f = Bsml.mkpar (fun i ! f (nat of int i))let apply = Bsml.applylet put vf =parfun (fun f ! compose f int of nat)(Bsml.put(parfun (fun f ! compose f nat of int)vf))

let proj v = compose (Bsml.proj v) int of natend

Coq

BSML

Extraction

Programmes BSMLet

Preuves de correction

Module Type PRIMITIVES.

Parameter bsp p : nat.

Axiom bsp pLtZero : 0 < bsp p.

Parameter par : Type ! Type.

Parameter mkpar :8 f : processor ! A,{ X: par A | 8 i : processor, get X i = f i }.

Parameter apply :8 (B : Type) (vf : par (8 a :A, B)) (vx : par A), . . .

Parameter put :8 (vf : par (processor ! A)), . . .

Parameter proj :8 (v : par A), . . . End PRIMITIVES.

PRIMITIVES

Programmes BSML

PRIMITIVES

J. Tesson, Developpement et preuve de correction de programmes paralleles fonctionnels. LTP 2011 16 / 39


Bulk Synchronous Parallel ML (BSML) in Coq

Parallel vectors

I In Coq: Parameter par: Type! Type.I Informally: h a0 , . . . , ap�1 i

Primitives

Parametermkpar : (processor!A)!par AParameter apply : par(A!B)! par A!parParameter proj : par A!processor!AParameter put : par(processor!A)!par(processor!A)

mkpar f = h f 0 , . . . , f (p � 1) iapply h f0 , . . . , fp�1 i h v0 , . . . , vp�1 i = h f0 v0 , . . . , fp�1 vp�1 iproj h v0 , . . . , vp�1 i = �i.viput h f0 , . . . , fp�1 i = h�j.fj 0 , . . . , �j.fj (p � 1) i


AnOverview of the ParallelizationMechanism (1)

Type correspondance

Repartition des donnees

join partition

Structure de donnees partitionnable

join partition

Structure de donneespartitionnable

join � partition = idpartition � join = ?


with join is surjective



Function correspondance

Parallelisation correcte composable

join

fp

f

join


Class FunCorr‘{ACorr : TypeCorr A Ap join_A}‘{BCorr : TypeCorr B Bp join_B}(f:A!B) (fp:Ap!Bp) := {fun_corr : 8 ap,join_B (fp ap) = f (join_A ap)}.

Variants :I sequential input typesI sequential output types

Instances :I compositionsI “algorithmic skeletons”



map and reduce skeletonsProgramDefinition par_map ‘(f:A!B)‘(v:par(list A)) : par(list B) :=parfun (List.map f) v.ProgramDefinition par_reduce‘(op:A!A!A)‘{m:LMonoid A op e}(v:par(list A)) : A :=reduce op (List.map (proj (parfun (reduce op) v)) processors).

list A

par(list A)

list B

par(list B)par_map f

map f

join join

list A

par(list A)

A

par_reduce op

reduce op

join

ParallelizationDefinition parallel ‘(f:A!B)‘{ACorr : TypeCorr A Ap join_A} ‘{BCorr : TypeCorr B Bp join_B}‘{fCorr : @FunCorr A Ap join_A ACorr B Bp join_B BCorr f fp} : Ap!Bp := fp.


Sommaire


2 Proofs in Coq


4 Conclusion

5 Bibliography


Summary

SummaryI Coq for mechanising the design of programs in the BirdMeertens

Formalism traditionI Automatic parallelisation with type classesI extraction of parametric modules appliqued to parallel

implementations of BSML inOCaml, C andMPII Small: 1900 LoCCoq, 600 LoCOCaml, 120 LoCC

Systematic Development of Programs for Parallel and Cloud Computing

http://traclifo.univ-orleans.fr/svn/SyDPaCC


http://traclifo.univ-orleans.fr/svn/SyDPaCC

To LearnMore about Coq

OnlineI B. C. Pierce et al. Software Foundations,

https://www.cis.upenn.edu/~bcpierce/sfI Ilya Sergey, Programs and Proofs, http://ilyasergey.net/pnp/

BooksI Y. Bertot and P. Castéran, Interactive Theorem Proving and Program

Development. Coq’Art: The Calculus of Inductive Constructions, Series:Texts in Theoretical Computer Science, Springer, 2004

I A. Chlipala, Certified Programming with Dependent Types, MIT Press,2013


https://www.cis.upenn.edu/~bcpierce/sf

http://ilyasergey.net/pnp/

To LearnMore about SyDPaCC

Parallelization of Homomorphism in CoqI Paper: [17]

Generate-Test-and-AggregateI Specifications : generator + tester + agregatorI Interactive Theorem Proving 2014: [11]

Bulk Synchronous Parallel HomomorphismsI Capture a larger class of BSP algorithmsI Papers: [12, 16]


CollaborativeWorkwith avec (lexicographic order)

I Dr. Frédéric Dabrowski (Université d’Orléans)I Dr. Sylvain Dailler (KUT&Université d’Orléans)I Dr. Kento Emoto (Kyushu University of Technology)I Pr. Zhenjiang Hu (National Institute of Informatics)I Dr. Sylvain Jubertie (Université d’Orléans)I Dr. Hab. Frédéric Gava (Université Paris-Est Créteil)I Dr. Louis Gesbert (OCamlPro)I Hideki Hashimoto (The University of Tokyo)I Dr. Joeffrey Légaux (Université d’Orléans)I Dr. Kiminori Matsuzaki (Kochi University of Technology)I Dr. Virginia Niculescu (Babes-Bolyai University of Cluj-Napoca)I Dr. Thomas Pinsard (Université d’Orléans)I Simon Robillard (Chalmers)I Pr. Masato Takeichi (The University of Tokyo)


Sommaire


2 Proofs in Coq


4 Conclusion

5 Bibliography


Bibliographie I

[1] M. Bamha andM. Exbrayat. Pipelining a Skew-Insensitive Parallel JoinAlgorithm. Parallel Processing Letters, 13(3):317–328, 2003.

[2] Y. Bertot. Coq in a hurry, 2006. http://hal.inria.fr/inria-00001173.

[3] Y. Bertot and P. Castéran. Interactive Theorem Proving and ProgramDevelopment. Springer, 2004. doi:10.1007/978-3-662-07964-5.

[4] R. Bird. The promotion and accumulation strategies in transformationalprogramming. ACM Trans Program Lang Syst, 6(4):487–504, Oct. 1984.doi:10.1145/1780.1781.

[5] R. Bisseling. Parallel Scientific Computation. A Structured Approach using BSPandMPI. Oxford University Press, 2004.

[6] A. Braud and C. Vrain. A parallel genetic algorithm based on the BSPmodel. In Evolutionary Computation and Parallel Processing GECCO& AAAIWorkshop, Orlando (Florida), USA, 1999.

[7] A. Chlipala. An Introduction to Programming and Proving with DependentTypes in Coq. Journal of Formalized Reasoning, 3(2), 2010.doi:10.6092/issn.1972-5787/1978.


http://hal.inria.fr/inria-00001173

http://dx.doi.org/10.1007/978-3-662-07964-5

http://dx.doi.org/10.1145/1780.1781

http://dx.doi.org/10.6092/issn.1972-5787/1978

Bibliographie II

[8] M. Cole. Algorithmic Skeletons: StructuredManagement of ParallelComputation. MIT Press, 1989. Available athttp://homepages.inf.ed.ac.uk/mic/Pubs.

[9] M. Cole. Parallel Programming with List Homomorphisms. ParallelProcessing Letters, 5(2):191–203, 1995.

[10] D. C. Dracopoulos and S. Kent. Speeding up genetic programming: Aparallel BSP implementation. In First Annual Conference on GeneticProgramming. MIT Press, July 1996.

[11] K. Emoto, F. Loulergue, and J. Tesson. A Verified Generate-Test-AggregateCoq Library for Parallel Programs Extraction. In Interactive TheoremProving (ITP), number 8558 in LNCS, pages 258–274,Wien, Austria, 2014.Springer. doi:10.1007/978-3-319-08970-6_17.

[12] L. Gesbert, Z. Hu, F. Loulergue, K.Matsuzaki, and J. Tesson. SystematicDevelopment of Correct Bulk Synchronous Parallel Programs. In Paralleland Distributed Computing, Applications and Technologies (PDCAT), pages334–340. IEEE, 2010. doi:10.1109/PDCAT.2010.86.


http://homepages.inf.ed.ac.uk/mic/Pubs

http://dx.doi.org/10.1007/978-3-319-08970-6_17

http://dx.doi.org/10.1109/PDCAT.2010.86

Bibliographie III

[13] J. Gibbons. The third homomorphism theorem. Journal of FunctionalProgramming, 6(4):657–665, 1996. doi:10.1017/S0956796800001908.

[14] L. Granvilliers, G. Hains, Q.Miller, and N. Romero. A system for thehigh-level parallelization and cooperation of constraint solvers. In Y. Pan,S. G. Akl, and K. Li, editors, Proceedings of International Conference onParallel and Distributed Computing and Systems (PDCS), pages 596–601, LasVegas, USA, 1998. IASTED/ACTA Press.

[15] Z. Hu, H. Iwasaki, andM. Takechi. Formal derivation of efficient parallelprograms by construction of list homomorphisms. ACMTrans Program LangSyst, 19(3):444–461, 1997. ISSN 0164-0925.doi:10.1145/256167.256201.

[16] F. Loulergue, S. Robillard, J. Tesson, J. Légaux, and Z. Hu. Formal Derivationand Extraction of a Parallel Program for the All Nearest Smaller ValuesProblem. In ACM Symposium on Applied Computing (SAC), pages1577–1584, Gyeongju, Korea, 2014. ACM.doi:10.1145/2554850.2554912.


http://dx.doi.org/10.1017/S0956796800001908

http://dx.doi.org/10.1145/256167.256201

http://dx.doi.org/10.1145/2554850.2554912

Bibliographie IV

[17] F. Loulergue,W. Bousdira, and J. Tesson. Calculating Parallel Programs inCoq using List Homomorphisms. Int J Parallel Prog, 2016.doi:10.1007/s10766-016-0415-8.

[18] L. Meertens. Algorithmics – towards programming as amathematicalactivity. In Proceedings of CWI Symposium onMathematics and ComputerScience, pages 289 – 334. North-Holland, 1986.

[19] A.Morihata, K. Matsuzaki, Z. Hu, andM. Takeichi. The thirdhomomorphism theorem on trees: downward & upward lead todivide-and-conquer. In Z. Shao and B. C. Pierce, editors, POPL’09, pages177–185. ACM, 2009. doi:10.1145/1480881.1480905.

[20] K.Morita, A. Morihata, K. Matsuzaki, Z. Hu, andM. Takeichi. AutomaticInversion Generates Divide-and-Conquer Parallel Programs. In Conferenceon Programming Language Design and Implementation (PLDI), pages146–155. ACM, 2007. doi:10.1145/1250734.1250752.

[21] R. O. Rogers andD. B. Skillicorn. Using the BSP cost model to optimiseparallel neural network training. Future Generation Computer Systems, 14(5-6):409–424, 1998.


http://dx.doi.org/10.1007/s10766-016-0415-8

http://dx.doi.org/10.1145/1480881.1480905

http://dx.doi.org/10.1145/1250734.1250752

Bibliographie V

[22] The CoqDevelopment Team. The Coq Proof Assistant.http://coq.inria.fr.

[23] L. G. Valiant. A bridgingmodel for parallel computation. Commun. ACM, 33(8):103, 1990. doi:10.1145/79173.79181.


http://coq.inria.fr

http://dx.doi.org/10.1145/79173.79181

development of correct-by-construction functional parallel ... ·...

Documents