dependent types for high- confidence distributed systems paul sivilotti - ohio state hongwei xi –...
Post on 21-Dec-2015
215 views
TRANSCRIPT
![Page 1: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/1.jpg)
Dependent Types for High-Confidence Distributed
Systems
Paul Sivilotti - Ohio StateHongwei Xi – Cincinnati
(Boston Univ.)
![Page 2: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/2.jpg)
Specifying and Testing Quantified Progress Properties 2
Gaining confidence that program text satisfies required behavior
Two Challenges Specifying com-
ponent behavior and reasoning about its com-position
SystemProperties
Component AProperties
Component BProperties
Component CProperties
Component AProgram Text
Component B Program Text
Component C Program Text
![Page 3: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/3.jpg)
Specifying and Testing Quantified Progress Properties 3
Dependent Types enriched types familiar, low cost
Two Synergistic Solutions
Synergy: locality Surprising Connection: termination
Certificates temporal logic local properties
Typically not used for reasoning about progress
Typically not tied to real programs
![Page 4: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/4.jpg)
Specifying and Testing Quantified Progress Properties 4
Talk Outline Background and Motivation Phase I: foundational
Dependent types for metrics local termination
Certificates for progress with metrics functional transient, functional next
Phase II: integration DXanadu: a distributed dependently
typed language
![Page 5: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/5.jpg)
Phase I
Dependent Types for Metrics
![Page 6: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/6.jpg)
Specifying and Testing Quantified Progress Properties 6
Dependent Types Dependent types are types that
can depend on the values of expressions
Examples int(i) is a singleton type that contains
the only integer equal to i int array(n) is the type for integer
arrays of size n
![Page 7: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/7.jpg)
Specifying and Testing Quantified Progress Properties 7
Metrics and Integer Constraints Index variables and expressions
i+j, i-j, i*j, i/j, … Let = <i1,...,in> be a tuple of
index expressions. We write : metric if we have ij:nat for 1 j n Lexicographic ordering Well-founded
![Page 8: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/8.jpg)
Specifying and Testing Quantified Progress Properties 8
McCarthy’s 91 Function
{i:nat} <max(0, 101-i)> =>[j:int | (i <= 100 j = 91) (i > 100 j = i-10)]
int(j)ninetyone (x:int(i)) { if (x <= 100) { return ninetyone (ninetyone (x+11)); } else { return (x-10); }}
![Page 9: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/9.jpg)
Specifying and Testing Quantified Progress Properties 9
Cost Effectiveness In general, termination of a
program is difficult to prove However, critical sections tend to
be small and manageable More importantly, we provide the
programmer with a range of choices higher effort lower effort higher benefit lower benefit
![Page 10: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/10.jpg)
Specifying and Testing Quantified Progress Properties 10
Spectrum of Choices Static Check
Programmer provides a metric Type-checker verifies monotonicity of metric
Dynamic Check Programmer provides a metric Type-checker inserts run-time tests to check
monotonicity of metric Checkpointing
Programmer does not provide a metric Checkpoint taken before “dangerous” action
![Page 11: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/11.jpg)
Specifying and Testing Quantified Progress Properties 11
Bit Reversing from FFT
{a:nat,b:pos} <max(0, a-b)> => intbitrev (j:int(a),k:int(b)) { if (k < j) { return bitrev (j-k,k/2); } else { return j+k; }}
![Page 12: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/12.jpg)
Specifying and Testing Quantified Progress Properties 12
Bit Reversing from FFT
{a:nat,b:pos} <max(0, a-b)> => intbitrev (j:int(a),k:int(b)) { if (k < j) { assert (k > 1); return bitrev (j-k,k/2); } else { return j+k; }}
![Page 13: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/13.jpg)
Phase I
Functional Certificates
![Page 14: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/14.jpg)
Specifying and Testing Quantified Progress Properties 14
Transient P for component C means: progress: if P ever becomes true, it
eventually becomes false locality: guaranteed by an action of C
alone More formally:
transient P a C : [ P wp.a.P ]
“Transient”: A Certificate for Progress
![Page 15: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/15.jpg)
Specifying and Testing Quantified Progress Properties 15
E.g.: Mutual Exclusion
System: every client request is eventually satisfied
Token-passingLayer
Client CClient D
Client E
Client B
Client A
Client: token is eventually returned transient holding
![Page 16: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/16.jpg)
Specifying and Testing Quantified Progress Properties 16
Client Program
To prove transient holding, show CS terminates (ie ninetyone terminates)
*[ non CS ! request ? token //holding is true CS: ninetyone(0); ! token //holding is false ]
![Page 17: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/17.jpg)
Specifying and Testing Quantified Progress Properties 17
Testing Transience Recall for transient.P:
If P ever becomes true, it is later false Note: P may never become true
Consequence of formal definition:transient.P infinitely often P
To test for transience, use:transient.P finitely often P Look for a finite trace after which only
P
![Page 18: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/18.jpg)
Specifying and Testing Quantified Progress Properties 18
Timestamped History
transient.P
true
false
P
timedanger dangerdanger
P
settimestamp
cleartimestamp
predicate evaluation…
![Page 19: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/19.jpg)
Specifying and Testing Quantified Progress Properties 19
Multiple Properties A component may have many
progress propertiestransient.(status = critical)transient.(status = idle ^ button_down)transient.( . . . )
![Page 20: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/20.jpg)
Specifying and Testing Quantified Progress Properties 20
Multiple Transient Prop’stransient.P ^ transient.Q ^ transient.R
Complexity: Space: n timestamps kept Time: n predicate evaluations with each step
P
Q
R
![Page 21: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/21.jpg)
Specifying and Testing Quantified Progress Properties 21
Quantification of Transient Transient properties often quantified
“state changes eventually”Ak :: transient.(status = k)
“value of metric changes eventually”Ak :: transient.(metric = k ^ status = critical)
Naïve expansion is costly to monitor If dummy ranges over a set D of values:
|D| timestamps to maintain |D| predicate evaluations to perform
![Page 22: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/22.jpg)
Specifying and Testing Quantified Progress Properties 22
Observation: Singularity Predicates are mutually exclusive Ak :: transient.(metric = k ^ status = critical) (P)
= transient.(metric = 0 ^ status = critical) (P0)
^ transient.(metric = 1 ^ status = critical) (P1) ^ transient.(metric = 2 ^ status = critical)… (P2)…
Truth of predicate functionally determines value of dummy variableFor P.(s,k) : predicate on state s, dummy k:
Ak :: transient.(P.(s,k)) is functional iffEf :: (P.(s,k) k = f.s)
![Page 23: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/23.jpg)
Specifying and Testing Quantified Progress Properties 23
Functional TransienceAk :: transient.(metric = k ^ status =
critical)
When is there “danger” of a possible violation?
P0
P1
P2
P3
![Page 24: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/24.jpg)
Specifying and Testing Quantified Progress Properties 24
Satisfying Functional Transience A functional transient property is
“satisfied” when either: The predicate that is true changes
Value(s) of dummy variable(s) that makes predicate true changes
All predicates become false Provide f: states dummy values
Evaluate k using f Evaluate P using k
![Page 25: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/25.jpg)
Specifying and Testing Quantified Progress Properties 25
Functional TransienceAk :: transient.(metric = k ^ status =
critical)
Complexity: Space: 1 timestamp & value(s) of dummy(s) Time: 1 function & 1 predicate evaluation
P0
P1
P2
P3
TS S R R R CR S
![Page 26: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/26.jpg)
Specifying and Testing Quantified Progress Properties 26
Generalization: Relational Transience Number of predicates that can be
simultaneously true is bounded (B)
Ak :: transient.(k <= metric <= k+1 ^ critical)
P0
P1
P2
P3
012
![Page 27: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/27.jpg)
Specifying and Testing Quantified Progress Properties 27
Monitoring Relational Transience
Complexity Space: B timestamps & dummy values Time: 1 relation eval’n & 2B timestamp
updates
P0
P1
P2
P3
TS1
TS0
S
S
R R C
R R
S C
C S
![Page 28: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/28.jpg)
Specifying and Testing Quantified Progress Properties 28
Ubiquity of Functional Transience Observation: Many quantifications
of transient appear to be functional E.g., timeouts and metrics
Method-response semantics “method M returns a value eventually”
Ak :: transient.(rcv_M = k+1 ^ snd_M = k)Ak :: transient.(rcv_M > k ^ snd_M = k)Ajk : j > k : transient.(rcv_M = j ^ snd_M = k)
![Page 29: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/29.jpg)
Specifying and Testing Quantified Progress Properties 29
Other Progress Operators Transient is a very basic operator
Nice compositional properties Higher-order operator: leads-to (+-
>) Testing leads-to does not always
benefit from notion of functionality E.g., (Ak :: x = k +-> y = k)
Other simplifications can be made (Ak :: x = k +-> x < k)
![Page 30: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/30.jpg)
Specifying and Testing Quantified Progress Properties 30
Quantification of Safety Properties Safety operator: P next Q
“if P holds, Q holds in the next state” Similar quantifications arise
Ak :: x = k next x <= k Also commonly functional
Truth of pre-predicate determines value(s) of dummy(s)
Similar performance benefit 1 function & 1 predicate evaluation
![Page 31: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/31.jpg)
Phase II
DXanadu: A distributed dependently-typed language
![Page 32: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/32.jpg)
Specifying and Testing Quantified Progress Properties 32
DXanadu Integration: dependent types and
distributed programming Metrics for termination and progress Certificates for specs and reasoning
Implementation test bed
![Page 33: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/33.jpg)
Specifying and Testing Quantified Progress Properties 33
Communication Primitives Sequential Xanadu augmented with
send() and receive() keywords Point-to-point communication, with
fair merge at destination unit send(m,d)
m : basic type, arrays, non-nested records d : integer id of destination node
m = receive() Asynchronous send, blocking receive
![Page 34: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/34.jpg)
Specifying and Testing Quantified Progress Properties 34
Architecture Java RMI for communication backbone
Serialization of message types Static bootstrapping and component binding
DXanadu #0
DXanadu #1
DXanadu #2
POBox A
DXanadu #3
POBox B
DXanadu #4
POBox C
PostCentral
java
ocaml
![Page 35: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/35.jpg)
Specifying and Testing Quantified Progress Properties 35
Challenges to Be Faced Message typing
Dynamic structures (nesting, lists) Preserving (dependent) type information
Fault tolerance Conservative checkpointing for rollback
Linking code with certificate spec Run-time tests for validation
Dynamic computations Discovery, binding, departure
Experimentation with DXanadu
![Page 36: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/36.jpg)
Specifying and Testing Quantified Progress Properties 36
Papers Acknowledging OBR LICS 01: “Dependent Types for Program
Termination Verification” FSE 01: “Increasing Client-Side Confidence in
Remote Component Implementations” ICFP 01: “A Dependently-Typed Assembly
Language” ICSE 01: “The Specification and Testing of
Quantified Progress Properties in Distributed Systems”
J. of Applied Systems Studies: “Testing the Protocol Conformance of Distributed components” (submitted)
![Page 37: Dependent Types for High- Confidence Distributed Systems Paul Sivilotti - Ohio State Hongwei Xi – Cincinnati (Boston Univ.)](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d5e5503460f94a3d02d/html5/thumbnails/37.jpg)
Dependent Types for High-Confidence Distributed
Systems
Paul Sivilotti - Ohio StateHongwei Xi – Cincinnati
(Boston Univ.)