Slide 1

Distributed Data Flow Language for Multi-Party Protocols Krzysztof Ostrowski , Ken Birman , Danny Dolev Cornell University, Hebrew University {krzys|ken}@cs.cornell.edu, dolev@cs.huji.ac.il

Slide 2

Introduction

Slide 3

http://liveobjects.cs.cornell.edu

Slide 4

distributed peer-to-peer protocols

Slide 5

http://liveobjects.cs.cornell.edu distributed peer-to-peer protocols multicast locking replication commit

Slide 6

http://liveobjects.cs.cornell.edu distributed peer-to-peer protocols multicast locking replication commit shared content

Slide 7

http://liveobjects.cs.cornell.edu distributed peer-to-peer protocols multicast locking replication commit replicated content

Slide 8

http://liveobjects.cs.cornell.edu multicast locking replication commit replicated content distributed peer-to-peer protocols synchronization, coordination

Slide 9

http://liveobjects.cs.cornell.edu

Slide 10

Slide 11

Slide 12

Slide 13

basic building block

Slide 14

http://liveobjects.cs.cornell.edu basic building block

Slide 15

http://liveobjects.cs.cornell.edu basic building block

Slide 16

http://liveobjects.cs.cornell.edu need lots of different objects (protocols)

Slide 17

http://liveobjects.cs.cornell.edu need lots of different objects (protocols) user-defined objects

Slide 18

How to Implement New Objects? custom user-defined object

Slide 19

custom user-defined object Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. How to Implement New Objects?

Slide 20

custom user-defined object Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. How to Implement New Objects?

Slide 21

custom user-defined object Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. How to Implement New Objects?

Slide 22

higher-level logic (making decisions) Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) intermingled, tightly-coupled How to Implement New Objects?

Slide 23

higher-level logic (making decisions) lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) intermingled, tightly-coupled How to Implement New Objects?

Slide 24

higher-level logic (making decisions) lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) intermingled, tightly-coupled How to Implement New Objects?

Slide 25

less flexibility harder to write/debug Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. error- prone less sophisticated How to Implement New Objects?

Slide 26

less flexibility harder to write/debug Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. error- prone less sophisticated How to Implement New Objects?

Slide 27

less flexibility harder to write/debug Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. error- prone less sophisticated How to Implement New Objects?

Slide 28

less flexibility harder to write/debug Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. error- prone less sophisticated How to Implement New Objects?

Slide 29

lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) higher-level logic (making decisions) separation of concerns

Slide 30

How to Implement New Objects? separation of concerns programmer compiler and runtime

Slide 31

How to Implement New Objects? separation of concerns programmer compiler and runtime

Slide 32

Our Approach: Flows Objects Aggregation Batching Recursion

Slide 33

Slide 34

Slide 35

Slide 36

Slide 37

Slide 38

Slide 39

Slide 40

Slide 41

Slide 42

Slide 43

Slide 44

Slide 45

Slide 46

Slide 47

Slide 48

Slide 49

Slide 50

Slide 51

Slide 52

Slide 53

Slide 54

Slide 55

Slide 56

Slide 57

Slide 58

Slide 59

Slide 60

Slide 61

Slide 62

Slide 63

Slide 64

Slide 65

Slide 66

e1e1

Slide 67

e2e2

Slide 68

x 1 crashed

Slide 69

e3e3

Slide 70

x 4 joined

Slide 71

e4e4

Slide 72

e5e5

Slide 73

e6e6

Slide 74

example: distributed locking protocol

Slide 75

application layer

Slide 76

example: distributed locking protocol application layer protocol layer

Slide 77

example: distributed locking protocol application layer protocol layer

Slide 78

example: distributed locking protocol

Slide 79

Slide 80

Slide 81

Slide 82

Slide 83

Slide 84

Slide 85

Slide 86

Slide 87

Slide 88

Slide 89

Slide 90

Slide 91

Slide 92

Slide 93

Slide 94

Slide 95

Slide 96

Slide 97

Our Approach: Flows Objects Aggregation Batching Recursion

Slide 98

example: distributed locking protocol

Slide 99

Slide 100

Slide 101

Slide 102

Slide 103

Slide 104

Slide 105

Slide 106

Slide 107

Slide 108

Slide 109

Slide 110

Slide 111

Slide 112

Slide 113

Slide 114

Slide 115

Our Approach: Flows Objects Aggregation Batching Recursion

Slide 116

input flow output flow aggregation

Slide 117

input flow output flow aggregation output event

Slide 118

input flow output flow output event input events

Slide 119

Slide 120

Slide 121

example: leader election protocol 3 3 9 6 aggregation with min

Slide 122

example: leader election protocol aggregation with min 3 9 6 3

Slide 123

example: leader election protocol aggregation with min 3 9 6 3

Slide 124

example: leader election protocol aggregation with min 3 9 6 3

Slide 125

Different flavors of aggregation: 1.In-order, 2.Guarded, 3.Coordinated, 4.Etc. formal, abstract properties can reason about global behavior Separation of Concerns

Slide 126

We dont specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

Slide 127

Separation of Concerns We dont specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

Slide 128

Separation of Concerns We dont specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

Slide 129

Separation of Concerns We dont specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

Slide 130

Separation of Concerns We dont specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

Slide 131

Separation of Concerns We dont specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

Slide 132

Aggregation with a Token Ring

Slide 133

{1..3,5..6}{2..4, 6}{1..6}

Slide 134

Aggregation with a Token Ring {1..3,5..6}{2..4, 6}{1..6} token leader node

Slide 135

Aggregation with a Token Ring {1..3,5..6}{2..4, 6}{1..6} token

Slide 136

Aggregation with a Token Ring {1..3,5..6} {2..4, 6}{1..6}

Slide 137

Aggregation with a Token Ring {1..3,5..6} {2..4, 6}{1..6}

Slide 138

Aggregation with a Token Ring {1..3,5..6} {2..4, 6} {1..6} intersect

Slide 139

Aggregation with a Token Ring {2..3, 6} {1..6}

Slide 140

Aggregation with a Token Ring {2..3, 6} {1..6}

Slide 141

Aggregation with a Token Ring {2..3, 6} {1..6} intersect

Slide 142

Aggregation with a Token Ring {2..3, 6}

Slide 143

Our Approach: Flows Objects Aggregation Batching Recursion

Slide 144

Batched Processing with Sets node x node y node z

Slide 145

Batched Processing with Sets node x node y node z

Slide 146

Batched Processing with Sets node x node y node z

Slide 147

Batched Processing with Sets node x node y node z 123456 123456 123456 7 7 7

Slide 148

Batched Processing with Sets node x node y node z 12356 47 2364 157 123456 7 DROPPED:

Slide 149

Batched Processing with Sets node x node y node z 123562364123456

Slide 150

Batched Processing with Sets node x node y node z 123562364123456

Slide 151

Batched Processing with Sets node x node y node z 123562364123456 Rec({1..3,5..6})

Slide 152

Batched Processing with Sets node x node y node z 123562364123456 Rec({1..3,5..6}) Rec({2..4, 6})

Slide 153

Batched Processing with Sets node x node y node z 123562364123456 Rec({1..3,5..6})Rec({2..4, 6}) Rec({1.. 6})

Slide 154

Batched Processing with Sets node x node y node z 123562364123456 Rec({1..3,5..6})Rec({2..4, 6}) Rec({1.. 6})

Slide 155

example: which packets are stable? {2..3,6} {1..3, 5..6} {1..6} aggregation with {2..4,6}

Slide 156

example: which packets are stable? {2..3,6} {1..3, 5..6} {1..6} aggregation with {2..4,6}

Slide 157

example: which packets are stable? {2..3,6} {1..3, 5..6} {1..6} aggregation with {2..4,6}

Slide 158

example: which packets are stable? {2..3,6} {1..3, 5..6} {1..6} aggregation with {2..4,6}

Slide 159

Our Approach: Flows Objects Aggregation Batching Recursion

Slide 160

Examples

Slide 161

01: object lock ( bool wants ) : bool holds 02: { 03: same int owner; 04: where ( wants ) 05: owner := elect ( id ); 06: holds := wants ( owner = id ); 07: } embedding distributed locking protocol

Slide 162

01: object elect ( up int candidate ) 02: : s-up int leader 03: { 04: s-up int elected := 0; 05: where ( fresh elected 06: elected candidate ) 07: elected := min candidate; 08: leader := elected; 09: } leader election protocol the core part

Slide 163

Conclusions

Slide 164

1.Collaboration requires custom protocols a)Fine-tuned for a particular application semantics b)Fine-tuned for different networks and workloads 2.Existing protocol languages dont suffice a)Developers need a clean separation of concerns 3.Modeling protocols as distributed flows a)Captures high-level protocol semantics concisely b)Reduces a coding burden on protocol developers c)Supports reasoning about the protocol behavior d)Provides a high degree of architectural flexibility Conclusions

Slide 165

Thanks