aran bergman & eddie bortnikov & alex shraer, principles of reliable distributed systems,...
Post on 21-Dec-2015
217 views
TRANSCRIPT
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 20081
Principles of Reliable Distributed Systems
Recitation 2: Broadcast Services
Spring 2009
Alex Shraer
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 20082
Broadcast Service for Replication
• Primitives: broadcast(m), deliver(m).– For simplicity, assume m is unique.
Network
BroadcastAlgorithm
Application
deliverbroadcast
receivesend
BroadcastAlgorithm
Application
deliverbroadcast
receivesend
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 20083
Reliable Broadcast Specifications
• Validity: if a correct process broadcasts m then all correct processes eventually deliver m
• Agreement: if a correct process delivers m then all correct processes eventually deliver m– Uniform Agreement: if any process delivers m then all
correct processes eventually deliver m
• Integrity: m is delivered by a correct process at most once, and only if it was previously broadcast
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 20084
Reliable Broadcast - Quiz
• What happens if a process fails during the broadcast of a message?
• Does a message delivery by a faulty process require the delivery of this message by correct processes?
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 20085
FIFO Broadcast• Why is FIFO important?• FIFO Order: If a process broadcasts a message m before it
broadcasts a message m’, then no correct process delivers m’ unless it has previously delivered m.
• FIFO Broadcast: Reliable broadcast + FIFO Order
• Alternative definition of FIFO Order?– “all messages broadcast by the same process are delivered to all
processes in the order they are sent”
• Quiz: Are these definitions equivalent?
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 20086
Example
m1
m2
m3
p (fau lty)
q (correct)
• Also, this alternative definition forces faulty processes to deliver messages. (impossible)
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 20087
Causal Broadcast• Why is causality important?• Event e causally precedes event f (e→f) iff:
– a process executes both e and f, in that order, or– e is the broadcast of some message m and f is the delivery of m,
or– There is an event h, such that e→h and h→f.
• Causal Order: If the broadcast of a message m causally precedes the broadcast of a message m’, then no correct process delivers m’ unless it has previously delivered m.
• Causal Broadcast: Reliable broadcast + Causal order
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 20088
Atomic Broadcast and Uniformity
• Why would we want more than Causal Broadcast? • Atomic Broadcast: Reliable Broadcast + Total Order• Total Order: if correct processes p and q both deliver
messages m and m’, then p delivers m before m’ if and only if q delivers m before m’.
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 20089
Broadcast Primitives
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 200810
Uniformity• Agreement, Integrity and Order place no restrictions on the behavior of faulty processes.• Uniform – limit the behavior of faulty processes
• Example 1: Agreement allows a faulty process to deliver a message that is never delivered by correct processes
• Uniform Agreement: If a process (whether correct or faulty) delivers a message m, then all correct processes eventually deliver m.
• Example 2: Integrity allows a faulty process to deliver a message more than once, and to deliver messages ‘out of thin air’
• Uniform Integrity: For any message m, every process (whether correct or faulty) delivers m at most once, and only if some process broadcast m.
• Likewise, we can strengthen the Order properties:• Uniform FIFO Order: If a process broadcasts a message m before it broadcasts a
message m’, then no process (whether correct of faulty) delivers m’ unless it has previously delivered m.
• Uniform Causal Order: If the broadcast of a message m causally precedes the broadcast of a message m’, then no process (whether correct or faulty) delivers m’ unless it has previously delivered m.
• Uniform Total Order: if any processes p and q (whether correct or faulty) both deliver messages m and m’, then p delivers m before m’ iff q delivers m before m’.
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 200811
Crash Failures
• Suppose processes are only subject to crash failures– They operate correctly up to the time they crash (by
definition).
• Can we assume that the message deliveries that a process makes before crashing are always ‘correct’ (consistent with those of correct processes)?– No
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 200812
Crash Failures (cont’d)
• Coordinator-based algorithm:– When a process intends to broadcast a message m, it
first sends m to a coordinator.
– The coordinator delivers messages in the order in which it receives them, and periodically informs the other processes of this message delivery order.
– Other processes deliver messages according to this order.
– If the coordinator crashes, another process takes over as coordinator.
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 200813
Crash Failures (cont’d)• The algorithm satisfies the specification Atomic
Broadcast• Suppose a coordinator delivers m before m’ and
crashes.• A new coordinator could think m’ is before m. • All correct processes follow the new coordinator• Thus, the old coordinator delivered messages out
of order before it crashed.– Inconsistency can occur even when there are only crash
failures. – Protocols should explicitly prevent inconsistency even
when there are only crash failures
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 200814
LTS Broadcast Algorithm - code for process pi
Logical Clock Assignment: TS[j] ← 0, j=0,…,n pending ← empty
broadcast (m) TS[i] ← TS[i] + 1 send (m, TS[i], i) to all
upon receive (m, t, j) TS[j] ← t add (m, t, j) to pending TS[i] ← max (TS[i], t) + 1
Delivery Rule let (m, t, j) be the entry in pending with the smallest t, j if t, j TS[k],k k=0,…n then
deliver (m) remove (m, t, j) from pending
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 200815
p1
p2
p3
m1
m1
m2
m2
m3
m3 m4
m4
t=0 t=5 t=10
0
0
1,2
2
3,1
3,3
4
5
6,2
7
14
3
6
האם הרשת מעבירה את כל ההודעות בסדר שמשמר את יחס •)? happens-before (כלומר causalityה-
בכל פעם שהוא משתנה. סמנו את ערך logical clockסמנו את ערך • המצורף לכל הודעה.LTSה-
. אם m3 ו m1 להודעות delivery) כל תהליך מבצע tציינו מתי (באיזה • ציינו זאת בטבלה.deliveryבריצה המתוארת לא מתבצע
m3 m1
p1
p2
p3
Example Exam Question
0 2 73 8
1212
77
710
Delivery according to LTS
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 200816
Vector Clocks
• At process pi, on broadcast(m)
– VC[i] := VC[i]+1– use reliable broadcast to send m with VC to all– deliver m locally
• Upon receive m– place in message buffer
• Deliver m from pj from buffer if
– VC[j] = m.VC[j] – 1 – forall k≠j : VC[k] ≥ m.VC[k]
• Upon deliver – VC[j] := VC[j] + 1
VC[j] is the number of messages of pj that causally precedepi’s subsequent
messages
FIFO
Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 200817
Example Exam Question – Cont.
שהיה מצורף לכל הודעה אם היינו משתמשים בהם.Vector Clockסמנו את ה-•. אם בריצה m3 ו m1 להודעות delivery) כל תהליך מבצע tציינו מתי (באיזה •
ציינו זאת בטבלה.deliveryהמתוארת לא מתבצע
p1
p2
p3
m1
m1
m2m
2
m3
m3 m4
m4
t=0 t=5 t=10
[0,0,0]
[0,0,0]
[0,0,0]
[0,1,0]
[0,1,0]
[0,1,0]
[1,1,0]
[0,1,1]
[1,1,1]
[1,1,1]
[1,1,0]
[1,2,1]
[1,2,1]
[0,1,0][0,1,1]
[1,1,0]
[0,1,1]
[1,2,1]
[1,2,1]
m3 m1
p1
p2
p3
34
17
27
Delivery according to VC