CPSC 668 Set 10: Consensus with Byzantine Failures 1
CPSC 668Distributed Algorithms and Systems
Fall 2009
Prof. Jennifer Welch
CPSC 668 Set 10: Consensus with Byzantine Failures 2
Consensus with Byzantine Failures• How many processors total are needed to
solve consensus when f = 1 ?
• Suppose n = 2. If p0 has input 0 and p1 has 1, someone has to change, but not both. What if one processor is faulty? How can the other one know?
• Suppose n = 3. If p0 has input 0, p1 has input 1, and p2 is faulty, then a tie-breaker is needed, but p2 can act maliciously.
CPSC 668 Set 10: Consensus with Byzantine Failures 3
Processor Lower Bound for f = 1
Theorem (5.7): Any consensus algorithm for 1 Byzantine failure must have at least 4 processors.
Proof: Suppose in contradiction there is a consensus algorithm A = (A,B,C) for 3 processors and 1 Byzantine failure.
p0 p2
p1
A C
B
CPSC 668 Set 10: Consensus with Byzantine Failures 4
Specifying Faulty Behavior• Consider a ring of 6 nonfaulty processors running components of
A like this:
• This execution probably doesn't solve consensus (it doesn't have to).
• But the processors do something -- this behavior is used to specify the behavior of faulty processors in executions of A in the triangle.
p0A
p2
Cp1
B
p3 A
p4
Bp5
C
1 1
0
00
1
CPSC 668 Set 10: Consensus with Byzantine Failures 5
Getting a Contradiction
• Let 0 be this execution:
p0 p2
p1
C
B
00
0act like p3to p4 in
act like p0to p5 in
p1 and p2must decide 0
CPSC 668 Set 10: Consensus with Byzantine Failures 6
Getting a Contradiction
• Let 1 be this execution:
p0 p2
p1B
11
1act like p2to p1 in
act like p5to p0 in
p0 and p1must decide 1
A
CPSC 668 Set 10: Consensus with Byzantine Failures 7
The Contradiction
• Let be this execution:
p0 p2
p1
C01
?
act like p4to p5 in
act like p1to p0 in
What do p0 andp2 decide?
A
view of p0 in = view of p0 in = view of p0 in 1 p0 decides 1
view of p2 in = view of p2 in = view of p2 in 0 p2 decides 0
CPSC 668 Set 10: Consensus with Byzantine Failures 8
Views
p0A
p2
Cp1
B
p3 A
p4
B
p5
C
p0 p2
p1
C01
?
act like p4to p5 in
act like p1to p0 in
Ap0 p2
p1B
11
1act like p2to p1 in
act like p5to p0 in
A
0 0
011 1
:1:
CPSC 668 Set 10: Consensus with Byzantine Failures 9
Processor Lower Bound for Any f
Theorem: Any consensus algorithm for f Byzantine failures must have at least 3f+1 processors.
Proof: Use a reduction to the 3:1 case.• Suppose in contradiction there is an algorithm A for f > 1 failures and n = 3f total processors.
• Use A to construct an algorithm for 1 failure and 3 processors, a contradiction.
CPSC 668 Set 10: Consensus with Byzantine Failures 10
The Reduction• Partition the n ≤ 3f processors into three sets, P0, P1,
and P2, each of size at most f.
• In the n = 3 case, let– p0 simulate P0
– p1 simulate P1
– p2 simulate P2
• If one processor is faulty in the n = 3 system, then at most f processors are faulty in the simulated system.
• Thus the simulated system is correct.
• Let the processors in the n = 3 system decide the same as the simulated processors, and their decisions will also be correct.
CPSC 668 Set 10: Consensus with Byzantine Failures 11
Exponential Tree Algorithm
• This algorithm uses– f + 1 rounds (optimal)– n = 3f + 1 processors (optimal)– exponential size messages (sub-optimal)
• Each processor keeps a tree data structure in its local state
• Values are filled in the tree during the f + 1 rounds
• At the end, the values in the tree are used to calculate the decision.
CPSC 668 Set 10: Consensus with Byzantine Failures 12
Local Tree Data Structure• Each tree node is labeled with a sequence of
unique processor indices.• Root's label is empty sequence ; root has
level 0• root has n children, labeled 0 through n - 1• Child node labeled i has n - 1 children, labeled
i : 0 through i : n-1 (skipping i : i)• Node at level d labeled v has n - d children,
labeled v : 0 through v : n-1 (skipping any index appearing in v)
• Nodes at level f + 1 are leaves.
CPSC 668 Set 10: Consensus with Byzantine Failures 13
Example of Local Tree
The tree when n = 4 and f = 1 :
CPSC 668 Set 10: Consensus with Byzantine Failures 14
Filling in the Tree Nodes• Initially store your input in the root (level 0)• Round 1:
– send level 0 of your tree to all– store value x received from each pj in tree node labeled
j (level 1); use a default if necessary– "pj told me that pj 's input was x"
• Round 2:– send level 1 of your tree to all– store value x received from each pj for each tree node k
in tree node labeled k : j (level 2); use a default if necessary
– "pj told me that pk told pj that pk's input was x"
• Continue for f + 1 rounds
CPSC 668 Set 10: Consensus with Byzantine Failures 15
Calculating the Decision
• In round f + 1, each processor uses the values in its tree to compute its decision.
• Recursively compute the "resolved" value for the root of the tree, resolve(), based on the "resolved" values for the other tree nodes:
resolve() =
value in tree node labeled if it is a leaf
majority{resolve( ') : ' is a child of } otherwise (use a default if tied)
CPSC 668 Set 10: Consensus with Byzantine Failures 16
Example of Resolving Values
The tree when n = 4 and f = 1 :
0 0 1 0 0 0 1 1 1 1 1 0
0 0 1 1
0(assuming 0 is the default)
CPSC 668 Set 10: Consensus with Byzantine Failures 17
Resolved Values are ConsistentLemma (5.9): If pi and pj are nonfaulty, then pi 's
resolved value for tree node labeled ' j (what pj tells pi for node ') equals what pj stores in its node '.
'
'j
'
part of pi's tree part of pj's tree
resolvedvalue = v
originalvalue = v
CPSC 668 Set 10: Consensus with Byzantine Failures 18
Resolved Values are Consistent
Proof Ideas:• By induction on the height of the tree node.• Uses inductive hypothesis to know that resolved
values for children of the tree node corresponding to nonfaulty procs are consistent.
• Uses fact that n > 3f and fact that each tree node has at least n - f children to know that majority of children are nonfaulty.
CPSC 668 Set 10: Consensus with Byzantine Failures 19
Validity
• Suppose all inputs are v.• Nonfaulty proc. pi decides resolve(), which
is the majority among resolve(j), 0 ≤ j ≤ n-1, based on pi 's tree.
• Since resolved values are consistent, resolve(j) (at pi) is value stored at the root of pj 's tree, which is pj 's input value if pj is nonfaulty.
• Since there are a majority of nonfaulty processors, pi decides v.
CPSC 668 Set 10: Consensus with Byzantine Failures 20
Common Nodes• A tree node is common if all nonfaulty
procs. compute the same value of resolve().
' '
part of pi's tree part of pj's tree
same resolved value
CPSC 668 Set 10: Consensus with Byzantine Failures 21
Common Frontiers• A tree node has a common frontier if every path
from to a leaf contains a common node.
pink meanscommon
CPSC 668 Set 10: Consensus with Byzantine Failures 22
Common Nodes and Frontiers
Lemma (5.10): If has a common frontier, then is common.
Proof Ideas: By induction on height of . Uses fact that resolve is defined using majority.
CPSC 668 Set 10: Consensus with Byzantine Failures 23
Agreement
• The nodes on each path from a child of the root to a leaf correspond to f + 1 different processors.
• Since there are at most f faulty processors, at least one such node corresponds to a nonfaulty processor.
• This node is common (by the lemma about the consistency of resolved values).
• Thus the root has a common frontier.• Thus the root is common (by preceding
lemma).
CPSC 668 Set 10: Consensus with Byzantine Failures 24
Complexity
Exponential tree algorithm uses
• n > 3f processors
• f + 1 rounds
• exponential size messages:– each msg in round r contains
n(n-1)(n-2)…(n-(r-2)) values– When r = f + 1, this is exponential if f is
more than constant relative to n
CPSC 668 Set 10: Consensus with Byzantine Failures 25
A Polynomial Algorithm
• We can reduce the message size to polynomial with a simple algorithm
• The number of processors increases to n > 4f
• The number of rounds increases to 2(f + 1)
• Uses f + 1 phases, each taking two rounds
CPSC 668 Set 10: Consensus with Byzantine Failures 26
Phase King AlgorithmCode for each processor pi:pref := my inputfirst round of phase k, 1 ≤ k ≤ f+1:
send pref to allreceive prefs of otherslet maj be value that occurs > n/2 times (0 if none)let mult be number of times maj occurs
second round of phase k:if i = k then send maj to all // I am the phase king
receive tie-breaker from pk (0 if none)if mult > n/2 + f
then pref := majelse pref := tie-breaker
if k = f + 1 then decide pref
CPSC 668 Set 10: Consensus with Byzantine Failures 27
Unanimous Phase Lemma
Lemma (5.12): If all nonfaulty processors prefer v at start of phase k, then all do at end of phase k.
Proof:• Each nonfaulty proc. receives at least n - f
preferences for v in first round of phase k• Since n > 4f, it follows that n - f > n/2 + f• So each nonfaulty proc. still prefers v.
CPSC 668 Set 10: Consensus with Byzantine Failures 28
Phase King Validity
Unanimous phase lemma implies validity:• Suppose all procs have input v.• Then at start of phase 1, all nf procs prefer v.• So at end of phase 1, all nf procs prefer v.• So at start of phase 2, all nf procs prefer v.• So at end of phase 2, all nf procs prefer v.• …• At end of phase f + 1, all nf procs prefer v and
decide v.
CPSC 668 Set 10: Consensus with Byzantine Failures 29
Nonfaulty King Lemma
Lemma (5.13): If king of phase k is nonfaulty, then all nonfaulty procs have same preference at end of phase k.
Proof: Let pi and pj be nonfaulty.
Case 1: pi and pj both use pk 's tie-breaker. Since pk is nonfaulty, they both have same preference.
CPSC 668 Set 10: Consensus with Byzantine Failures 30
Nonfaulty King Lemma
Case 2: pi uses its majority value v and pj uses king's tie-breaker.
• Then pi receives more than n/2 + f preferences for v
• So pk receives more than n/2 preferences for v
• So pk 's tie-breaker is v
CPSC 668 Set 10: Consensus with Byzantine Failures 31
Nonfaulty King Lemma
Case 3: pi and pj both use their own majority values.
• Suppose pi 's majority value is v
• Then pi receives more than n/2 + f preferences for v
• So pj receives more than n/2 preferences for v
• So pj 's majority value is also v
CPSC 668 Set 10: Consensus with Byzantine Failures 32
Phase King Agreement
Use previous two lemmas to prove agreement:• Since there are f + 1 phases, at least one has
a nonfaulty king.• Nonfaulty King Lemma implies at the end of
that phase, all nonfaulty processors have same preference
• Unanimous Phase Lemma implies that from that phase onward, all nonfaulty processors have same preference
• Thus all nonfaulty decisions are same.