![Page 1: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/1.jpg)
Lab 2Group Communication
Andreas Larsson
2010-02-01
![Page 2: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/2.jpg)
DSII: Group Comm. 2
Overview
• Introduction to group communication
• Desired group communication
• Multicast communication
• Group membership service
![Page 3: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/3.jpg)
DSII: Group Comm. 3
Coordination in distributed systems
• Coordination is needed by distributed systems but hard to achieve:– Events happen concurrently– Communication links are not reliable– Computers can crash– New nodes can join the systems– Asynchronous environments
Need of an efficient way to coordinate a group of processes
![Page 4: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/4.jpg)
DSII: Group Comm. 4
A Distributed System in war:Synchronous Example
Friendlyfighter
Radar
Missile
Commander
Enemy missile
Friend
Readyto Fire
Cancel
Ask Prepare
Friendly fighter
![Page 5: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/5.jpg)
DSII: Group Comm. 5
A Distributed System in war:Reality
Radar
Missile
Commander
Enemy missile
Friend
Readyto Fire
Fire
Ask Prepare
Friendly fighter
Friendlyfighter
![Page 6: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/6.jpg)
DSII: Group Comm. 6
Group communication
• What is a group?– A number of processes which cooperate to
provide a service.– An abstract identity to name a collection of
processes.
• Group Communication– For coordination among processes of a group.
![Page 7: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/7.jpg)
DSII: Group Comm. 7
Who Needs Group Communication?
• Highly available servers (client-server)
• Database Replication
• Multimedia Conferencing
• Online Games
• Cluster management
• …
![Page 8: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/8.jpg)
DSII: Group Comm. 8
Distributed Web Server
• High availability
![Page 9: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/9.jpg)
DSII: Group Comm. 9
Online Game
• Fault-tolerance, Order
![Page 10: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/10.jpg)
DSII: Group Comm. 10
Different Comm. Methods
• Unicast– Point-to-Point Communication– Multiple copies are sent.
• Broadcast– One-to-All Communication– Abuse of Network Bandwidth
• Multicast– One-to-multiple Communication
![Page 11: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/11.jpg)
DSII: Group Comm. 11
Overview
• Introduction to group communication
• Desired group communication
• Multicast communication
• Group membership service
![Page 12: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/12.jpg)
DSII: Group Comm. 12
Desired Group Communication
• Name Abstraction
• Efficiency
• Delivery Guarantees
• Dynamic Membership
Multicast
Reliability, Ordering
Group membership service
![Page 13: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/13.jpg)
DSII: Group Comm. 13
Properties of Communication
• Ordering– Total ordering, causal ordering
• Failure behavior
• Reliability– Validity, integrity, agreement
![Page 14: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/14.jpg)
DSII: Group Comm. 14
Properties of Group
• Name of group
• Addresses of group members
• Dynamic group membership
• Options:– Peer group or client-server group– Closed or Open Group
![Page 15: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/15.jpg)
DSII: Group Comm. 15
Peer Group
• All the members are equal.
• All the members send messages to the group.
• All the members receive all the messages.
![Page 16: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/16.jpg)
DSII: Group Comm. 16
Client-Server Group
• Replicated servers.• Clients do not care
which server answers.
![Page 17: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/17.jpg)
DSII: Group Comm. 17
Overview
• Introduction to group communication
• Desired group communication
• Multicast communication
• Group membership service
![Page 18: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/18.jpg)
DSII: Group Comm. 18
Multicast communication
• Use network hardware support for broadcast or multicast when it is available.
• Send message over a distribution tree.
• Minimize the time and bandwidth utilization
![Page 19: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/19.jpg)
DSII: Group Comm. 19
Reliability
Correct processes: those that never fail.
• IntegrityA correct process delivers a message at most once.
• ValidityA message from a correct process will be delivered by
the process eventually.• Agreement
A message delivered by a correct process will be delivered by all other correct processes in the group.
Validity + Agreement = Liveness
![Page 20: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/20.jpg)
DSII: Group Comm. 20
OrderingAssumptions: a process belongs to
at most one group.
• FIFO– if mp m’p, all correct
processes that deliver m’p will deliver mp (that is from the same sender) before m’p.
• Causal– if m m’, all correct
processes that deliver m’ will deliver m before m’.
• Total– if a correct process delivers m
before m’, all other correct processes that deliver m’ will deliver m before m’.
p1
p2
p3
p1
p2
p3
p1
p2
p3
![Page 21: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/21.jpg)
DSII: Group Comm. 21
Examples
• Assumption:– Reliable one-to-one send operation (e.g. TCP)
• Basic multicast– Requirement:
• All correct processes will eventually deliver the message from a correct sender.
– Implementation: • B-multicast( g, m): p g: send( p, m);• On receive( m) at p: B-deliver( m) at p.
Properties: integrity, validity.
![Page 22: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/22.jpg)
DSII: Group Comm. 22
Basic multicast: Agreement?
4
crash1
2
3
AgreementX
![Page 23: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/23.jpg)
DSII: Group Comm. 23
Examples (cont.)
• Reliable multicast– Requirements: integrity, validity, agreement– Implementation:
• Received := {};• R-multicast( g, m) at process p: B-multicast( g, m);• On B-deliver( m) at process p from process q
if( m Received)Received := Received {m};if( q p) B-multicast( g, m);R-deliver( m);
end if
Inefficient: each message is sent O(|g|) times to each process
![Page 24: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/24.jpg)
DSII: Group Comm. 24
Examples (cont.)• FIFO-ordered multicast:
– Assumption: • a process belongs to at most one group.
– Implementation:• Local variables at p: Sp = 1, Rp[ |g| ]={0};• FO-multicast( g, m) at p:
B-multicast( g, <m, Sp>);
Sp++;• On B-deliver( <m, S>) at p from q:
if( S = Rp[q] + 1) FO-deliver( m);Rp[q] := S;
else if( S > Rp[q] + 1) place <m, S> in the queue until S = Rp[q] + 1;FO-deliver( m);Rp[q] := S;
end if
– Your task: totally ordered multicasts.
![Page 25: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/25.jpg)
DSII: Group Comm. 25
Overview
• Introduction to group communication
• Desired group communication
• Multicast communication
• Group membership service
![Page 26: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/26.jpg)
DSII: Group Comm. 26
Group membership service• Four tasks:
– Interface for group membership changes
– Failure detector– Membership change
notification– Group address
expansion
Group address
expansion
MulticastCommunication
Groupmembershipmanagement
• Group partition:– Primary-partition – one partition only– Partitionable – many partiations at once
![Page 27: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/27.jpg)
DSII: Group Comm. 27
Group views• Group views:
– Lists of the current ordered group members– A new one is generated when processes join or
leave/fail.• View delivery
– When a member is notified of a membership change– Requirements
• Order– if p delivers v(g) v’(g), no other process delivers v’(g) v(g).
• Integrity– if p delivers v(g), p v(g).
• Non-triviality– if q joins a group and becomes indefinitely reachable from p,
eventually q is always in the view p delivers.
![Page 28: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/28.jpg)
DSII: Group Comm. 28
View-synchronous group comm.
• Extend the reliable multicast semantics with group views.– Agreement
• Correct processes deliver the same set of messages in any given view
– Validity (closed group)• Correct processes always deliver
the messages they send.
• p v0(g) does not deliver m in v0(g) p v1(g) for processes that deliver m.
– Integrity
p
q
r
(p,q,r) (q,r)
Xcrash
![Page 29: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/29.jpg)
DSII: Group Comm. 29
Examples
• Ensemble: reliable group communication toolkit– Previous talk
![Page 30: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/30.jpg)
DSII: Group Comm. 30
IP-multicast
• Multicast:
– Yes:• efficiency
– No:• Reliability• Ordering
• Group membership service:– Yes:
• Interface for group membership change
• Group address expansion
– No:• Failure detector• Membership change
notification
• IP: 224.0.0.1 - 239.255.255.255
![Page 31: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/31.jpg)
DSII: Group Comm. 31
References
• Distributed Systems: Concepts and Design by G. Coulouris et al., ISBN 0-201-61918-0– Section 4.5 Group Communication– Section 11.4 Multicast Communication– Section 14.2.2 Group Communication
• …
![Page 32: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/32.jpg)
DSII: Group Comm. 32
Lab 2: Construct a reliable and ordered multicast
• Reliable– Integrity, Validity, Agreement
• Ordered– All machines should agree on the order of all
received messages.– Total and causal order.
• No membership service– Processes can still crash though!
![Page 33: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/33.jpg)
The GUI
DSII: Group Comm. 33
Message display
Debug message display
Time for stress test24-hour time format
# of messages for stress test
![Page 34: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/34.jpg)
The interface
• Provide– cast
– setCallback
• Use– deliver– debug
– enableSending
DSII: Group Comm. 34
![Page 35: Lab 2 Group Communication Andreas Larsson 2010-02-01](https://reader035.vdocuments.us/reader035/viewer/2022062423/56649f4c5503460f94c6cb4f/html5/thumbnails/35.jpg)
Tips & pointers
• Extend SkeletonCaster
• You can assume startup times:– All programs starts within 5 seconds– Connect– Call enableSending
• Stress the system and check diffs
• Don’t use the buttons for anything else than what they are intended for
DSII: Group Comm. 35