xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7x24 –...

84
Service Computing http://xuwang.tech

Upload: others

Post on 01-May-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

����Service Computing

��http://xuwang.tech

Page 2: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

� ���������

Page 3: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�$�• �+��)'.��– Facebook� �'&�– Google�'0�#�– ��'�����

• �,": XaaS, Internet of Services– Amazon�,"���

• !�,"��(�LBS• �$��%���.��/*�1���� �-�

Page 4: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��“��”�����

���������� ��������������������

Page 5: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

����• ���������(scalability)– ����

• Facebook ��#"���– � ��

• 2013���Google����!���'(100PB– �����

• 2012�11�11���%�+�$0�*'(1000� �&-�/���)19.2�/�*

• 2014���. ,3�

• ����������������

Page 6: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��� �������

Page 7: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

Scalability�����

���� ����

Page 8: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��• �������(availability): 7X24��

– ������(SLA):��• Amazon EC2 99.95%, Google Cloud Storage 99.9%,

Microsoft Windows Azure 99.9%– ���������������

� �� ��

1 Amazon 2011"4421b221�_VEC25��/)I��,#�=(-�2012"64�104�7�=�a6��

2 %S 2012"74261�Azure,^��NO9@'�$�c:8�/)I2.5 2c

3 Google GAE.*�&AgoogleE?��/2\32012"104261�)I4 2

4 %S %SHotmailbOutlook.com��SkyDriveD5�MK�2013"34121��R�G��`�ZT17� 2Ba6��c

5 Twitter 2013"6431�@'0;�!�Q�tweets�)I�F45�Y

6 Google 2013"84171����W�BGoogle5��=a6��N >LJ<X�]+U40%�)I��Y

7 Adobe 2014"54251�Adobe Creative Cloud0;P[+U24 2

8 %S 2014"6423b241�Lync instant messaging servicebOutlook Exchange onlineCH�=,^

Page 9: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

������(�/�)���� Scalability

� ��Load Balance

������Failover&Recovery

�Replication

�� Availability

R1 R2 RnRk… … ����

Load Balancer

RWrite(A,2)Read(A)=?

A=1 A=1 A=1A=1A=2

Read(A)���2����� �������

Page 10: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�)"*(1)• CAP(2000�SIGMOD�%keynote)– )"*(Consistency),��*(Availability)� ����*(Partition)�(���!�� ��&

• PACELC(2012 IEEE Computer)– ���#+(���(Partition),��*

(Availability)�)"*(Consistency) ���$���(Else)��'(Latency)�)"*(Consistency)�(���$

Page 11: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

Scalability: CAP��

PODC 2000: C�A�P�� ��������

2012�IEEE Computer ���The Growing Impact of the CAP Theorem�����ACID, BASE, PACELC

Page 12: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

���CAP����

Data Center A Data Center B

S1 S2

Load Balance

???

P: Partition Tolerance

C: Consistency

A: Availability

A: Availability

• Consistency: all nodes have same data at any time• Availability: the system allows operations all the time• Partition-tolerance: the system continues to work in spite of network

partitions

Page 13: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�)C9D(2)• �F�; 8Jim Gray– �C9D*��):G�"!�3�?��9�)@34���1�+# 53�?

• Amazon3�6<0>(�– E&��– '7C9D

• Yahoo3�2�/��(�– �2����3�/"!�C9D�,��B�%=

– .AC-$@��,

Page 14: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

#�*�(!+• ��(!+���+��– �+(!+(Linearizability), ACID

• �(!+à�(!+– ��(!+(eventual consistency)– ACIDàBASE(Basic Availability, Soft state,

Eventual consistency)• �(!+ ��$'– ���(Write conflicts)��!�%����)�(!+

– "&�(Read Staleness), � "&����

Page 15: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�!$�%��• �' (– ������(�����(�

• ������������������������– #�&��(

• ������� �������� �����������

• �' (– � "������' (������(��

Page 16: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��)!• ����– 3&���� �'5��'5.(��

• �1����)– 2PC�3PC

• #/– �,�����. 0��4�

• Paxos�)��� ���4�-�3%• Quorum"$– �2N����2W(�*�)�R(+*�)

Page 17: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�����• ����– �����SMATER�POSTGRES-R�HBase

• ����– ������Eventual Consistency�– �����Timeline consistency�– ������Session consistency�– � ����Causal consistency�– �������Snapshot Isolation consistency�

Page 18: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������• ����– !�""����– !�"�"���– �!�"�����"���– ���� !�"���!�"

18

Page 19: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������• ����– !�""����– !�"�"���– �!�"�����"���– ���� !�"���!�"

19

Page 20: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

#�$�$����• ������#�$� ���– K���� ����K-Versions�

• ������� ��– �� �������

• ������ ����� ��– ������TACT�

• ������������ ��– "���!������PBS�

• ����� ����������������� ��

Page 21: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

=/> �%>&�0• 8%6�Quorum),�31��2�5�<�%>�� ��%>&9�– Amazon&Dynamo),– �!������Cassandra– ��*;7.W,R�

• W,R ������������

• $����Quorum),�%>&9��:�(-+– �4�-+?#(Ring)"-+'

Page 22: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�& '���!�'���• �#��(Fast Paxos)• ��"�$(Generalized Paxos)• IP����• ������• �%������(MenciusMulti-Ring

Paxos)• � SSD��(SpinnakerCORFU)

Page 23: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

0!1�1 ��"• �0!1� !�*.– &�+���– �%����,#�� -

• �': (#���1 �1– : +2s à�$/�� -1.8% and ��/�� -

4.3%– : +500msà -25% ����– : +100msà -1%)/

23

Page 24: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

����������

Client

Replica1

Replica2Replican

(Coordinator)…

([n/2]+1) ACKs

ACK

ACK

Client

Replica1

Replica2Replican

(Coordinator)…

n ACKs

ACK

ACK

Client

Replica1

Replica2Replican

(Coordinator)…

� �����Non-Uniform

Total Order

Broadcast

������Distributed

Consensus

(Paxos)

�����Uniform

Total Order

Broadcast

COMMITC

OM

MIT

COMMITC

OM

MIT

COMMITC

OM

MIT

WRITE

WRITE

WRITE

log

log

commit

���� ����ACK �����

1, [n/2]+1,n

24

Page 25: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

����: RSM-dClient

Replica2Replica3

(Coordinator)

WRITE

Replican

d ACKs

ACK

ACK

REPL

Y

COMMIT

(1) f : �!��&+$�'�� f ≤ [n/2] �.23�-

Replica1

(2)� #��"�%�/*(coordinator)��, ������ �(log history)��� �(commit history)1��&�/*�0,()��.

log history commit history

PROPOSEFINISHED

Agreement Phase Commit Phase

(1 ≤ d ≤ n)

d=1 ������(Non-Uniform Total Order Broadcastd=[n/2]+1 Paxosd=n �����(Uniform Total Order Broadcast)Otherwise ����

25

Page 26: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

G5H�H4,D�

• �G5H� <(�� ( �.),')– <(,��:�B��( ��)– <(,�=6I7E;,9��!( C�)– 80�A�1���+# <(,�� ')

• �@– �/ <( �$2?�,"��*>-– 80"�>-,�3#F,&%�

Page 27: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������

RSM-d ��(!%Pwc=0

Paxos)��

d ≥[n/2]+1

�1�

���

(�.&��� �"���)3#���0��%�

�62)��!$���,/3#�'4)��/�)3#5'

1.�� "�100% �';

2. ”PROPOSE” #�� /*�����&d��-$(��.,);

3. �.,0��)+���������.

�*�)��

�����Write Log Lost

Pwc= Pwll

2+3�Non-uniform Write

Pwc= Pwll Pwnu

���1

��13

���Write LostPwc= Pwl

��12

�1�(Write Duplicaton)

Pwc= Pwl+ Pwd

��11

�+-�27

Page 28: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

���� �: �����Client

Replica2Replica3

(Coordinator)

WRITE

Replican

d ACKs

ACK

ACK

(1 ≤ d ≤ [n/2])

Replica1

log history

PROPOSE d(d ≤f) ������� ��

• �� 1,2,3• d(d ≤f)���� ����!��, �������������

• Pwll : �����������d

cwllwc PPP )(==

��� �����Pc

28

Page 29: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�!�(Non-uniform Write)Client

Replica2Replica3

WRITE

Replican

d ACKs

ACK

ACK

(1 ≤ d ≤ [n/2])

Replica1

log history

PROPOSEd(d ≤f) ���� ����� k���������

COMMIT

commit history

• �� 1,2, �����"�������(commit history)• Pwnu : ��� ������#�������

kdnc

dn

k

kcwnu PPP --

-

=

-= å )1()(]2/[

0

wnuwllwc PPP =

(Coordinator)

k��������

29

Page 30: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

%!7@2��• �: 1. 68�-�<2“PROPOSE”/?• >)�=D: �#�1&%!9 2�$. • Pelw(D) : D253�� $• �� $�

• �+ D=d (��:1,2��&), • 4�, �+D=m(d ≤ m ≤ [n/2]) , �(• ��"D2*�,0.�( ��'�;.��2,0Pwl):

kxnc

xn

k

kc

xc PPPxQ --

-

=

-= å )1()()()(]2/[

0

)(dQPwc =)()( mQmDPwc ==

å=

==]2/[

)()(n

dmelwwlwc mQmPPP

30

Page 31: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�* (Write Duplication)• ���#)��

– ������ +$�,�&�"-��– �(��&���* ������'�����!�%��*

Client WRITE

d ACKs

(1 ≤ d ≤ [n/2])

SuspectedCoordinator

NewCoordinator

WR

ITE

d ACKs

�����

WRITE d ACKs

(1 ≤ d ≤ [n/2])

WR

ITE

d ACKs

�����������

÷÷ø

öççè

æ--

÷÷ø

öççè

æ--

÷÷ø

öççè

æ---

÷÷ø

öççè

æ--

=

11

11

11

12

dn

dn

ddn

dn

PnoPno : ����������� ���������

31

Page 32: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�����• ��������

Coordinator

Replica

To

Te

To

Te Te

To ToTo

Te Te

Trust Suspicion

T�������� ��f(t): T�����Thb2� Thb1 : T�����.Pfs : ��������������

)Pr()Pr( 1212 eohbhbohbehbfs TTTTTTTTP ->-=+>+=

heartbeat

���heartbeat����32

Page 33: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

����� • n���������������� �

• ������� :

))1(1()1()(1 11

]2/[

0

fnfs

fnc

fc

n

fgfs PPP

fn

P ----

=

---÷÷ø

öççè

æ -= å

nogfscwd PPPP )1( -=

� �������� #���"���� �

���������������!�

33

Page 34: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�'& #• ���(Write lost)– ��*4���)�."&����,�0�� (��/()

• �1�(Write duplication)– ��/(!�����+2-3%� $

• �����1������, �:

wdwlwc PPP +=

34

Page 35: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������Client

Replica2Replica3

(Coordinator)

WRITE

Replican

d ACKsAC

K

ACK

REPLY

COMMIT

(1 ≤ d ≤ n)

Replica1 PROPOSE

LAi = tp(i)+tlog(i)+ta(i)� LA1, LA2,…, LAn����:

LA(1) ≤ LA(2) ≤… ≤ LA(n-1) (���)

Latency Lw (d)= LA(d-1)+min(LCi)

Lw (d)

tp ta

LAi LCi

FINISHED

tlog tcom

tc tf

35

Page 36: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�Lw (d)����• LAi=tp(i)+ta(i) ������ ������� G(t)• LA(k)���������������h(k)(t)H(k)(t)(1 ≤k ≤n-1)

• � , ����:

• ��:

dxxtfxftg )()()( -= ò+¥

¥-

knkn

iik tLAPtLAP

kn

ttLAPttLAP ----

=

+><÷÷ø

öççè

æ--

+Î=+Î å 111

1)( )()(

12

]),[(]),[( eee

ee

e

]),[(lim)( )(

0)(

+Î=

®

ttLAPth k

k

��LAi ∈[t,t+ε] (k-1) �LAi ��t

�� LAi ��t+ ε

kth �� LAi ∈[t,t+ε]�

)()( th k

dttthdttthLALAE dddd òò+¥

¥- -

¥-- -=- )()()( )1()()1()(

)1(0)(11

)())(())1(( )1()( ³>D÷÷ø

öççè

æ--

=-=-+ - ddLAdn

LALAEdLEdLE ddww

ΔLA(d) = 0))(1())((0

1 >-ò+¥ -- dttGtG dnd

��36

Page 37: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

������������• �����B(d)���� ����Bc���� ���� Bl :

• ��:

����

37

Page 38: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�"�#���• ����

– �� ����"����

– �����������"��!���

– ������� "�������"�#

j0=j

brc=j

C=j

w: ���&�$����b:�$����� ����(��� � ��")�rb: ���$� ���!��rl:���%��#�$����� ���

Page 39: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��� �% ��'#

• ��(���$��!���– "�$������Nwl &������ Nwd

– Pwc!���(Nwl + Nwd)/10,000,000• ���– ��n∈[2,9] d∈[1,[n/2]]– RMSE= 0.0009%, std.dev.=0.0052%

• �%– ��n∈[2,9] d∈[1,n]– RMSE=0.013ms, std.dev.=0.019ms

39

Page 40: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

d��������

-6

-5

-4

-3

-2

-1

0

d=1d=2d=3d=4

n=2 n=3 n=4 n=5 n=6 n=7 n=8 n=9

wcPlg

0

50

100

150

200

250

n=2 n=3 n=4 n=5 n=6 n=7 n=8 n=9

Lw(2)

Lw(3)Lw(4)Lw(5)

Latency-L w(1)(ms)

Impact of d on Consistency Impact of d on Latency

40

Page 41: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��� vs. ��

1-10-1

1-10-2

1-10-3

1-10-4

1-10-5

1-10-6

Consistency

Latency-Lw(1)(ms)050100150200

n=3n=5n=7

(3,2)(5,3)

(7,4)

(7,3)

(5,2)

(7,2)

(3,1)(5,1)

(7,1)

(n,d)

41

Page 42: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

1��?D6-�• 4� ,#�F�S���, #��0�V

– 4�3"(G@):D��9���0rb57:�H– 42&"(�G@):D��@)�!(��C� <:A��8!/:(.)

– -��%E:*';$• Microsoft Bing� rl=0.0009%• Amazon.com� rl=0.01%• Google Search�rl=0.05%

– ��B=��>�A��+�:v(d)�

42

Page 43: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

d��������

Page 44: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������• ����– !�""����– !�"�"���– �!�"�����"���– ���� !�"���!�"

44

Page 45: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

Quorum�������

÷÷ø

öççè

æ÷÷ø

öççè

æ -=

RN

RWN

prs

�� ����������

Page 46: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������

W/R �,�����

Client

Replica1

Replica2(Coordinator)

COMMIT

WRITE

ReplicaN

Send to N replicas

Wait forW responses

N=3, W=2,R=1

Replica2 Replica3

Partition 1

Partition 2Replica1

Replica2…

COMMIT

ReplicaN

Send to N replicas

Waiting, timeoutand then unavailable

Client

(Coordinator)

Replica1

Replica1

Replica2(Coordinator)

COMMIT

ReplicaN

Send to N replicas

Wait forW responses

Client

Page 47: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�)�(�• ������!���Quorum� �)

– &��'�����$*� �Quorum� ���– #%�/�����(2/3-tier basic tree), +�(fat

tree), folded clos!�����"

top of rack (ToR) switches

Aggregation switches (AS)

Core switches (CS)

Page 48: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

����• Quorum����QS(DCN,PM,W/R)

– DCN∈{2����(bt2)�3����(bt3)�K��(ft)�folded clos��(fc)}

– PM: ��������0/1����(��� )– W/R :�Quorum��Quorum� �

��(1) ����: Pc(2) �����: Pa(3) ToR���: Pt(4) ���: Ps�� ��������

PM tor

Page 49: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������(1)

��������

Page 50: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������(2)

���������

��� ��

�������������

Page 51: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�����

Server

ToRSEdge

CSCore

CoordinatorReplica Replica

ToRi

Ai : �$(&"� �!���ToRi��)+�&#���&� �'����:

�%����*�)+�&$(&� �'��W

å Õ=+++

-- ==xmmm i

iitorbtN

mAPxQ...

221

)()(

å=

---=N

Wxtorbts xQPWPMbtQSAvail )()1()),,2(( 2

��

��

WRITE

… … …

Page 52: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

Server

ToRS

CS

AS

CoordinatorReplica Replica

���� !� )(2 xQ torbt --

������

WRITE

� �

��

… … … …

Xbt3-as: ���������� " �������

å=

- =-=N

Wxasbts xXPWPMbtQSAvail

'3 )'Pr()1()),,3((

Page 53: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

AS

ToRS

CS

�k/2������(CS)������GCS��pod ��������(AS)��SAS

Pod1 Podk

Server

WRITE

… … … …

…… …

Coordinator ReplicaReplica

Group1 Groupk/2�

��

� �

K��

Page 54: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

AS

ToRS

CS

Pod1 Podk

Server

WRITE

… … … …

… …

Coordinator ReplicaReplica

GCS1 GCSx�

��

SAS1 SASk

����K��

�)���x GCS'"�!���*$3����%&(�����# �����

å=

- ==N

Wxsasft xXWPMftQSAvail

')'Pr()),,((

Page 55: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

Folded Clos��

� ���(CS)������GCS�������(AS)��SAS

Page 56: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

ToRS

CS

Server

AS

WRITE�

Coordinator ReplicaReplica

��

SAS1 SASDI/2

GCS

����Folded Clos

��3����������

å=

- =-=N

Wxsasfcgc xXPWPMfcQSAvail

')'Pr()'1()),,((

Page 57: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��• -��%�����– "�����%���#'��� !%�$0�,)

– ���$0(�� �+ !�*%�������$0%�&��

• !%���*./

Page 58: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

����• ����( ����8K)����N=3• =⟨1,1,1⟩� =⟨2,1,0⟩� =⟨3,0,0⟩torPM1

torPM 2torPM 2

���Hadoop����Cassandra��

Page 59: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��� vs. ���• ������α�������

• ���������������

• ����-���(W,R)��������– ����[!n]����– �� [!n]� ��������– �� [!n]���� ����αW+(1- α)R���

Page 60: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

���������� N∈[3,9] W∈[1,N], ���������

�����4�9��

Page 61: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

W����� �

• N=3, W �������9 ��(ft, fc)

Page 62: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

� ��N�����

• ��, N�����������9���

Page 63: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

����PM�����• Folded Clos �, N=3�6�����

• �• �• �

ñá= 0,0,1,0,0,1,0,0,11torPM ñá= 0,0,0,0,0,1,0,0,22

torPM

ñá= 0,0,0,0,0,1,0,1,13torPM ñá= 0,0,0,0,0,0,0,1,24

torPMñá= 0,0,0,0,0,0,1,1,15

torPM ñá= 0,0,0,0,0,0,0,0,36torPM

���Hadoop��

Page 64: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��-�������• ������� (α=0.05)

• �����– 99.9%à(W,R)=(2,1)��– 99%à(W,R)=(3,1)��

Page 65: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������• ����– !�""����– !�"�"���– �!�"�����"���– ���� !�"���!�"

65

Page 66: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

Web������• Web������ �HTTP��– Ebay Web��– AWS

• �������,�����– �������

• ����• �����(2PC)• � �• Paxos

Page 67: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

����Rep4WS

Page 68: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��Paxos�����Client

WS Replica1

WS Replica2

WS Replica3

(the leader)

time

REQUEST

NUMBER

PROPOSE

PROPO

SE

PROPOSE

LOGGING

LOGGING

LOGGING

ACK

AC

K

LEARN EXECUTE

EXECUTE

EXECUTELEA

RN

LEARN

RESPOND

RESPONDRESPOND

REP

LY

NUMBER: �.!�2�*��1,2,3…

ACK: �+$(�n/2+1�ACK#�

EXECUTE: �%.!&*����,����*�.!�����0,(�

-1�(AGREEMENT PHASE) �,1�(EXECUTION PHASE)

(follower)

(follower)

��.!/�(RDG)&���,�

AGREEMENT PHASE: �'���α0," )�,(Pipeline Concurrency)

Page 69: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

' .!���1• )��-+API������

– ����!' �.���&/����0%1�,�' "����!(commutable)

– �' �2������ $(�"��( $)�#*���!��' �(�(RDG)

• WebShopService– getMyCart– addToCart– searchProduct– orderProduct– modifyMyProfile

getMyCart

searchProduct

Commutable

getMyCart

addToCart

Can not commute

Page 70: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�����

WSOP1 × × × √ ×OP2 √ × √ √OP3 √ × √OP4 × ×OP5 √

OP2 OP1 OP2 OP4

OP4 OP5

OP2

OP3

n+1 n+2 n+3 n+4 n+5 n+6 n+7 n+8

����

���

���OP2��n+1

Page 71: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��RDG�����

LEARN:(n+1)(n+5)(n+3)(n+8)(n+4)(n+6)(n+2)(n+7)

n+1 n+2 n+3 n+4 n+5 n+6 n+7 n+8

n+1

n+2

n+3n+7

n+8

n+4

n+5

n+6

EXECUTE ExecutedWaitWaitWaitEXECUTE Executed

EXECUTE Executed

WaitEXECUTE Executed

EXECUTE Executed

EXECUTE Executed

EXECUTE Executed

EXECUTE Executed

����: 3+4+4+1=12 units

���RDG����� : 5+4+4+2+1=16 units

RDG of Pipeline[n+1,n+8]

Page 72: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

RDG������������&• RDG��&

• ������&– ��RDG�����"��!$���������"��# � ����%

n+8

RDGn+8

n+1 n+2

n+7

n+3

n+4 n+5n+6

n

(1)���0�RDG����0�(2)����(i+1)�����"!���RDG�����,�(i/2)�

��"(� )�RDG��������

Page 73: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��0�• �$1(leader)��– ��Paxos!��$1)�– ����.��

• �$1(follower)��– ��0�(checkpoint)– ���+leader ���%",�*�leader�-#!&�

– ('��leader!�.• 0�à�!-#/�.

Page 74: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

����

Replica Number

Res

pons

e Ti

me(

ms)

0

20

40

60

80

100

120

3 4 5 6 7

2PC

GC-TCP

Rep4WS/Basic Paxos

����

Load(reqs/sec)R

espo

nse

Tim

e(m

s)

����

Replica Number=3

0

50

100

150

200

250

300

350

10 20 30 40 50 60 70 80 90 100

2PC

GC-TCP

Basic Paxos

Rep4WS

Page 75: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

RDG�����

…n n+1 n+2 n+12 n+13 n+14

n n+1 n+5 n+6

n+7

…n+8 n+9 n+13 n+14

…n n+1 n+2 n+12 n+13 n+14

RDG1

RDG2

RDG3

read operation write operation

����RDGs

0

50

100

150

200

250

10 20 30 40 50 60 70 80 90 100

RDG

RDG

RDG

Res

pons

e Ti

me(

ms)

1

2

3

(No RDG)

Load(reqs/sec)

������7�4.2�0

RDG3 ����RDG1�15%-66%�� ��26%�

Page 76: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������• ����– !�""����– !�"�"���– �!�"�����"���– ���� !�"���!�"

76

Page 77: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�-J:K• ��+)�H1�-�6�– )�O1(<7-���)�P1(<7-�G�O�PH!�2�-�6�A�P⇝O

• � �3 841.0'A�=CB2.0'B�=CB$A1CBF<�L– E��CB)�H!��-�6à*H���6

• �-J:K�@�)��%– ;J�"'5?#�.��)�1(<7-�G�D"'5&I>9?#��/�E�)��1�,�-�61���)�1(<7-

77

Page 78: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

���������% • &���&����$– �$(�– �������

• ����– �&"�� ��– ��"����'����

• Lamport�$– #!�$

78

Page 79: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�,I9J+�3 J1E• �@?�Monotonic reads�

– �I�$4���6?�1�I��7.�F?1�D*• �@��Monotonic writes�

– ���$4%?�1I��$4��( 1&;G"�>�$4�( 12�&;G"I9

• ��?�Read your writes�– �I�$4����?�1)'�7.�1)'D*

• ?���Writes Follow Reads�– �I�$4��?�-��( a15,���b�C����$4:7?�b15,��#H8�<7?�a15,

• �/A���7B�0$�,!=79

Page 80: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������• ���– ������������������

• ��������– ������������������

• � – � w(x=1) ⇝ w(x=2)��x��������1

80

Page 81: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�P�.S>T5B:• �)50D– <*(R5N, LG+O">��;

• �.�95� ��J– 3��)$F�)50D��A'��– 3��S�*(��#8���� �)50D

• �.�92/– �#8E�*(+�QB1(-�C!7��I��.�95LG+O" ts

– �,$?�*(5+O"�4 ts�%<KH2/– ���&@ME)�6=E��<KH2/5*(

81

Page 82: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

�� ���

�����������20% 40% 60% 80%

Cassandra-Eventual 0.16% 0.08% 0.03% 0.02%Cassandra-RYW >0 >0 >0 >0

CoCaCo 0 0 0 0

82

���� ������

Cassandra: �� ����������CoCaCo��������

Page 83: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

��������

83

Page 84: xuwangxuwang.tech/course/sc2019fall/08-sc2019fall-20190731.pdf · • availability): 7X24 – (SLA): •Amazon EC2 99.95%, Google Cloud Storage 99.9%, Microsoft Windows Azure 99.9%

Q&A