rabin’s(( informaon(dispersal(algorithm:(( a(prescientlook(at(...
TRANSCRIPT
![Page 1: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/1.jpg)
Rabin’s Informa.on Dispersal Algorithm:
A Prescient Look at Coding on Networks
Michael Mitzenmacher
![Page 2: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/2.jpg)
Coding in Networks, Backwards
Network Coding
Fountain Codes
Informa.on Dispersal
![Page 3: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/3.jpg)
JACM, April 1989
![Page 4: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/4.jpg)
The IDA Approach
• A file consists of N symbols = numbers mod p for large prime p.
• Split the file up into N/m pieces of m symbols • For each piece, derive n encoded symbols, each by a linear combina.on of the m symbols. – Use the same coefficients for the linear combina.ons for each piece
• Derive n packets, with the ith packet containing the ith derived symbol for each of the N/m pieces.
• Given m packets, with N total symbols, can invert a matrix to solve for the original message.
![Page 5: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/5.jpg)
Coding
€
b1 b2 b3 ... bmbm+1 bm+2 bm+3 ... b2m....
bN −m+1 bN −m+2 bN −m+3 bN
€
€
c1 c2 ... cncn+1 cn+2 ... c2n....
cNn /m−n+1 cNn /m−n+2 cNn /m
€
cin+ j = (a j,1,a j,2,...a j ,m ) • (bim+1,bim+2,...bi(m+1))
Packets
![Page 6: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/6.jpg)
Decoding
€
€
c1 c2 ... cmcn+1 cn+2 ... cn+m
....cNn /m−n+1 cNn /m−n+2 cNn /m−n+m
For convenience, assume you get first m packets
€
Let A = (ai, j ),1≤ i, j ≤ m
€
A •
b1bm
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥
=
c1cm
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥
€
b1bm
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥
= A−1 •
c1cm
⎡
⎣
⎢ ⎢ ⎢
⎤
⎦
⎥ ⎥ ⎥
Need to be able to invert coefficient matrix, for any set of packets that is received.
![Page 7: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/7.jpg)
Coefficients
• Need the corresponding matrix to be inver.ble.
• Solu.on 1: Coefficient chosen according to a Cauchy matrix. – Resul.ng m by m matrix of received coefficients can be inverted in Θ(m2) .me.
• Solu.on 2: Vandermonde matrix.
• Solu.on 3: Choose the coefficients ai,j randomly. – Linearly independent with high probability. – But Θ(m3) decoding by standard means.
![Page 8: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/8.jpg)
Path to Digital Fountains
• Quadra.c decoding + field opera.ons too slow for big files. – “Obvious” solu.on – break file up into smaller subfiles and encode those – has various problems in many seangs.
• Inspired Michael Luby (and others) to search for improvements.
![Page 9: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/9.jpg)
Cauchy Codes
![Page 10: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/10.jpg)
Cauchy Codes
![Page 11: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/11.jpg)
XOR-‐Based Cauchy Codes
• Paper: An XOR-‐Based Erasure-‐Resilient Coding Scheme (Blomer, Kalfane, Karp, Karpinski, Luby, Zuckerman)
• Design code that allows field opera.ons to be replaced by XOR opera.ons.
• Directly u.lizes the Cauchy code framework.
• S.ll Θ(m2) .me, but much faster in prac.ce.
![Page 12: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/12.jpg)
Tornado Codes
• Paper: Efficient Erasure Correc.ng Codes (Luby, Mitzenmacher, Shokrollahi, Spielman)
• Replace field opera.ons with XORS • Packets are derived by sparse random combina.ons of symbols. – Prescience of Rabin’s use of random matrices. – Re-‐deriving ideas by Gallager : low-‐density parity-‐check codes.
• Technical challenge : designing the right sparsity. – Irregular combina.ons of symbols.
![Page 13: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/13.jpg)
13
Low Density Parity Check Codes Setup
• View code as a bipar.te graph. • Variable nodes on leg. • Check nodes on right. • Codewords sa.sfy: sum of all
neighbors of a check node is 0. • Note: sum allows general q.
For bits/ packets, sum is XOR. • Regular degree distribu.ons
(all nodes on leg have same degree, all nodes on right have same degree NOT op.mal.)
• Sparsity leads to faster decoding.
D
A
B
C
H
F
E
I
J
G
K
![Page 14: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/14.jpg)
14
Decoding Process: Recovery
![Page 15: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/15.jpg)
Digital Fountain Paradigm
• Paper : A Digital Fountain Approach to Reliable Distribu.on of Bulk Data (Byers, Luby, Mitzenmacher, Rege) – See also Digital Fountains: A Survey and Look Forward (Mitzenmacher)
![Page 16: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/16.jpg)
What is a Digital Fountain?
• A digital fountain is an idealized paradigm for data transmission. – Vs. the standard (TCP) paradigm: data is an ordered finite sequence of bytes.
• Instead, with a digital fountain, a m symbol file yields an infinite data stream; once you have received any m symbols from this stream, you can quickly reconstruct the original file.
![Page 17: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/17.jpg)
Ideals for Digital Fountain
• O(1) .me to generate new encoded packet • “Infinite” supply of packets that can be generated online.
• Informa.on-‐theore.cally op.mal – receive only m packets to decode m packet message.
• Linear decoding .me.
• …
![Page 18: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/18.jpg)
Tornado to LT to Raptor…
• Con.nuing improvements in codes to get closer and closer to Digital Fountain paradigm.
• Rabin’s scheme is informa.on-‐theore.cally op.mal. – Varia.on of Reed-‐Solomon codes.
• Others are not – require slightly more than m packets received to recover m packets of info. – To get the large decoding speed gain.
![Page 19: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/19.jpg)
Raptor Codes
• Developed by Amin Shokrollahi. • Closest to Digital Fountain paradigm. • Key ideas: – Precode data with a fixed-‐rate erasure code.
• So you only need to recover 99% of the data. – Then encode by taking a random XOR of a random number of symbols. • According to right distribu.on on number of symbols.
– Near-‐infinite supply of packets, generated online in constant .me, very close to informa.on theore.cally op.mal
![Page 20: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/20.jpg)
Prescience : Applica.ons
• The IDA paper suggests many applica.ons. – Fault-‐tolerant distributed storage.
• Node failures. – Fault-‐tolerant transmission.
• Link failures. • Mul.path rou.ng ; load distribu.on.
– Rou.ng in parallel computers. • Studies the hypercube; Valiant-‐style rou.ng for large messages.
![Page 21: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/21.jpg)
Recurring Themes
• Same issues re-‐examined with new codes. – Parallel downloading
• Accessing Mul.ple Mirror Sites in Parallel – Dissemina.on in overlay networks
• Bullet : High Bandwidth Data Dissemina.on Using an Overlay Mesh
• Informed Content Delivery Across Adap.ve Overlay Networks
– Storage • Distributed Fountain Codes for Network Storage • On the Prac.cal Use of LDPC Codes for Distributed Storage
• And many, many more papers.
![Page 22: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/22.jpg)
Network Coding
• What we missed : coding beyond the endpoints, inside the network.
• Main idea of network coding : can “combine” encoded packets inside the network to derive new encoded packets.
![Page 23: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/23.jpg)
Network Coding : A Standard Approach
• E.g., A Random Linear Network Coding Approach to Mul.cast (Ho, Medard, Koeqer, Karger, Effros, Shi, Leong)
• A packet contains (one or more) linear combina.ons (over a suitable field) of symbols.
• If two packets meet in the network (say at a buffer), can derive a new linear combina.on by mul.plying each packet by a random mul.plier.
• If old packets had random coefficients, so do the new packets! – Implies likelihood of inver.bility s.ll high.
![Page 24: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/24.jpg)
Prescience : Random Coding
……
![Page 25: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/25.jpg)
Keys to Random Linear Network Coding
• Random linear network coding depends significantly on random coefficients, keeping coefficients in the header. – Both ideas in the IDA paper…
• Neither really needed to be in the IDA paper. – Had alternate schemes, including more efficient Cauchy schemes.
• But Michael has always understood the importance and u.lity of random methods. – Even if the applica.on was not immediate.
![Page 26: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/26.jpg)
Network Coding + TCP
4 2 5 0 0 0 0 3 1 2 5 0 0 0 1 2 3 4 1 0 0 3 3 1 2 1 0 0 1 2 5 4 5 0 0
4 2 5 0 0 0 0 3 1 2 5 0 0 0 1 2 3 4 1 0 0 3 3 1 2 1 0 0 1 2 5 4 5 0 0
4p1 + 2p2 + 5p3
Original message : p1, p2, p3…
Coded Packets
c1 c2 c3 c4 c5
When c1 comes in, you’ve “seen” packet 1; eventually you’ll be able to decode it. And so on…
![Page 27: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/27.jpg)
Network Coding + TCP
1 4 5 3 0 0 0 0 1 3 2 6 0 0 0 0 1 6 2 0 0 0 0 0 1 5 0 0 0 0 0 0 1 0 0
4 2 5 0 0 0 0 3 1 2 5 0 0 0 1 2 3 4 1 0 0 3 3 1 2 1 0 0 1 2 5 4 5 0 0
4p1 + 2p2 + 5p3
Original message : p1, p2, p3…
Coded Packets
c1 c2 c3 c4 c5
Use par.al Gaussian elimina.on as packets arrive to check for a packet that can be removed from the TCP buffer at the sender and send a corresponding ACK.
![Page 28: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/28.jpg)
Prescience : Security
• Revela.on of informa.on (IDA is not a “secret sharing scheme”). – Lightweight encryp.on schemes.
• Aqack : inser.on of fake packets. – Digital signature schemes.
![Page 29: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/29.jpg)
Repeated Themes
• XORs – XORs in the Air: Prac.cal Wireless Network Coding
• Applica.ons – Network Coding for Large Scale Content Distribu.on – A Random Linear Network Coding Approach to Mul.cast
• Security – Secure Network Coding – An Efficient Signature-‐Based Scheme for Securing Network Coding Against Pollu.on Aqacks
![Page 30: Rabin’s(( Informaon(Dispersal(Algorithm:(( A(PrescientLook(at( …people.seas.harvard.edu/~salil/rabin2011-slides/rabin... · 2011-09-06 · Network(Coding(:((A(Standard(Approach(•](https://reader033.vdocuments.us/reader033/viewer/2022042313/5edc5e14ad6a402d6667003e/html5/thumbnails/30.jpg)
Conclusion
• Coding schemes for networks on the rise in the last twenty ears. – In research, and increasingly in prac.ce.
• Rabin’s IDA paper laid founda.ons for much of what has followed. – Covers many ideas and issues that re-‐appear throughout subsequent work.
• A paper ahead of its .me. – An inspira.on to those working on the boundary of coding and theore.cal computer science.