the capacity of the relay channel - isr.umd.edu · summary solved an problem posed by cover and...
TRANSCRIPT
![Page 1: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/1.jpg)
Cover’s Open Problem:“The Capacity of the Relay Channel”
Ayfer Ozgur
Stanford University
Advanced Networks Colloquia SeriesUniversity of Maryland, March 2017
Joint work with Xiugang Wu and Leighton Pate Barnes.
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 1 / 26
![Page 2: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/2.jpg)
Father of the Information Age
Claude Shannon (1916-2001)
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 2 / 26
![Page 3: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/3.jpg)
The Bell System Technical JournalVol. XXVII J Illy, 1948 No.3
A Mathematical Theory of CommunicationBy c. E. SHANNON
IXTRODUCTION
T HE recent development of various methods of modulation such as reMand PPM which exchange bandwidth for signal-to-noise ratio has in-
tensified the interest in a general theory of communication. A basis forsuch a theory is contained in the important papers of Nyquist! and Hartley"on this subject. In the present paper we will extend the theory to include anumber of new factors, in particular the effect of noise in the channel, andthe savings possible due to the sta tistiral structure of the original messageand due to the nature of the final destination of the information.
The fundamental problem of communication is that of reproducing atone point either exactly or approximately a message selected at anotherpoint. Frequently the messages have meaning; that is they refer to or arecorrelated according to some system with certain physical or conceptualentities. These semantic aspects of communication are irrelevant to theengineering problem. The significant aspect is that the actual message isone selected from a set of possible messages. The system must be designedto operate for each possible selection, not just the one which will actuallybe chosen since this is unknown at the time of design.
If the number of messages in the set is finite then this number or anymonotonic function of this number can be regarded as a measure of the in-formation produced when one message is chosen from the set, all choicesbeing equally likely. As was pointed out by Hartley the most naturalchoice is the logarithmic function. Although this definition must be gen-eralized considerably when we consider the influence of the statistics of themessage and when we have a continuous range of messages, we will in allcases use an essentially logarithmic measure.
The logarithmic measure is more convenient for various reasons:1. It is practically more useful. Parameters of engineering importance
1 Nyquist, H., "Certain Factors Affecting Telegraph Speed," Belt System Tectmical J OUT-
nal, April 1924, p, 324; "Certain Topics in Telegraph Transmission Theory," A. I. E. E.TI aIlS., v. 47, April 1928, p. 617.
2 Hartley. R. V. L.. "Transmission oi Information.' Belt System Technical Journal, July1928, p. .'US.
379
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 3 / 26
![Page 4: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/4.jpg)
“A method is developed for representing any communication systemgeometrically...”
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 4 / 26
![Page 5: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/5.jpg)
AWGN Channel
Transmitter Receiver
P
Capacity
C = log
(1 +
P
N
)
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 5 / 26
![Page 6: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/6.jpg)
Converse: Sphere Packing
pnN
pn(P + N)
pnN
pnN
Y sphere
0
Xn(1)
Xn(2)
Xn(3)
noise sphere
# of X n ≤
∣∣∣Sphere(√
n(P + N))∣∣∣
∣∣∣Sphere(√
nN)∣∣∣
.=
2n2log 2πe(P+N)
2n2log 2πeN
= 2n2log(1+ P
N )
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 6 / 26
![Page 7: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/7.jpg)
Achievability: Geometric Random Coding
Pr(∃ false X n) ≤ |Lens|∣∣∣Sphere(√
nP)∣∣∣× 2nR
.=
2n2log 2πe PN
P+N
2n2log 2πeP
× 2nR
= 2−n2log(1+ P
N ) × 2nR
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 7 / 26
![Page 8: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/8.jpg)
The story goes...
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 8 / 26
![Page 9: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/9.jpg)
Cover’s Open Problem
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 9 / 26
![Page 10: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/10.jpg)
Gaussian case
In = f(Zn)
Source
Relay
Destination
W1 ⇠ N (0, N)
W2 ⇠ N (0, N)
Xn Y n
Zn
C0
C (∞) =1
2log
(1 +
2P
N
)
Achievability: C ∗0 =∞.
Cutset-bound (Cover and El Gamal’79):
C ∗0 ≥1
2log
(1 +
2P
N
)− 1
2log
(1 +
P
N
).
Potentially, C ∗0 → 0 as P/N → 0.
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 10 / 26
![Page 11: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/11.jpg)
Main Result
In = f(Zn)
Source
Relay
Destination
W1 ⇠ N (0, N)
W2 ⇠ N (0, N)
Xn Y n
Zn
C0
Theorem
C ∗0 =∞
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 11 / 26
![Page 12: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/12.jpg)
Upper Bound on the Capacity
0 0.1 0.2 0.3 0.7 0.8 0.9 12.5
2.6
2.7
2.8
2.9
3
3.1SNR = 15 dB
Cut-set boundC-FOld boundNew bound
0.4 0.5 0.6 C0 (bit/channel use)
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 12 / 26
![Page 13: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/13.jpg)
Cutset Bound
In = f(Zn)
Source
Relay
Destination
W1 ⇠ N (0, N)
W2 ⇠ N (0, N)
Xn Y n
Zn
C0
nR ≤ I (X n;Y n, In) + nεn
= I (X n;Y n) + I (X n; In|Y n) + nεn
= I (X n;Y n) + H(In|Y n)︸ ︷︷ ︸≤nC0
−H(In|Y n,X n)︸ ︷︷ ︸≥0
+nεn
≤ n(I (X ;Y ) + C0 + εn)
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 13 / 26
![Page 14: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/14.jpg)
Cutset Bound
In = f(Zn)
Source
Relay
Destination
W1 ⇠ N (0, N)
W2 ⇠ N (0, N)
Xn Y n
Zn
C0
nR ≤ I (X n;Y n, In) + nεn
= I (X n;Y n) + I (X n; In|Y n) + nεn
= I (X n;Y n) + H(In|Y n)︸ ︷︷ ︸≤nC0
−H(In|X n)︸ ︷︷ ︸≥0
+nεn
≤ n(I (X ;Y ) + C0 + εn)
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 13 / 26
![Page 15: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/15.jpg)
pnN
Xn
Typical set of Zn/Y n
✓ In-th bin
Zn
Y n
If H(In|X n) = 0, then H(In|Y n) = 0.
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26
![Page 16: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/16.jpg)
pnN
Xn
Typical set of Zn/Y n
✓ In-th bin
Zn
Y n
If H(In|X n) = 0,
then H(In|Y n) = 0.
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26
![Page 17: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/17.jpg)
pnN
Xn
Typical set of Zn/Y n
✓ In-th bin
Zn
Y n
If H(In|X n) = 0, then H(In|Y n) = 0.
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26
![Page 18: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/18.jpg)
In = f(Zn)
Source
Relay
Destination
W1 ⇠ N (0, N)
W2 ⇠ N (0, N)
Xn Y n
Zn
C0
R ≤ I (X n;Y n) + H(In|Y n)︸ ︷︷ ︸≤?
− H(In|X n)︸ ︷︷ ︸=−n log sin θn
+nεn
Goal:
In = f (Zn)− Zn − X n
︸ ︷︷ ︸H(In|X n)=−n log sin θn
−Y n
︸ ︷︷ ︸H(In|Y n)≤?
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 15 / 26
![Page 19: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/19.jpg)
H(In|X n) 6= 0
Multiple bins
Xn
pnN
Typical set of Zn
# of bins =? P(each bin) =?
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 16 / 26
![Page 20: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/20.jpg)
From n- to nB- Dimensional Space
X,Y,Z, I : B-length i.i.d. from {(X n(b),Y n(b),Zn(b), In(b))}Bb=1.
If H(In|X n) = −n log sin θn, then for any typical (x, i)
p(i|x).
= 2nB log sin θn ,
P(Z ∈ A(i)|x).
= 2nB log sin θn
pnBN
x
Typical set of Z/Y
Pr.= 2nB log sin ✓n
ith bin Ax(i)
|Ax(i)| .= 2nB( 1
2 log 2⇡eN sin2 ✓n)
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 17 / 26
![Page 21: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/21.jpg)
Isoperimetric Inequalities
Isoperimetric Inequality in the Plane (Steiner 1838)
Among all closed curves in the plane with a given enclosed area, thecircle has the smallest perimeter.
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 18 / 26
![Page 22: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/22.jpg)
Isoperimetric Inequalities
Isoperimetric Inequality on the Sphere (Levy 1919)
Among all sets on the sphere with a given volume, the spherical caphas the smallest boundary, or the smallest volume of ω-neighborhoodfor any ω > 0.
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 19 / 26
![Page 23: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/23.jpg)
Blowing-up Lemma
|Ax(i)| .= 2nB( 1
2 log 2⇡eN sin2 ✓n)
Isoperimetric Inequality on the Sphere + Measure Concentration:
P(Z ∈ blow-up of Ax(i)|x) ≈ 1.
⇓P(Y ∈ blow-up of Ax(i)|x) ≈ 1.
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 20 / 26
![Page 24: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/24.jpg)
Geometry of Typical Sets
n-dimensional space:Almost all (X n,Y n,Zn, In)
pnN
XnY n
Zn ! In
nB-dimensional space:Almost all (x, y, i)
pnBN
z ! i
xy
⇡
2� ✓n
Information Inequality: (Wu and Ozgur, 2015)
H(In|Y n) ≤ n(2 log sin θn +√
2 log sin θn ln 2 log e).
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 21 / 26
![Page 25: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/25.jpg)
A new approach
Ax(i)
Y
Ax(i)
Y
Control the intersection of a sphere drawn around a randomly chosen Yand Ax(i).
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 22 / 26
![Page 26: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/26.jpg)
Easy if Ax(i) is a spherical cap
pnBN
z0
y0x
✓n
Cap(z0, ✓n)
Cap(y0,!n)
Cap(z0, ✓n) \ Cap(y0,!n)
!n
|Cap(z0, θn) ∩ Cap(Y, ωn)| .= 2nB( 12log 2πeN(sin2θn−cos2 ωn))
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 23 / 26
![Page 27: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/27.jpg)
Strengthening of the Isoperimetric Inequality
Strengthening of the Isoperimetric Inequality:
Among all sets on the sphere with a given volume, the spherical cap hasminimal intersection volume at distance ω for almost all points on the spherefor any ω > π/2− θ.
Proof: builds on the Riesz rearrangement inequality.
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 24 / 26
![Page 28: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/28.jpg)
A Packing Argument
Given Y,# of I ≤
∣∣∣Sphere(Y,√nBN4sin2 ωn
2
)∣∣∣
2nB( 12log 2πeN(sin2θn−cos2 ωn))
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 25 / 26
![Page 29: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,](https://reader030.vdocuments.us/reader030/viewer/2022041210/5dd0b29dd6be591ccb624167/html5/thumbnails/29.jpg)
Summary
Solved an problem posed by Cover and named “The Capacity ofRelay Channel” in Open Problems in Communication andComputation, Springer-Verlag, 1987.
Developed a converse technique that significantly deviates fromstandard converse techniques based on single-letterization and hassome new ingredients:
I TypicalityI Measure ConcentrationI Isoperimetric Inequality
Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 26 / 26