![Page 1: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/1.jpg)
Third-Order Asymptotics: Old and New
Vincent Y. F. Tan (National University of Singapore; NUS)
Joint works with M. Tomamichel, Y. Sakai and M. Kovacevic
Workshop on Probability and Information Theory(held at the University of Hong Kong, on 20th August 2019)
1 / 42
![Page 2: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/2.jpg)
Outline
1 Introduction
2 Old Contribution
3 New Contribution
2 / 42
![Page 3: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/3.jpg)
Outline
1 Introduction
2 Old Contribution
3 New Contribution
3 / 42
![Page 4: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/4.jpg)
Introduction: Transmission of InformationShannon’s Figure 1
TRANSMITTER
MESSAGE
SIGNAL RECEIVEDSIGNAL
RECEIVER DESTINATION
MESSAGE
NOISESOURCE
INFORMATIONSOURCE
Shannon abstracted away information meaning, “semantics”• treat all data equally — bits as a “universal currency”• crucial abstraction for modern communication and computing systems
Also relaxed computation and delay constraints to discover a fundamental limit: capacity, providing a goal-post to work toward
Saturday, June 11, 2011
Figure: Shannon’s Figure 1
• Information theory ≡ Finding fundamental limits for reliableinformation transmission
• Channel coding: Concerned with the maximum rate ofcommunication in bits/channel use
4 / 42
![Page 5: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/5.jpg)
Introduction: Transmission of InformationShannon’s Figure 1
TRANSMITTER
MESSAGE
SIGNAL RECEIVEDSIGNAL
RECEIVER DESTINATION
MESSAGE
NOISESOURCE
INFORMATIONSOURCE
Shannon abstracted away information meaning, “semantics”• treat all data equally — bits as a “universal currency”• crucial abstraction for modern communication and computing systems
Also relaxed computation and delay constraints to discover a fundamental limit: capacity, providing a goal-post to work toward
Saturday, June 11, 2011
Figure: Shannon’s Figure 1
• Information theory ≡ Finding fundamental limits for reliableinformation transmission
• Channel coding: Concerned with the maximum rate ofcommunication in bits/channel use
4 / 42
![Page 6: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/6.jpg)
Channel Coding (One-Shot)
M X Ye W d
M
• A code is an triple C = {M, e, d} where M is the message set
• The average error probability perr(C) is
perr(C) := Pr[M 6= M
]where M is uniform on M
• Maximum code size at ε-error is
M∗(W , ε) := sup{m∣∣ ∃ C s.t. m = |M|, perr(C) ≤ ε
}
5 / 42
![Page 7: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/7.jpg)
Channel Coding (One-Shot)
M X Ye W d
M
• A code is an triple C = {M, e, d} where M is the message set
• The average error probability perr(C) is
perr(C) := Pr[M 6= M
]where M is uniform on M
• Maximum code size at ε-error is
M∗(W , ε) := sup{m∣∣ ∃ C s.t. m = |M|, perr(C) ≤ ε
}
5 / 42
![Page 8: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/8.jpg)
Channel Coding (One-Shot)
M X Ye W d
M
• A code is an triple C = {M, e, d} where M is the message set
• The average error probability perr(C) is
perr(C) := Pr[M 6= M
]where M is uniform on M
• Maximum code size at ε-error is
M∗(W , ε) := sup{m∣∣ ∃ C s.t. m = |M|, perr(C) ≤ ε
}5 / 42
![Page 9: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/9.jpg)
Channel Coding (n-Shot)
M X n Y ne W n d
M
• Consider n independent uses of a channel
• Assume W is a discrete memoryless channel
• For vectors x = (x1, . . . , xn) ∈ X n and y := (y1, . . . , yn) ∈ Yn,
W n(y|x) =n∏
i=1
W (yi |xi )
• Maximum code size at average error ε and blocklength n is
M∗(W n, ε)
• Consider both discrete- and continuous-time channels.
6 / 42
![Page 10: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/10.jpg)
Channel Coding (n-Shot)
M X n Y ne W n d
M
• Consider n independent uses of a channel
• Assume W is a discrete memoryless channel
• For vectors x = (x1, . . . , xn) ∈ X n and y := (y1, . . . , yn) ∈ Yn,
W n(y|x) =n∏
i=1
W (yi |xi )
• Maximum code size at average error ε and blocklength n is
M∗(W n, ε)
• Consider both discrete- and continuous-time channels.
6 / 42
![Page 11: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/11.jpg)
Channel Coding (n-Shot)
M X n Y ne W n d
M
• Consider n independent uses of a channel
• Assume W is a discrete memoryless channel
• For vectors x = (x1, . . . , xn) ∈ X n and y := (y1, . . . , yn) ∈ Yn,
W n(y|x) =n∏
i=1
W (yi |xi )
• Maximum code size at average error ε and blocklength n is
M∗(W n, ε)
• Consider both discrete- and continuous-time channels.
6 / 42
![Page 12: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/12.jpg)
Channel Coding (n-Shot)
M X n Y ne W n d
M
• Consider n independent uses of a channel
• Assume W is a discrete memoryless channel
• For vectors x = (x1, . . . , xn) ∈ X n and y := (y1, . . . , yn) ∈ Yn,
W n(y|x) =n∏
i=1
W (yi |xi )
• Maximum code size at average error ε and blocklength n is
M∗(W n, ε)
• Consider both discrete- and continuous-time channels.
6 / 42
![Page 13: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/13.jpg)
Channel Coding (n-Shot)
M X n Y ne W n d
M
• Consider n independent uses of a channel
• Assume W is a discrete memoryless channel
• For vectors x = (x1, . . . , xn) ∈ X n and y := (y1, . . . , yn) ∈ Yn,
W n(y|x) =n∏
i=1
W (yi |xi )
• Maximum code size at average error ε and blocklength n is
M∗(W n, ε)
• Consider both discrete- and continuous-time channels.
6 / 42
![Page 14: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/14.jpg)
Outline
1 Introduction
2 Old Contribution
3 New Contribution
7 / 42
![Page 15: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/15.jpg)
Old Contribution
• Upper bound logM∗(W n, ε) for n large(converse)
• Concerned with the third-order term ofthe asymptotic expansion
• Going beyond the normal approx termsM. Tomamichel
Theorem (Tomamichel-Tan (2013))
For all DMCs with positive ε-dispersion Vε,
logM∗(W n, ε) ≤ nC +√nVεΦ
−1(ε) +1
2log n + O(1)
where Φ(a) :=∫ a−∞
1√2π
exp(−1
2x2)dx
8 / 42
![Page 16: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/16.jpg)
Old Contribution
• Upper bound logM∗(W n, ε) for n large(converse)
• Concerned with the third-order term ofthe asymptotic expansion
• Going beyond the normal approx termsM. Tomamichel
Theorem (Tomamichel-Tan (2013))
For all DMCs with positive ε-dispersion Vε,
logM∗(W n, ε) ≤ nC +√
nVεΦ−1(ε) +
1
2log n + O(1)
where Φ(a) :=∫ a−∞
1√2π
exp(−1
2x2)dx
8 / 42
![Page 17: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/17.jpg)
Old Contribution: Remarks
• Our bound
logM∗(W n, ε) ≤ nC +√
nVεΦ−1(ε) +
1
2log n + O(1)
• Best upper bound till date:
logM∗(W n, ε) ≤ nC +√
nVεΦ−1(ε)+
(|X | − 1
2
)log n+O(1)
V. Strassen (1964) Polyanskiy-Poor-Verdu (2010)
• Requires new converse techniques
9 / 42
![Page 18: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/18.jpg)
Old Contribution: Remarks
• Our bound
logM∗(W n, ε) ≤ nC +√
nVεΦ−1(ε) +
1
2log n + O(1)
• Best upper bound till date:
logM∗(W n, ε) ≤ nC +√
nVεΦ−1(ε)+
(|X | − 1
2
)log n+O(1)
V. Strassen (1964) Polyanskiy-Poor-Verdu (2010)
• Requires new converse techniques
9 / 42
![Page 19: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/19.jpg)
Old Contribution: Remarks
• Our bound
logM∗(W n, ε) ≤ nC +√
nVεΦ−1(ε) +
1
2log n + O(1)
• Best upper bound till date:
logM∗(W n, ε) ≤ nC +√
nVεΦ−1(ε)+
(|X | − 1
2
)log n+O(1)
V. Strassen (1964) Polyanskiy-Poor-Verdu (2010)
• Requires new converse techniques
9 / 42
![Page 20: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/20.jpg)
Related Work: Third-Order Term
• Recall that we are interested in quantifying the third-orderterm ρn
ρn = logM∗(W n, ε)−[nC +
√nVεΦ
−1(ε)]
• ρn = O(log n) if channel is non-exotic
• ρn may be important at very short blocklengths
10 / 42
![Page 21: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/21.jpg)
Related Work: Third-Order Term
• Recall that we are interested in quantifying the third-orderterm ρn
ρn = logM∗(W n, ε)−[nC +
√nVεΦ
−1(ε)]
• ρn = O(log n) if channel is non-exotic
• ρn may be important at very short blocklengths
10 / 42
![Page 22: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/22.jpg)
Related Work: Third-Order Term
• Recall that we are interested in quantifying the third-orderterm ρn
ρn = logM∗(W n, ε)−[nC +
√nVεΦ
−1(ε)]
• ρn = O(log n) if channel is non-exotic
• ρn may be important at very short blocklengths
10 / 42
![Page 23: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/23.jpg)
Related Work: Third-Order Term
ρn = logM∗(W n, ε)−[nC +
√nVΦ−1(ε)
]• For the BSC [PPV10]
ρn =1
2log n + O(1)
• For the BEC [PPV10]
ρn = O(1)
• For the AWGN under maximum (or peak) power constraints[PPV10, Tan-Tomamichel (2015)]
ρn =1
2log n + O(1)
11 / 42
![Page 24: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/24.jpg)
Related Work: Third-Order Term
ρn = logM∗(W n, ε)−[nC +
√nVΦ−1(ε)
]• For the BSC [PPV10]
ρn =1
2log n + O(1)
• For the BEC [PPV10]
ρn = O(1)
• For the AWGN under maximum (or peak) power constraints[PPV10, Tan-Tomamichel (2015)]
ρn =1
2log n + O(1)
11 / 42
![Page 25: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/25.jpg)
Related Work: Third-Order Term
ρn = logM∗(W n, ε)−[nC +
√nVΦ−1(ε)
]• For the BSC [PPV10]
ρn =1
2log n + O(1)
• For the BEC [PPV10]
ρn = O(1)
• For the AWGN under maximum (or peak) power constraints[PPV10, Tan-Tomamichel (2015)]
ρn =1
2log n + O(1)
11 / 42
![Page 26: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/26.jpg)
Related Work: Achievability for Third-Order Term
Proposition (Polyanskiy (2010))
Assume that all elements of {W (y |x) : x ∈ X , y ∈ Y} are positiveand C > 0. Then,
ρn ≥1
2log n + O(1)
• This is an achievability result but BEC doesn’t satisfy assumptions
• Assumption may be relaxed to
∃P ∈ Π s.t. V r(P,W ) := V
(PW ,
P ×W
PW
)> 0
• Based on the concentration bound [Polyanskiy’s thesis]
E
[exp
(n∑
i=1
Xi
)I
{n∑
i=1
Xi ≥ γ
}]≤ 2
(log 2√
2π+
12T
σ
)exp(−γ)
σ√n
.
12 / 42
![Page 27: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/27.jpg)
Related Work: Achievability for Third-Order Term
Proposition (Polyanskiy (2010))
Assume that all elements of {W (y |x) : x ∈ X , y ∈ Y} are positiveand C > 0. Then,
ρn ≥1
2log n + O(1)
• This is an achievability result but BEC doesn’t satisfy assumptions
• Assumption may be relaxed to
∃P ∈ Π s.t. V r(P,W ) := V
(PW ,
P ×W
PW
)> 0
• Based on the concentration bound [Polyanskiy’s thesis]
E
[exp
(n∑
i=1
Xi
)I
{n∑
i=1
Xi ≥ γ
}]≤ 2
(log 2√
2π+
12T
σ
)exp(−γ)
σ√n
.
12 / 42
![Page 28: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/28.jpg)
Related Work: Achievability for Third-Order Term
Proposition (Polyanskiy (2010))
Assume that all elements of {W (y |x) : x ∈ X , y ∈ Y} are positiveand C > 0. Then,
ρn ≥1
2log n + O(1)
• This is an achievability result but BEC doesn’t satisfy assumptions
• Assumption may be relaxed to
∃P ∈ Π s.t. V r(P,W ) := V
(PW ,
P ×W
PW
)> 0
• Based on the concentration bound [Polyanskiy’s thesis]
E
[exp
(n∑
i=1
Xi
)I
{n∑
i=1
Xi ≥ γ
}]≤ 2
(log 2√
2π+
12T
σ
)exp(−γ)
σ√n
.
12 / 42
![Page 29: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/29.jpg)
Related Work: Achievability for Third-Order Term
Proposition (Polyanskiy (2010))
Assume that all elements of {W (y |x) : x ∈ X , y ∈ Y} are positiveand C > 0. Then,
ρn ≥1
2log n + O(1)
• This is an achievability result but BEC doesn’t satisfy assumptions
• Assumption may be relaxed to
∃P ∈ Π s.t. V r(P,W ) := V
(PW ,
P ×W
PW
)> 0
• Based on the concentration bound [Polyanskiy’s thesis]
E
[exp
(n∑
i=1
Xi
)I
{n∑
i=1
Xi ≥ γ
}]≤ 2
(log 2√
2π+
12T
σ
)exp(−γ)
σ√n
.
12 / 42
![Page 30: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/30.jpg)
Related Work: Converse for Third-Order Term
Proposition (Polyanskiy (2010))
If W is weakly input-symmetric
ρn ≤1
2log n + O(1)
• This is a converse result
• Gallager-symmetric channels are weakly input-symmetric
• The set of weakly input-symmetric channels is very thin
• We dispense of this symmetry assumption
13 / 42
![Page 31: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/31.jpg)
Related Work: Converse for Third-Order Term
Proposition (Polyanskiy (2010))
If W is weakly input-symmetric
ρn ≤1
2log n + O(1)
• This is a converse result
• Gallager-symmetric channels are weakly input-symmetric
• The set of weakly input-symmetric channels is very thin
• We dispense of this symmetry assumption
13 / 42
![Page 32: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/32.jpg)
Related Work: Converse for Third-Order Term
Proposition (Polyanskiy (2010))
If W is weakly input-symmetric
ρn ≤1
2log n + O(1)
• This is a converse result
• Gallager-symmetric channels are weakly input-symmetric
• The set of weakly input-symmetric channels is very thin
• We dispense of this symmetry assumption
13 / 42
![Page 33: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/33.jpg)
Related Work: Converse for Third-Order Term
Proposition (Polyanskiy (2010))
If W is weakly input-symmetric
ρn ≤1
2log n + O(1)
• This is a converse result
• Gallager-symmetric channels are weakly input-symmetric
• The set of weakly input-symmetric channels is very thin
• We dispense of this symmetry assumption
13 / 42
![Page 34: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/34.jpg)
Related Work: Converse for Third-Order Term
Proposition (Polyanskiy (2010))
If W is weakly input-symmetric
ρn ≤1
2log n + O(1)
• This is a converse result
• Gallager-symmetric channels are weakly input-symmetric
• The set of weakly input-symmetric channels is very thin
• We dispense of this symmetry assumption
13 / 42
![Page 35: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/35.jpg)
Main Result: Tight Third-Order Term
Theorem (Tomamichel-Tan (2013))
If W is a DMC with positive ε-dispersion,
ρn ≤1
2log n + O(1)
• The 12 cannot be improved
• For BSC
ρn =1
2log n + O(1)
• We can dispense of the positive ε-dispersion assumption
• No need for unique CAID
• “A Tight Upper Bound for the Third-Order Asymptotics forMost DMCs” M. Tomamichel and V. Y. F. Tan, IEEE T-IT,Nov 2013
14 / 42
![Page 36: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/36.jpg)
Main Result: Tight Third-Order Term
Theorem (Tomamichel-Tan (2013))
If W is a DMC with positive ε-dispersion,
ρn ≤1
2log n + O(1)
• The 12 cannot be improved
• For BSC
ρn =1
2log n + O(1)
• We can dispense of the positive ε-dispersion assumption
• No need for unique CAID
• “A Tight Upper Bound for the Third-Order Asymptotics forMost DMCs” M. Tomamichel and V. Y. F. Tan, IEEE T-IT,Nov 2013
14 / 42
![Page 37: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/37.jpg)
Main Result: Tight Third-Order Term
Theorem (Tomamichel-Tan (2013))
If W is a DMC with positive ε-dispersion,
ρn ≤1
2log n + O(1)
• The 12 cannot be improved
• For BSC
ρn =1
2log n + O(1)
• We can dispense of the positive ε-dispersion assumption
• No need for unique CAID
• “A Tight Upper Bound for the Third-Order Asymptotics forMost DMCs” M. Tomamichel and V. Y. F. Tan, IEEE T-IT,Nov 2013
14 / 42
![Page 38: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/38.jpg)
Main Result: Tight Third-Order Term
Theorem (Tomamichel-Tan (2013))
If W is a DMC with positive ε-dispersion,
ρn ≤1
2log n + O(1)
• The 12 cannot be improved
• For BSC
ρn =1
2log n + O(1)
• We can dispense of the positive ε-dispersion assumption
• No need for unique CAID
• “A Tight Upper Bound for the Third-Order Asymptotics forMost DMCs” M. Tomamichel and V. Y. F. Tan, IEEE T-IT,Nov 2013
14 / 42
![Page 39: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/39.jpg)
Main Result: Tight Third-Order Term
Theorem (Tomamichel-Tan (2013))
If W is a DMC with positive ε-dispersion,
ρn ≤1
2log n + O(1)
• The 12 cannot be improved
• For BSC
ρn =1
2log n + O(1)
• We can dispense of the positive ε-dispersion assumption
• No need for unique CAID
• “A Tight Upper Bound for the Third-Order Asymptotics forMost DMCs” M. Tomamichel and V. Y. F. Tan, IEEE T-IT,Nov 2013
14 / 42
![Page 40: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/40.jpg)
Main Result: Tight Third-Order Term
Theorem (Tomamichel-Tan (2013))
If W is a DMC with positive ε-dispersion,
ρn ≤1
2log n + O(1)
• The 12 cannot be improved
• For BSC
ρn =1
2log n + O(1)
• We can dispense of the positive ε-dispersion assumption
• No need for unique CAID
• “A Tight Upper Bound for the Third-Order Asymptotics forMost DMCs” M. Tomamichel and V. Y. F. Tan, IEEE T-IT,Nov 2013
14 / 42
![Page 41: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/41.jpg)
Main Result: Tight Third-Order TermAll cases are covered
Yes
No
Vε > 0
≤nC+√nVεΦ−1(ε)+ 1
2log n+O(1)
Yes
No
not exoticor ε< 1
2
≤nC+O(1)
Yes
No
exoticand ε= 1
2
≤nC+ 12
log n+O(1)
≤nC+O(n
13)
[PPV10]
W is exotic if Vmax(W ) = 0 and ∃ x0 ∈ X such that
D(W (·|x0)‖Q∗) = C , and V (W (·|x0)‖Q∗) > 0.
15 / 42
![Page 42: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/42.jpg)
Main Result: Tight Third-Order TermAll cases are covered
Yes
No
Vε > 0
≤nC+√nVεΦ−1(ε)+ 1
2log n+O(1)
Yes
No
not exoticor ε< 1
2
≤nC+O(1)
Yes
No
exoticand ε= 1
2
≤nC+ 12
log n+O(1)
≤nC+O(n
13)
[PPV10]
W is exotic if Vmax(W ) = 0 and ∃ x0 ∈ X such that
D(W (·|x0)‖Q∗) = C , and V (W (·|x0)‖Q∗) > 0.
15 / 42
![Page 43: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/43.jpg)
Main Result: Tight Third-Order TermAll cases are covered
Yes
No
Vε > 0
≤nC+√nVεΦ−1(ε)+ 1
2log n+O(1)
Yes
No
not exoticor ε< 1
2
≤nC+O(1)
Yes
No
exoticand ε= 1
2
≤nC+ 12
log n+O(1)
≤nC+O(n
13)
[PPV10]
W is exotic if Vmax(W ) = 0 and ∃ x0 ∈ X such that
D(W (·|x0)‖Q∗) = C , and V (W (·|x0)‖Q∗) > 0.
15 / 42
![Page 44: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/44.jpg)
Main Result: Tight Third-Order TermAll cases are covered
Yes
No
Vε > 0
≤nC+√nVεΦ−1(ε)+ 1
2log n+O(1)
Yes
No
not exoticor ε< 1
2
≤nC+O(1)
Yes
No
exoticand ε= 1
2
≤nC+ 12
log n+O(1)
≤nC+O(n
13)
[PPV10]
W is exotic if Vmax(W ) = 0 and ∃ x0 ∈ X such that
D(W (·|x0)‖Q∗) = C , and V (W (·|x0)‖Q∗) > 0.
15 / 42
![Page 45: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/45.jpg)
Main Result: Tight Third-Order TermAll cases are covered
Yes
No
Vε > 0
≤nC+√nVεΦ−1(ε)+ 1
2log n+O(1)
Yes
No
not exoticor ε< 1
2
≤nC+O(1)
Yes
No
exoticand ε= 1
2
≤nC+ 12
log n+O(1)
≤nC+O(n
13)
[PPV10]
W is exotic if Vmax(W ) = 0 and ∃ x0 ∈ X such that
D(W (·|x0)‖Q∗) = C , and V (W (·|x0)‖Q∗) > 0.
15 / 42
![Page 46: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/46.jpg)
Proof Technique for Tight Third-Order Term
• For the regular case, ρn ≤ 12 log n + O(1)
• The type-counting trick and upper bounds on M∗P(W n, ε) arenot sufficiently tight
• We need a convenient converse bound for general DMCs
• Information spectrum divergence
Dεs (P‖Q) := sup
{R : P
(log
P(X )
Q(X )≤ R
)≤ ε}
“Information Spectrum Methods in Information Theory”by T. S. Han (2003)
16 / 42
![Page 47: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/47.jpg)
Proof Technique for Tight Third-Order Term
• For the regular case, ρn ≤ 12 log n + O(1)
• The type-counting trick and upper bounds on M∗P(W n, ε) arenot sufficiently tight
• We need a convenient converse bound for general DMCs
• Information spectrum divergence
Dεs (P‖Q) := sup
{R : P
(log
P(X )
Q(X )≤ R
)≤ ε}
“Information Spectrum Methods in Information Theory”by T. S. Han (2003)
16 / 42
![Page 48: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/48.jpg)
Proof Technique for Tight Third-Order Term
• For the regular case, ρn ≤ 12 log n + O(1)
• The type-counting trick and upper bounds on M∗P(W n, ε) arenot sufficiently tight
• We need a convenient converse bound for general DMCs
• Information spectrum divergence
Dεs (P‖Q) := sup
{R : P
(log
P(X )
Q(X )≤ R
)≤ ε}
“Information Spectrum Methods in Information Theory”by T. S. Han (2003)
16 / 42
![Page 49: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/49.jpg)
Proof Technique for Tight Third-Order Term
• For the regular case, ρn ≤ 12 log n + O(1)
• The type-counting trick and upper bounds on M∗P(W n, ε) arenot sufficiently tight
• We need a convenient converse bound for general DMCs
• Information spectrum divergence
Dεs (P‖Q) := sup
{R : P
(log
P(X )
Q(X )≤ R
)≤ ε}
“Information Spectrum Methods in Information Theory”by T. S. Han (2003)
16 / 42
![Page 50: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/50.jpg)
Proof Technique: Information Spectrum Divergence
Dεs (P‖Q) := sup
{R∣∣P (log
P(X )
Q(X )≤ R
)≤ ε}
“Density” of log P(X )Q(X )
R∗
ε 1− ε
If X n is i.i.d. P, the Berry-Esseen theorem yields
Dεs (Pn‖Qn) = nD(P‖Q) +
√nV (P‖Q)Φ−1(ε) + O(1)
17 / 42
![Page 51: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/51.jpg)
Proof Technique: Information Spectrum Divergence
Dεs (P‖Q) := sup
{R∣∣P (log
P(X )
Q(X )≤ R
)≤ ε}
“Density” of log P(X )Q(X )
R∗
ε
1− ε
If X n is i.i.d. P, the Berry-Esseen theorem yields
Dεs (Pn‖Qn) = nD(P‖Q) +
√nV (P‖Q)Φ−1(ε) + O(1)
17 / 42
![Page 52: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/52.jpg)
Proof Technique: Information Spectrum Divergence
Dεs (P‖Q) := sup
{R∣∣P (log
P(X )
Q(X )≤ R
)≤ ε}
“Density” of log P(X )Q(X )
R∗
ε 1− ε
If X n is i.i.d. P, the Berry-Esseen theorem yields
Dεs (Pn‖Qn) = nD(P‖Q) +
√nV (P‖Q)Φ−1(ε) + O(1)
17 / 42
![Page 53: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/53.jpg)
Proof Technique: Information Spectrum Divergence
Dεs (P‖Q) := sup
{R∣∣P (log
P(X )
Q(X )≤ R
)≤ ε}
“Density” of log P(X )Q(X )
R∗
ε 1− ε
If X n is i.i.d. P, the Berry-Esseen theorem yields
Dεs (Pn‖Qn) = nD(P‖Q) +
√nV (P‖Q)Φ−1(ε) + O(1)
17 / 42
![Page 54: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/54.jpg)
Proof Technique: Symbol-Wise Converse Bound
Lemma (Tomamichel-Tan (2013))
For every channel W , every ε ∈ (0, 1) and δ ∈ (0, 1− ε), we have
logM∗(W , ε) ≤ minQ∈P(Y)
maxx∈X
Dε+δs (W (·|x)‖Q) + log
1
δ
• When DMC is used n times,
logM∗(W n, ε) ≤ minQ(n)∈P(Yn)
(maxx∈X n
Dε+δs (W n(·|x)‖Q(n))
)+log
1
δ
• Choose δ = n−12 so log 1
δ = 12 log n
• Since all x within a type class result in the same Dε+δs (if Q(n)
is permutation invariant), it’s really a max over typesPx ∈ Pn(X )
18 / 42
![Page 55: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/55.jpg)
Proof Technique: Symbol-Wise Converse Bound
Lemma (Tomamichel-Tan (2013))
For every channel W , every ε ∈ (0, 1) and δ ∈ (0, 1− ε), we have
logM∗(W , ε) ≤ minQ∈P(Y)
maxx∈X
Dε+δs (W (·|x)‖Q) + log
1
δ
• When DMC is used n times,
logM∗(W n, ε) ≤ minQ(n)∈P(Yn)
(maxx∈X n
Dε+δs (W n(·|x)‖Q(n))
)+log
1
δ
• Choose δ = n−12 so log 1
δ = 12 log n
• Since all x within a type class result in the same Dε+δs (if Q(n)
is permutation invariant), it’s really a max over typesPx ∈ Pn(X )
18 / 42
![Page 56: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/56.jpg)
Proof Technique: Symbol-Wise Converse Bound
Lemma (Tomamichel-Tan (2013))
For every channel W , every ε ∈ (0, 1) and δ ∈ (0, 1− ε), we have
logM∗(W , ε) ≤ minQ∈P(Y)
maxx∈X
Dε+δs (W (·|x)‖Q) + log
1
δ
• When DMC is used n times,
logM∗(W n, ε) ≤ minQ(n)∈P(Yn)
(maxx∈X n
Dε+δs (W n(·|x)‖Q(n))
)+log
1
δ
• Choose δ = n−12 so log 1
δ = 12 log n
• Since all x within a type class result in the same Dε+δs (if Q(n)
is permutation invariant), it’s really a max over typesPx ∈ Pn(X )
18 / 42
![Page 57: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/57.jpg)
Proof Technique: Symbol-Wise Converse Bound
Lemma (Tomamichel-Tan (2013))
For every channel W , every ε ∈ (0, 1) and δ ∈ (0, 1− ε), we have
logM∗(W , ε) ≤ minQ∈P(Y)
maxx∈X
Dε+δs (W (·|x)‖Q) + log
1
δ
• When DMC is used n times,
logM∗(W n, ε) ≤ minQ(n)∈P(Yn)
(maxx∈X n
Dε+δs (W n(·|x)‖Q(n))
)+log
1
δ
• Choose δ = n−12 so log 1
δ = 12 log n
• Since all x within a type class result in the same Dε+δs (if Q(n)
is permutation invariant), it’s really a max over typesPx ∈ Pn(X )
18 / 42
![Page 58: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/58.jpg)
Proof Technique: Choice of Output Distribution
logM∗(W n, ε) ≤ maxx∈X n
Dε+δs (W n(·|x)‖Q(n))+log
1
δ, ∀Q(n) ∈ P(Yn)
• Q(n)(y): invariant to permutations of the n channel uses
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
• First term: Qk’s and λ(k)’s designed to form an n−12 -cover of
P(Y):
∀Q ∈ P(Y), ∃ k ∈ K s.t. ‖Q − Qk‖2 ≤ n−12 .
• Second term: Uniform mixture over output distributionsinduced by input types [Hayashi (2009)]
19 / 42
![Page 59: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/59.jpg)
Proof Technique: Choice of Output Distribution
logM∗(W n, ε) ≤ maxx∈X n
Dε+δs (W n(·|x)‖Q(n))+log
1
δ, ∀Q(n) ∈ P(Yn)
• Q(n)(y): invariant to permutations of the n channel uses
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
• First term: Qk’s and λ(k)’s designed to form an n−12 -cover of
P(Y):
∀Q ∈ P(Y), ∃ k ∈ K s.t. ‖Q − Qk‖2 ≤ n−12 .
• Second term: Uniform mixture over output distributionsinduced by input types [Hayashi (2009)]
19 / 42
![Page 60: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/60.jpg)
Proof Technique: Choice of Output Distribution
logM∗(W n, ε) ≤ maxx∈X n
Dε+δs (W n(·|x)‖Q(n))+log
1
δ, ∀Q(n) ∈ P(Yn)
• Q(n)(y): invariant to permutations of the n channel uses
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
• First term: Qk’s and λ(k)’s designed to form an n−12 -cover of
P(Y):
∀Q ∈ P(Y), ∃ k ∈ K s.t. ‖Q − Qk‖2 ≤ n−12 .
• Second term: Uniform mixture over output distributionsinduced by input types [Hayashi (2009)]
19 / 42
![Page 61: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/61.jpg)
Proof Technique: Choice of Output Distribution
logM∗(W n, ε) ≤ maxx∈X n
Dε+δs (W n(·|x)‖Q(n))+log
1
δ, ∀Q(n) ∈ P(Yn)
• Q(n)(y): invariant to permutations of the n channel uses
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
• First term: Qk’s and λ(k)’s designed to form an n−12 -cover of
P(Y):
∀Q ∈ P(Y), ∃ k ∈ K s.t. ‖Q − Qk‖2 ≤ n−12 .
• Second term: Uniform mixture over output distributionsinduced by input types [Hayashi (2009)]
19 / 42
![Page 62: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/62.jpg)
Proof Technique: Novel Choice of Output Distribution• First term is∑
k∈Kλ(k)Qn
k (y) where λ(k) =exp
(− γ‖k‖2
2
)F
and k indexes distance to the capacity-achieving outputdistribution (CAOD). Can be shown that F <∞.
• Choose each Qk as follows:
Qk(y) := Q∗(y) +ky√nζ,
where K :={
k ∈ Z|Y| :∑
y ky = 0, ky ≥ −Q∗(y)√nζ}
• By construction, ensures that
∀Q ∈ P(Y), ∃ k ∈ K, s.t. ‖Q − Qk‖2 ≤1√n.
20 / 42
![Page 63: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/63.jpg)
Proof Technique: Novel Choice of Output Distribution• First term is∑
k∈Kλ(k)Qn
k (y) where λ(k) =exp
(− γ‖k‖2
2
)F
and k indexes distance to the capacity-achieving outputdistribution (CAOD). Can be shown that F <∞.
• Choose each Qk as follows:
Qk(y) := Q∗(y) +ky√nζ,
where K :={
k ∈ Z|Y| :∑
y ky = 0, ky ≥ −Q∗(y)√nζ}
• By construction, ensures that
∀Q ∈ P(Y), ∃ k ∈ K, s.t. ‖Q − Qk‖2 ≤1√n.
20 / 42
![Page 64: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/64.jpg)
Proof Technique: Novel Choice of Output Distribution• First term is∑
k∈Kλ(k)Qn
k (y) where λ(k) =exp
(− γ‖k‖2
2
)F
and k indexes distance to the capacity-achieving outputdistribution (CAOD). Can be shown that F <∞.
• Choose each Qk as follows:
Qk(y) := Q∗(y) +ky√nζ,
where K :={
k ∈ Z|Y| :∑
y ky = 0, ky ≥ −Q∗(y)√nζ}
• By construction, ensures that
∀Q ∈ P(Y), ∃ k ∈ K, s.t. ‖Q − Qk‖2 ≤1√n.
20 / 42
![Page 65: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/65.jpg)
Proof Technique: Novel Choice of Output Distribution
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
Q(0)
Q(1)
(0, 1)
(1, 0) P(Y)
Q∗
Q[−1,1]
Q[1,−1]
Q[2,−2]
Q[−2,2]
1√2n
1√2n
21 / 42
![Page 66: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/66.jpg)
Proof Technique: Novel Choice of Output Distribution
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
Q(0)
Q(1)
(0, 1)
(1, 0) P(Y)
Q∗
Q[−1,1]
Q[1,−1]
Q[2,−2]
Q[−2,2]
1√2n
1√2n
21 / 42
![Page 67: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/67.jpg)
Proof Technique: Novel Choice of Output Distribution
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
Q(0)
Q(1)
(0, 1)
(1, 0) P(Y)
Q∗
Q[−1,1]
Q[1,−1]
Q[2,−2]
Q[−2,2]
1√2n
1√2n
21 / 42
![Page 68: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/68.jpg)
Proof Technique: Novel Choice of Output Distribution
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
Q(0)
Q(1)
(0, 1)
(1, 0) P(Y)
Q∗
Q[−1,1]
Q[1,−1]
Q[2,−2]
Q[−2,2]
1√2n
1√2n
21 / 42
![Page 69: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/69.jpg)
Proof Technique: Novel Choice of Output Distribution
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
Q(0)
Q(1)
(0, 1)
(1, 0) P(Y)
Q∗
Q[−1,1]
Q[1,−1]
Q[2,−2]
Q[−2,2]
1√2n
1√2n
21 / 42
![Page 70: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/70.jpg)
Proof Technique: Novel Choice of Output Distribution
{Qk : k ∈ K} ⊂ P(Y)
Q∗
λ(k′) ∝ exp(−γ‖k′‖22)
Qk′
∀Q ∈ P(Y), ∃ k ∈ K, s.t. ‖Q − Qk‖2 ≤1√n.
22 / 42
![Page 71: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/71.jpg)
Proof Technique: Novel Choice of Output Distribution
{Qk : k ∈ K} ⊂ P(Y)
Q∗
λ(k′) ∝ exp(−γ‖k′‖22)
Qk′
∀Q ∈ P(Y), ∃ k ∈ K, s.t. ‖Q − Qk‖2 ≤1√n.
22 / 42
![Page 72: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/72.jpg)
Proof Technique: Novel Choice of Output Distribution
{Qk : k ∈ K} ⊂ P(Y)
Q∗
λ(k′) ∝ exp(−γ‖k′‖22)
Qk′
∀Q ∈ P(Y), ∃ k ∈ K, s.t. ‖Q − Qk‖2 ≤1√n.
22 / 42
![Page 73: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/73.jpg)
Proof Technique: Novel Choice of Output Distribution
{Qk : k ∈ K} ⊂ P(Y)
Q∗
λ(k′) ∝ exp(−γ‖k′‖22)
Qk′
∀Q ∈ P(Y), ∃ k ∈ K, s.t. ‖Q − Qk‖2 ≤1√n.
22 / 42
![Page 74: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/74.jpg)
Proof Technique: Standard Choice of Output Distn.
• Recall the output distribution
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
• Second term: Uniform mixture over output distributionsinduced by input types [Hayashi (2009)]∑
P∈Pn(X )
1
|Pn(X )|(PW )n(y).
• Serves to take care of “bad input types” (i.e., typesP ∈ Pn(X ) such that PW is far from Q∗)
23 / 42
![Page 75: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/75.jpg)
Proof Technique: Standard Choice of Output Distn.
• Recall the output distribution
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
• Second term: Uniform mixture over output distributionsinduced by input types [Hayashi (2009)]∑
P∈Pn(X )
1
|Pn(X )|(PW )n(y).
• Serves to take care of “bad input types” (i.e., typesP ∈ Pn(X ) such that PW is far from Q∗)
23 / 42
![Page 76: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/76.jpg)
Proof Technique: Standard Choice of Output Distn.
• Recall the output distribution
Q(n)(y) :=1
2
∑k∈K
λ(k)Qnk (y) +
1
2
∑P∈Pn(X )
1
|Pn(X )|(PW )n(y)
• Second term: Uniform mixture over output distributionsinduced by input types [Hayashi (2009)]∑
P∈Pn(X )
1
|Pn(X )|(PW )n(y).
• Serves to take care of “bad input types” (i.e., typesP ∈ Pn(X ) such that PW is far from Q∗)
23 / 42
![Page 77: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/77.jpg)
Outline
1 Introduction
2 Old Contribution
3 New Contribution
24 / 42
![Page 78: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/78.jpg)
Mathematical Model of Poisson Channel (1/3)Consider the following optical communication:
inp
ut
wa
vefo
rmλ
(t)
time t
(peak power A)
light source photon-detector
cou
nti
ng
pro
cessν
(t)
time t
(# of photons)
modulates
(optical signal)
outputs
Remark: This is a continuous-time channel (0 ≤ t < T ).
25 / 42
![Page 79: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/79.jpg)
Mathematical Model of Poisson Channel (1/3)Consider the following optical communication:
inp
ut
wa
vefo
rmλ
(t)
time t
(peak power A)
light source photon-detector
cou
nti
ng
pro
cessν
(t)
time t
(# of photons)
modulates
(optical signal)
outputs
Remark: This is a continuous-time channel (0 ≤ t < T ).
25 / 42
![Page 80: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/80.jpg)
Mathematical Model of Poisson Channel (2/3)λ(t) (peak power A)
t
light source
modulates
(optical signal)
Optical Signal is Modulated by Input Waveform λ(t)
• an integrable function λ(·) defined on the time block [0,T );
• with peak power constraint (A > 0):
0 ≤ λ(t) ≤ A ∀t ∈ [0,T );
• with average power constraint (0 ≤ σ ≤ 1):
1
T
∫ T
0λ(t) dt ≤ σA.
26 / 42
![Page 81: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/81.jpg)
Mathematical Model of Poisson Channel (2/3)λ(t) (peak power A)
t
light source
modulates
(optical signal)
Optical Signal is Modulated by Input Waveform λ(t)
• an integrable function λ(·) defined on the time block [0,T );
• with peak power constraint (A > 0):
0 ≤ λ(t) ≤ A ∀t ∈ [0,T );
• with average power constraint (0 ≤ σ ≤ 1):
1
T
∫ T
0λ(t) dt ≤ σA.
26 / 42
![Page 82: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/82.jpg)
Mathematical Model of Poisson Channel (3/3)
(light)
photon-detector
outputsν(t)
(# of photons)
t
Output is Poisson counting process {ν(t)}0≤t<T
ν(0) = 0 a.s. and P{ν(t + τ)− ν(t) = k} =eΛ Λk
k!
for each t, τ ∈ R≥0 and k ∈ Z≥0, where Λ is given by
Λdef=
∫ t+τ
t
(λ(u) + λ0
)du.
• input waveform (intensity of light) λ : [0,T )→ [0,A]
• dark current (background noise level) 0 ≤ λ0 <∞
27 / 42
![Page 83: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/83.jpg)
Mathematical Model of Poisson Channel (3/3)
(light)
photon-detector
outputsν(t)
(# of photons)
t
Output is Poisson counting process {ν(t)}0≤t<T
ν(0) = 0 a.s. and P{ν(t + τ)− ν(t) = k} =eΛ Λk
k!
for each t, τ ∈ R≥0 and k ∈ Z≥0, where Λ is given by
Λdef=
∫ t+τ
t
(λ(u) + λ0
)du.
• input waveform (intensity of light) λ : [0,T )→ [0,A]
• dark current (background noise level) 0 ≤ λ0 <∞27 / 42
![Page 84: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/84.jpg)
Block Coding Scheme for Poisson Channel
m φ Poisson channel ψ mλ(t) ν(t)
• input alphabet is the set of waveforms λ(·)
W(T ,A, σ)def=
{λ : [0,T )→ [0,A]
∣∣∣∣ 1
T
∫ T
0λ(t) dt ≤ σA
},
where A (resp. σ) is the peak (resp. average) power constraint.
• output alphabet is the set of possible counting processes ν(·)
S(T )def= {g : [0,T )→ Z≥0 | g(0) = 0 and g(t1) ≥ g(t2), t1 < t2}
A (T ,M ,A, σ)-code (φ, ψ) for Poisson channel
• encoder φ : {1, 2, . . . ,M} → W(T ,A, σ)
• decoder ψ : S(T )→ {1, 2, . . . ,M}
28 / 42
![Page 85: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/85.jpg)
Block Coding Scheme for Poisson Channel
m φ Poisson channel ψ mλ(t) ν(t)
• input alphabet is the set of waveforms λ(·)
W(T ,A, σ)def=
{λ : [0,T )→ [0,A]
∣∣∣∣ 1
T
∫ T
0λ(t) dt ≤ σA
},
where A (resp. σ) is the peak (resp. average) power constraint.
• output alphabet is the set of possible counting processes ν(·)
S(T )def= {g : [0,T )→ Z≥0 | g(0) = 0 and g(t1) ≥ g(t2), t1 < t2}
A (T ,M ,A, σ)-code (φ, ψ) for Poisson channel
• encoder φ : {1, 2, . . . ,M} → W(T ,A, σ)
• decoder ψ : S(T )→ {1, 2, . . . ,M}
28 / 42
![Page 86: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/86.jpg)
Block Coding Scheme for Poisson Channel
m φ Poisson channel ψ mλ(t) ν(t)
• input alphabet is the set of waveforms λ(·)
W(T ,A, σ)def=
{λ : [0,T )→ [0,A]
∣∣∣∣ 1
T
∫ T
0λ(t) dt ≤ σA
},
where A (resp. σ) is the peak (resp. average) power constraint.
• output alphabet is the set of possible counting processes ν(·)
S(T )def= {g : [0,T )→ Z≥0 | g(0) = 0 and g(t1) ≥ g(t2), t1 < t2}
A (T ,M ,A, σ)-code (φ, ψ) for Poisson channel
• encoder φ : {1, 2, . . . ,M} → W(T ,A, σ)
• decoder ψ : S(T )→ {1, 2, . . . ,M}
28 / 42
![Page 87: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/87.jpg)
Block Coding Scheme for Poisson Channel (Cont’d)
m φ Poisson channel ψ mλ(t) ν(t)
A (T ,M ,A, σ)-code (φ, ψ) for Poisson channel
• encoder φ : {1, 2, . . . ,M} → W(T ,A, σ)
• decoder ψ : S(T )→ {1, 2, . . . ,M}
A (T ,M ,A, σ, ε)avg-code (φ, ψ) for Poisson channel
A (T ,M,A, σ)-code (φ, ψ) is called a (T ,M,A, σ, ε)avg-code if
1
M
M∑m=1
P{ψ(ν) = m | λ = φ(m)} ≥ 1− ε.
Here, λ is the r.v. induced by the encoder φ with uniform messages.
29 / 42
![Page 88: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/88.jpg)
Block Coding Scheme for Poisson Channel (Cont’d)
m φ Poisson channel ψ mλ(t) ν(t)
A (T ,M ,A, σ)-code (φ, ψ) for Poisson channel
• encoder φ : {1, 2, . . . ,M} → W(T ,A, σ)
• decoder ψ : S(T )→ {1, 2, . . . ,M}
A (T ,M ,A, σ, ε)avg-code (φ, ψ) for Poisson channel
A (T ,M,A, σ)-code (φ, ψ) is called a (T ,M,A, σ, ε)avg-code if
1
M
M∑m=1
P{ψ(ν) = m | λ = φ(m)} ≥ 1− ε.
Here, λ is the r.v. induced by the encoder φ with uniform messages.
29 / 42
![Page 89: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/89.jpg)
Block Coding Scheme for Poisson Channel (Cont’d)
m φ Poisson channel ψ mλ(t) ν(t)
A (T ,M ,A, σ)-code (φ, ψ) for Poisson channel
• encoder φ : {1, 2, . . . ,M} → W(T ,A, σ)
• decoder ψ : S(T )→ {1, 2, . . . ,M}
A (T ,M ,A, σ, ε)avg-code (φ, ψ) for Poisson channel
A (T ,M,A, σ)-code (φ, ψ) is called a (T ,M,A, σ, ε)avg-code if
1
M
M∑m=1
P{ψ(ν) = m | λ = φ(m)} ≥ 1− ε.
Here, λ is the r.v. induced by the encoder φ with uniform messages.
29 / 42
![Page 90: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/90.jpg)
Poisson Channel Capacity (1st-Order Asymptotics)
Denote by M∗ the max. M s.t. ∃ a (T ,M,A, σ, ε)avg-code.
Theorem (Kabanov’78; Davis’80; Wyner’88)
logM∗ = T C ∗ + o(T ) (as T →∞),
where
C ∗def= A
((1− p∗) s log
s
p∗ + s+ p∗ (1 + s) log
1 + s
p∗ + s
),
sdef=λ0
A(ratio of dark current λ0 to PPC A),
p∗def= min{σ, p0} (role of CAID, where σ is APC),
p0def=
(1 + s)1+s
ss e− s.
30 / 42
![Page 91: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/91.jpg)
Poisson Channel Dispersion (2nd-Order Asymptotics)• Denote by M∗ the max s.t. ∃ a (T ,M,A, σ, ε)avg-code.
• We seek second- and third-order terms
logM∗ = T C ∗ +√T L + ρT , T →∞.
• Many works since 2013 on multi-terminal channels and sources
• First work on higher-order asymptotics for continuous-timechannels
Yuta Sakai Mladen Kovacevic
31 / 42
![Page 92: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/92.jpg)
Poisson Channel Dispersion (2nd-Order Asymptotics)• Denote by M∗ the max s.t. ∃ a (T ,M,A, σ, ε)avg-code.
• We seek second- and third-order terms
logM∗ = T C ∗ +√T L + ρT , T →∞.
• Many works since 2013 on multi-terminal channels and sources
• First work on higher-order asymptotics for continuous-timechannels
Yuta Sakai Mladen Kovacevic
31 / 42
![Page 93: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/93.jpg)
Poisson Channel Dispersion (2nd-Order Asymptotics)• Denote by M∗ the max s.t. ∃ a (T ,M,A, σ, ε)avg-code.
• We seek second- and third-order terms
logM∗ = T C ∗ +√T L + ρT , T →∞.
• Many works since 2013 on multi-terminal channels and sources
• First work on higher-order asymptotics for continuous-timechannels
Yuta Sakai Mladen Kovacevic
31 / 42
![Page 94: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/94.jpg)
Poisson Channel Dispersion (2nd-Order Asymptotics)• Denote by M∗ the max s.t. ∃ a (T ,M,A, σ, ε)avg-code.
• We seek second- and third-order terms
logM∗ = T C ∗ +√T L + ρT , T →∞.
• Many works since 2013 on multi-terminal channels and sources
• First work on higher-order asymptotics for continuous-timechannels
Yuta Sakai Mladen Kovacevic
31 / 42
![Page 95: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/95.jpg)
Poisson Channel Dispersion (2nd-Order Asymptotics)
Theorem (Sakai–Tan–Kovacevic’19: arXiv:1903.10438)
logM∗ = T C ∗ +√T V ∗Φ−1(ε) + ρT ,
where the Poisson channel dispersion V ∗ is given by
V ∗def= A
((1− p∗) s log2 s
p∗ + s+ p∗ (1 + s) log2 1 + s
p∗ + s
),
and the third-order term ρT satisfies
1
2logT + O(1) ≤ ρT ≤ logT + O(1) (as T →∞).
Result: 2nd-order term√V ∗Φ−1(ε) and bounds on 3rd-order term ρT
32 / 42
![Page 96: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/96.jpg)
Proof Ideas of Second- and Third-Order Asymptotics
In both converse and achievability parts, we shall employWyner’s discretization argument (Wyner’88):
x
{0, 1} n3
convertor Poisson ch. quantizer y ∈{0, 1}n
λ(t) ν(t)
Converse Part
• symbol-wise meta converse bound (Tomamichel–Tan’13)
• novel choice of output distribution (projected ε-net)
Achievability Part
• random coding union bound (PPV’10) with cost constraint
• some other techniques to handle the continuous nature(e.g., logarithmic Sobolev inequality)
33 / 42
![Page 97: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/97.jpg)
Proof Ideas of Second- and Third-Order Asymptotics
In both converse and achievability parts, we shall employWyner’s discretization argument (Wyner’88):
x
{0, 1} n3
convertor Poisson ch. quantizer y ∈{0, 1}n
λ(t) ν(t)
Converse Part
• symbol-wise meta converse bound (Tomamichel–Tan’13)
• novel choice of output distribution (projected ε-net)
Achievability Part
• random coding union bound (PPV’10) with cost constraint
• some other techniques to handle the continuous nature(e.g., logarithmic Sobolev inequality)
33 / 42
![Page 98: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/98.jpg)
Proof Ideas of Second- and Third-Order Asymptotics
In both converse and achievability parts, we shall employWyner’s discretization argument (Wyner’88):
x
{0, 1} n3
convertor Poisson ch. quantizer y ∈{0, 1}n
λ(t) ν(t)
Converse Part
• symbol-wise meta converse bound (Tomamichel–Tan’13)
• novel choice of output distribution (projected ε-net)
Achievability Part
• random coding union bound (PPV’10) with cost constraint
• some other techniques to handle the continuous nature(e.g., logarithmic Sobolev inequality)
33 / 42
![Page 99: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/99.jpg)
Wyner’s Discretization Part I: Input Restrictionλ(t) (peak power A)
t
light source
modulates
(optical signal)
Discretization of {λ(t)}0≤t<T into n Blocks (here, ∆ = T/n)
input waveform λ(t) is restricted to be square, e.g.,
0
λ(t) (peak power constraint A)
time t
∆3∆ 4∆ 5∆ 6∆ 7∆
That is, we may think of λ(t) as a binary sequence {xk}nk=1.
34 / 42
![Page 100: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/100.jpg)
Wyner’s Discretization Part I: Input Restrictionλ(t) (peak power A)
t
light source
modulates
(optical signal)
Discretization of {λ(t)}0≤t<T into n Blocks (here, ∆ = T/n)
input waveform λ(t) is restricted to be square, e.g.,
0
λ(t) (peak power constraint A)
time t
∆3∆ 4∆ 5∆ 6∆ 7∆
That is, we may think of λ(t) as a binary sequence {xk}nk=1. 34 / 42
![Page 101: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/101.jpg)
Wyner’s Discretization Part II: Output Quantization
(light)
photon-detector
outputsν(t)
(# of photons)
t
Discretization of {ν(t)}0≤t<T into n Blocks (here, ∆ = T/n)
∆t
(k − 1)∆ k∆
ν((k − 1)∆)
ν(k∆)the gap = 1?
Poisson counting process ν(t) is quantized as {yk}nk=1:
ykdef=
{0 if ν(k∆)− ν((k − 1)∆) 6= 1,
1 if ν(k∆)− ν((k − 1)∆) = 1.
35 / 42
![Page 102: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/102.jpg)
Wyner’s Discretization Part II: Output Quantization
(light)
photon-detector
outputsν(t)
(# of photons)
t
Discretization of {ν(t)}0≤t<T into n Blocks (here, ∆ = T/n)
∆t
(k − 1)∆ k∆
ν((k − 1)∆)
ν(k∆)the gap = 1?
Poisson counting process ν(t) is quantized as {yk}nk=1:
ykdef=
{0 if ν(k∆)− ν((k − 1)∆) 6= 1,
1 if ν(k∆)− ν((k − 1)∆) = 1.
35 / 42
![Page 103: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/103.jpg)
Wyner’s Discretization Part II: Output Quantization
(light)
photon-detector
outputsν(t)
(# of photons)
t
Discretization of {ν(t)}0≤t<T into n Blocks (here, ∆ = T/n)
∆t
(k − 1)∆ k∆
ν((k − 1)∆)
ν(k∆)the gap = 1?
Poisson counting process ν(t) is quantized as {yk}nk=1:
ykdef=
{0 if ν(k∆)− ν((k − 1)∆) 6= 1,
1 if ν(k∆)− ν((k − 1)∆) = 1.35 / 42
![Page 104: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/104.jpg)
Overall Diagram of Wyner’s Discretization
x convertor Poisson channel quantizer yλ(t) ν(t)
• input sequence x = (x1, . . . , xn) ∈ {0, 1}n(which is converted to a square wave λ(t): )
• output sequence y = (y1, . . . , yn) ∈ {0, 1}n(which is obtained by quantizing the counting process ν(t))
Discretized channel W nn : {0, 1}n → {0, 1}n
W nn (y | x)
def=
n∏i=1
Wn(yi | xi ),
where the single-letter channel Wn : {0, 1} → {0, 1} depends on n.
Remark: the discretization error is negligible as n→∞ (next slide).
36 / 42
![Page 105: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/105.jpg)
Overall Diagram of Wyner’s Discretization
x convertor Poisson channel quantizer yλ(t) ν(t)
• input sequence x = (x1, . . . , xn) ∈ {0, 1}n(which is converted to a square wave λ(t): )
• output sequence y = (y1, . . . , yn) ∈ {0, 1}n(which is obtained by quantizing the counting process ν(t))
Discretized channel W nn : {0, 1}n → {0, 1}n
W nn (y | x)
def=
n∏i=1
Wn(yi | xi ),
where the single-letter channel Wn : {0, 1} → {0, 1} depends on n.
Remark: the discretization error is negligible as n→∞ (next slide).
36 / 42
![Page 106: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/106.jpg)
Wyner’s Discretization Well-Approximates Poisson Channel
Denote by
• M∗Poisson(ε): fundamental limit of Poisson channel
• M∗(W nn , ε): fundamental limit of discretized channel W n
n
Lemma (Wyner’88)
There exist a sequence εn = o(1) and a subsequence {nk}∞k=1 s.t.
M∗Poisson(ε) = M∗(W nknk, ε+ εnk ) (∀k ≥ 1).
Therefore, we observe that
logM∗Poisson(ε) ≤ lim supn→∞
logM∗(W nn , ε+ εn),
implying that it suffices to examine the RHS in the converse part.
37 / 42
![Page 107: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/107.jpg)
Wyner’s Discretization Well-Approximates Poisson Channel
Denote by
• M∗Poisson(ε): fundamental limit of Poisson channel
• M∗(W nn , ε): fundamental limit of discretized channel W n
n
Lemma (Wyner’88)
There exist a sequence εn = o(1) and a subsequence {nk}∞k=1 s.t.
M∗Poisson(ε) = M∗(W nknk, ε+ εnk ) (∀k ≥ 1).
Therefore, we observe that
logM∗Poisson(ε) ≤ lim supn→∞
logM∗(W nn , ε+ εn),
implying that it suffices to examine the RHS in the converse part.
37 / 42
![Page 108: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/108.jpg)
Meta Converse Bound and Output DistributionApply the symbol-wise meta converse (Tomamichel–Tan’13):
logM∗(W nn , ε+ εn) ≤ max
x∈{0,1}nDε+εn+η
s ( W nn (· | x)︸ ︷︷ ︸
discretized Poisson channel
‖Q(n)) + log1
η
Since Q(n) ∈ P({0, 1}n) is arbitrary, we substitute
Q(n)(y) = 1
3
n∏i=1
P∗[−κ]Wn(yi ) + 1
3
n∏i=1
P∗[κ]Wn(yi )
+1
3F
∞∑m=−∞:
0≤p∗+m/T≤1
e−γm2/T
n∏i=1
P∗[m/T ]Wn(yi )
where κ = 12 min{σ, 1/e} > 0 and P∗[u](1) = p∗ + u.
• third term is the main part of our novel construction
• first and second terms are to apply Lipschitz properties
38 / 42
![Page 109: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/109.jpg)
Meta Converse Bound and Output DistributionApply the symbol-wise meta converse (Tomamichel–Tan’13):
logM∗(W nn , ε+ εn) ≤ max
x∈{0,1}nDε+εn+η
s ( W nn (· | x)︸ ︷︷ ︸
discretized Poisson channel
‖Q(n)) + log1
η
Since Q(n) ∈ P({0, 1}n) is arbitrary, we substitute
Q(n)(y) = 1
3
n∏i=1
P∗[−κ]Wn(yi ) + 1
3
n∏i=1
P∗[κ]Wn(yi )
+1
3F
∞∑m=−∞:
0≤p∗+m/T≤1
e−γm2/T
n∏i=1
P∗[m/T ]Wn(yi )
where κ = 12 min{σ, 1/e} > 0 and P∗[u](1) = p∗ + u.
• third term is the main part of our novel construction
• first and second terms are to apply Lipschitz properties
38 / 42
![Page 110: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/110.jpg)
Meta Converse Bound and Output DistributionApply the symbol-wise meta converse (Tomamichel–Tan’13):
logM∗(W nn , ε+ εn) ≤ max
x∈{0,1}nDε+εn+η
s ( W nn (· | x)︸ ︷︷ ︸
discretized Poisson channel
‖Q(n)) + log1
η
Since Q(n) ∈ P({0, 1}n) is arbitrary, we substitute
Q(n)(y) = 1
3
n∏i=1
P∗[−κ]Wn(yi ) + 1
3
n∏i=1
P∗[κ]Wn(yi )
+1
3F
∞∑m=−∞:
0≤p∗+m/T≤1
e−γm2/T
n∏i=1
P∗[m/T ]Wn(yi )
where κ = 12 min{σ, 1/e} > 0 and P∗[u](1) = p∗ + u.
• third term is the main part of our novel construction
• first and second terms are to apply Lipschitz properties
38 / 42
![Page 111: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/111.jpg)
ε-Net Argument: Tomamichel–Tan’s Original ChoiceConsider a binary-input binary-output channel W : {0, 1} → {0, 1}.
[input probab. simplex]
(1, 0)
(0, 1)
•CAID P∗
[output probab. simplex]
(1, 0)
(0, 1)
•CAOD P∗W
multiplied by W
put-upanε-net
aroundCAO
DP ∗W
Qk
Use a convex combination of the ε-net:∑
k µ(k)Qnk .
39 / 42
![Page 112: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/112.jpg)
ε-Net Argument: Tomamichel–Tan’s Original ChoiceConsider a binary-input binary-output channel W : {0, 1} → {0, 1}.
[input probab. simplex]
(1, 0)
(0, 1)
•CAID P∗
[output probab. simplex]
(1, 0)
(0, 1)
•CAOD P∗W
multiplied by W
put-upanε-net
aroundCAO
DP ∗W
Qk
Use a convex combination of the ε-net:∑
k µ(k)Qnk .
39 / 42
![Page 113: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/113.jpg)
ε-Net Argument: Tomamichel–Tan’s Original ChoiceConsider a binary-input binary-output channel W : {0, 1} → {0, 1}.
[input probab. simplex]
(1, 0)
(0, 1)
•CAID P∗
[output probab. simplex]
(1, 0)
(0, 1)
•CAOD P∗W
multiplied by W
put-upanε-net
aroundCAO
DP ∗W
Qk
Use a convex combination of the ε-net:∑
k µ(k)Qnk .
39 / 42
![Page 114: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/114.jpg)
ε-Net Argument: Tomamichel–Tan’s Original ChoiceConsider a binary-input binary-output channel W : {0, 1} → {0, 1}.
[input probab. simplex]
(1, 0)
(0, 1)
•CAID P∗
[output probab. simplex]
(1, 0)
(0, 1)
•CAOD P∗W
multiplied by W
put-upanε-net
aroundCAO
DP ∗W
Qk
Use a convex combination of the ε-net:∑
k µ(k)Qnk .
39 / 42
![Page 115: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/115.jpg)
ε-Net Argument: For Discretized Poisson Channels
Consider a (single-letter) discretized channel Wn : {0, 1} → {0, 1}.
[input probab. simplex]
(1, 0)
(0, 1)
•CAID P∗
put-up an ε-net
[output probab. simplex]
(1, 0)
(0, 1)
•
•
•
•
multiplied by Wn
•CAOD P∗W
Qk
get a projected ε-net
Use a convex combination of the projected ε-net:∑
k µ(k)Qnk
40 / 42
![Page 116: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/116.jpg)
ε-Net Argument: For Discretized Poisson Channels
Consider a (single-letter) discretized channel Wn : {0, 1} → {0, 1}.
[input probab. simplex]
(1, 0)
(0, 1)
•CAID P∗
put-up an ε-net
[output probab. simplex]
(1, 0)
(0, 1)
•
•
•
•
multiplied by Wn
•CAOD P∗W
Qk
get a projected ε-net
Use a convex combination of the projected ε-net:∑
k µ(k)Qnk
40 / 42
![Page 117: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/117.jpg)
ε-Net Argument: For Discretized Poisson Channels
Consider a (single-letter) discretized channel Wn : {0, 1} → {0, 1}.
[input probab. simplex]
(1, 0)
(0, 1)
•CAID P∗
put-up an ε-net
[output probab. simplex]
(1, 0)
(0, 1)
•
•
•
•
multiplied by Wn
•CAOD P∗W
Qk
get a projected ε-net
Use a convex combination of the projected ε-net:∑
k µ(k)Qnk
40 / 42
![Page 118: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/118.jpg)
ε-Net Argument: For Discretized Poisson Channels
Consider a (single-letter) discretized channel Wn : {0, 1} → {0, 1}.
[input probab. simplex]
(1, 0)
(0, 1)
•CAID P∗
put-up an ε-net
[output probab. simplex]
(1, 0)
(0, 1)
•
•
•
•
multiplied by Wn
•CAOD P∗W
Qk
get a projected ε-net
Use a convex combination of the projected ε-net:∑
k µ(k)Qnk
40 / 42
![Page 119: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/119.jpg)
Why This Choice of Output Distn. and not TT13?
• Recall that we chose
Third Term of Q(n)(y) =1
3F
∞∑m=−∞:
0≤p∗+m/T≤1
e−γm2/T
n∏i=1
P∗[m/T ]Wn(yi )
• Need to control normalization constant F .
• By the sifting property of Ds, appears as log F bound on logM∗.
• By direct calculation
F <
∞∑m=−∞
e−γm2/T < 1+
∫ ∞−∞
e−γm2/T dm = 1+
√πT
γ= O(
√T )
• Tomamichel-Tan’s construction in the output distn. space cannothandle the non-stationary W n
n .
41 / 42
![Page 120: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/120.jpg)
Why This Choice of Output Distn. and not TT13?
• Recall that we chose
Third Term of Q(n)(y) =1
3F
∞∑m=−∞:
0≤p∗+m/T≤1
e−γm2/T
n∏i=1
P∗[m/T ]Wn(yi )
• Need to control normalization constant F .
• By the sifting property of Ds, appears as log F bound on logM∗.
• By direct calculation
F <∞∑
m=−∞e−γm
2/T < 1+
∫ ∞−∞
e−γm2/T dm = 1+
√πT
γ= O(
√T )
• Tomamichel-Tan’s construction in the output distn. space cannothandle the non-stationary W n
n .
41 / 42
![Page 121: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/121.jpg)
Why This Choice of Output Distn. and not TT13?
• Recall that we chose
Third Term of Q(n)(y) =1
3F
∞∑m=−∞:
0≤p∗+m/T≤1
e−γm2/T
n∏i=1
P∗[m/T ]Wn(yi )
• Need to control normalization constant F .
• By the sifting property of Ds, appears as log F bound on logM∗.
• By direct calculation
F <
∞∑m=−∞
e−γm2/T < 1+
∫ ∞−∞
e−γm2/T dm = 1+
√πT
γ= O(
√T )
• Tomamichel-Tan’s construction in the output distn. space cannothandle the non-stationary W n
n .
41 / 42
![Page 122: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/122.jpg)
Why This Choice of Output Distn. and not TT13?
• Recall that we chose
Third Term of Q(n)(y) =1
3F
∞∑m=−∞:
0≤p∗+m/T≤1
e−γm2/T
n∏i=1
P∗[m/T ]Wn(yi )
• Need to control normalization constant F .
• By the sifting property of Ds, appears as log F bound on logM∗.
• By direct calculation
F <
∞∑m=−∞
e−γm2/T < 1+
∫ ∞−∞
e−γm2/T dm = 1+
√πT
γ= O(
√T )
• Tomamichel-Tan’s construction in the output distn. space cannothandle the non-stationary W n
n .
41 / 42
![Page 123: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/123.jpg)
Concluding Remarks
• Full understanding of third-order asymptotics for DMCs
• Second- and third-order asymptotics for the Poisson channel
logM∗(T ,Aσ, ε) = T C ∗ +√T V ∗Φ−1(ε) + ρT ,
where1
2logT + O(1) ≤ ρT ≤ logT + O(1)
• Different choices of output distributions.
• Check out arXiv:1903.10438.
42 / 42
![Page 124: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/124.jpg)
Concluding Remarks
• Full understanding of third-order asymptotics for DMCs
• Second- and third-order asymptotics for the Poisson channel
logM∗(T ,Aσ, ε) = T C ∗ +√T V ∗Φ−1(ε) + ρT ,
where1
2logT + O(1) ≤ ρT ≤ logT + O(1)
• Different choices of output distributions.
• Check out arXiv:1903.10438.
42 / 42
![Page 125: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/125.jpg)
Concluding Remarks
• Full understanding of third-order asymptotics for DMCs
• Second- and third-order asymptotics for the Poisson channel
logM∗(T ,Aσ, ε) = T C ∗ +√T V ∗Φ−1(ε) + ρT ,
where1
2logT + O(1) ≤ ρT ≤ logT + O(1)
• Different choices of output distributions.
• Check out arXiv:1903.10438.
42 / 42
![Page 126: Third-Order Asymptotics: Old and Newghan/WPI/VincentSlides.pdf · log n + O(1) • This is an achievability result but BEC doesn’t satisfy assumptions • Assumption may be relaxed](https://reader033.vdocuments.us/reader033/viewer/2022042417/5f32ef977eeeab43f731e147/html5/thumbnails/126.jpg)
Concluding Remarks
• Full understanding of third-order asymptotics for DMCs
• Second- and third-order asymptotics for the Poisson channel
logM∗(T ,Aσ, ε) = T C ∗ +√T V ∗Φ−1(ε) + ρT ,
where1
2logT + O(1) ≤ ρT ≤ logT + O(1)
• Different choices of output distributions.
• Check out arXiv:1903.10438.
42 / 42