a neural attention model for sentence summarization
TRANSCRIPT
![Page 2: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/2.jpg)
![Page 3: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/3.jpg)
![Page 4: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/4.jpg)
![Page 5: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/5.jpg)
![Page 6: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/6.jpg)
![Page 7: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/7.jpg)
x
M x1, ...,xM
y1, ...,yNy N
y (N < M)
argmax s(x,y)y 2
s
![Page 8: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/8.jpg)
argmax s(x,y)y 2
argmax s(x,x[m1,...,mN ])m 2 {1, ...,M}N
m 2 {1, ...,M}N , mi�1 < mi
argmax s(x,x[m1,...,mN ])
x[i,j,k]
![Page 9: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/9.jpg)
s(x,y) ⇡N�1X
i=0
g(yi+1,x,yc)
log p(y|x; ✓) ⇡N�1X
i=0
log p(yi+1|x,yc; ✓)
s(x,y) = log p(y|x; ✓)
Cyc
![Page 10: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/10.jpg)
Cs(x,y) ⇡N�1X
i=0
g(yi+1,x,yc)
log p(y|x; ✓) ⇡N�1X
i=0
log p(yi+1|x,yc; ✓)
s(x,y) = log p(y|x; ✓)
C
yc
![Page 11: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/11.jpg)
s(x,y) ⇡N�1X
i=0
g(yi+1,x,yc)
log p(y|x; ✓) ⇡N�1X
i=0
log p(yi+1|x,yc; ✓)
s(x,y) = log p(y|x; ✓)
Cyc
![Page 12: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/12.jpg)
x
y y
argmax s(x,y)y 2
argmax s(x,y)y 2
argmax s(x,x[m1,...,mN ])m 2 {1, ...,M}N
m 2 {1, ...,M}N , mi�1 < mi
argmax s(x,x[m1,...,mN ])
s(x,y) = log p(y|x; ✓) ⇡N�1X
i=0
log p(yi+1|x,yc; ✓)
![Page 13: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/13.jpg)
![Page 14: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/14.jpg)
log p(yi+1|x,yc; ✓)
![Page 15: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/15.jpg)
![Page 16: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/16.jpg)
![Page 17: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/17.jpg)
![Page 18: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/18.jpg)
p(yi+1|yc,x; ✓) / exp(Vh+Wenc(x,yc))
yc = [Eyi�C+1, ...,Eyi]h = tanh(Uyc)
✓ = (E,U,V,W)
![Page 19: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/19.jpg)
p(yi+1|yc,x; ✓) / exp(Vh+Wenc(x,yc))
yc = [Eyi�C+1, ...,Eyi]h = tanh(Uyc)
✓ = (E,U,V,W)
E =Eyi
![Page 20: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/20.jpg)
p(yi+1|yc,x; ✓) / exp(Vh+Wenc(x,yc))
yc = [Eyi�C+1, ...,Eyi]h = tanh(Uyc)
✓ = (E,U,V,W)
ycU h
tanhCV
H H
![Page 21: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/21.jpg)
p(yi+1|yc,x; ✓) / exp(Vh+Wenc(x,yc))
yc = [Eyi�C+1, ...,Eyi]h = tanh(Uyc)
✓ = (E,U,V,W)
H H
V
Vh +
+ V V
W enc(x,yc)
![Page 22: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/22.jpg)
p(yi+1|yc,x; ✓) / exp(Vh+Wenc(x,yc))
yc = [Eyi�C+1, ...,Eyi]h = tanh(Uyc)
✓ = (E,U,V,W)
![Page 23: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/23.jpg)
enc1(x,yc) = p
Tx
p = [1/M, ..., 1/M ] x = [Fx1, ...,FxM ]
![Page 24: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/24.jpg)
8i, l 2 {1, ..., L}, xlj = tanh(max{xl
2i�1, xl2i})
x
0 = [Fx1, ...,FxM ]
8j, enc2(x,yc)j = max x
Li,j
i
8i, l 2 {1, ..., L}, xli = Q
lx
l�1[1�Q,...,1+Q]
![Page 25: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/25.jpg)
enc3(x,yc) = p
Tx
p / exp(xPy
0
c)
y0
c = [Gyi�C+1, ...,Gyi]
x = [Fx1, ...,FxM ]
i+Q
q = i�Q
8i xi =X
xq/Q
![Page 26: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/26.jpg)
![Page 27: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/27.jpg)
(x(1),y(1)), ..., (x(J),y(J))
J
![Page 28: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/28.jpg)
y
⇤ = argmax
Xg(yi+1,x,yc)
y 2
N � 1
i = 0
y⇤
![Page 29: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/29.jpg)
![Page 30: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/30.jpg)
![Page 31: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/31.jpg)
s(y,x) =N�1X
i=0
↵T f(yi+1,x,yc)
![Page 32: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/32.jpg)
f(yi+1,x,yc)
↵ =< 1, 0, ..., 0 >
![Page 33: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/33.jpg)
![Page 34: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/34.jpg)
![Page 35: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/35.jpg)
![Page 36: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/36.jpg)
![Page 37: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/37.jpg)
![Page 38: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/38.jpg)
![Page 39: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/39.jpg)
![Page 40: A neural attention model for sentence summarization](https://reader033.vdocuments.us/reader033/viewer/2022052606/5877bf0b1a28ab2c668b745b/html5/thumbnails/40.jpg)