cross-lingual ccg induction
TRANSCRIPT
![Page 1: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/1.jpg)
Introduction Derivation Projection Experiments Conclusions References
Cross-lingual CCG Induction
Kilian Evang@texttheater
University of Dusseldorf
2019-06-04NAACL-HLT
1 / 23
![Page 2: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/2.jpg)
Introduction Derivation Projection Experiments Conclusions References
Outline
Introduction
Derivation Projection
Experiments
Conclusions
2 / 23
![Page 3: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/3.jpg)
Introduction Derivation Projection Experiments Conclusions References
Outline
Introduction
Derivation Projection
Experiments
Conclusions
3 / 23
![Page 4: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/4.jpg)
Introduction Derivation Projection Experiments Conclusions References
Combinatory Categorial Grammar
We
NP
sang
S \NP
S<0
3 / 23
![Page 5: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/5.jpg)
Introduction Derivation Projection Experiments Conclusions References
Combinatory Categorial Grammar
We
NP
saw
(S \NP)/NP
the
NP /N
car
N
that
(N \N)/(S /NP)
John
N
bought
(S \NP)/NP
NP∗
S /(S \NP)T>
S /NP>1
N \N>0
N<0
NP>0
NP>0
S \NP>0
S<0
4 / 23
![Page 6: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/6.jpg)
Introduction Derivation Projection Experiments Conclusions References
Appeal
• coordination
• universal rules
• syntax-semantics interface
5 / 23
![Page 7: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/7.jpg)
Introduction Derivation Projection Experiments Conclusions References
Most CCG Parsers
• trained on large treebanks, or
• hand-crafted
6 / 23
![Page 8: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/8.jpg)
Introduction Derivation Projection Experiments Conclusions References
David Blackwell, CC-BY-NC
What about low-resource languages?
7 / 23
![Page 9: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/9.jpg)
Introduction Derivation Projection Experiments Conclusions References
Unsupervised CCG Induction?
target-language text+
magic=
target-language CCG parser
(Bisk and Hockenmaier, 2013; Bisk et al., 2015)
8 / 23
![Page 10: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/10.jpg)
Introduction Derivation Projection Experiments Conclusions References
Cross-lingual CCG Induction?
English CCG parser+
parallel corpus+
magic=
target-language CCG parser
9 / 23
![Page 11: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/11.jpg)
Introduction Derivation Projection Experiments Conclusions References
Cross-lingual CCG Induction via Derivation Projection
parallel corpus+
English CCG derivations+
word alignments+
derivation projection=
target-language CCG derivations=
target-language training data
10 / 23
![Page 12: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/12.jpg)
Introduction Derivation Projection Experiments Conclusions References
Outline
Introduction
Derivation Projection
Experiments
Conclusions
11 / 23
![Page 13: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/13.jpg)
Introduction Derivation Projection Experiments Conclusions References
Derivation Projection
• project lexical categories along word alignments
• n:1 alignment → merge
• word order difference → flip slash
11 / 23
![Page 14: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/14.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva tre figli
12 / 23
![Page 15: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/15.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva tre
N4 /N5
figli
12 / 23
![Page 16: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/16.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva tre
N4 /N5
figli
N5
12 / 23
![Page 17: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/17.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva
S2 /NP3
tre
N4 /N5
figli
N5
12 / 23
![Page 18: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/18.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva
S2 /NP3
tre
N4 /N5
figli
N5
N4
>0
12 / 23
![Page 19: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/19.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva
S2 /NP3
tre
N4 /N5
figli
N5
N4
>0
NP3
∗
12 / 23
![Page 20: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/20.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva
S2 /NP3
tre
N4 /N5
figli
N5
N4
>0
NP3
∗
S2
>0
12 / 23
![Page 21: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/21.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una pianta molto decorativa
13 / 23
![Page 22: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/22.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta molto decorativa
13 / 23
![Page 23: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/23.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto decorativa
13 / 23
![Page 24: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/24.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto
(N2 \N3)/(N4 \N5)
decorativa
13 / 23
![Page 25: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/25.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto
(N2 \N3)/(N4 \N5)
decorativa
N4 \N5
13 / 23
![Page 26: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/26.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto
(N2 \N3)/(N4 \N5)
decorativa
N4 \N5
N2 \N3
>0
13 / 23
![Page 27: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/27.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto
(N2 \N3)/(N4 \N5)
decorativa
N4 \N5
N2 \N3
>0
N2
<0
13 / 23
![Page 28: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/28.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto
(N2 \N3)/(N4 \N5)
decorativa
N4 \N5
N2 \N3
>0
N2
<0
NP1
>0
13 / 23
![Page 29: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/29.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau es nicht
14 / 23
![Page 30: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/30.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau es
NP8
nicht
14 / 23
![Page 31: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/31.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau es
NP8
nicht
(S5 \NP6)\(S3 \NP4)
14 / 23
![Page 32: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/32.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau
(S3 \NP4)/NP8
es
NP8
nicht
(S5 \NP6)\(S3 \NP4)
14 / 23
![Page 33: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/33.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau
(S3 \NP4)/NP8
es
NP8
nicht
(S5 \NP6)\(S3 \NP4)
S3 \NP4
>0
14 / 23
![Page 34: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/34.jpg)
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau
(S3 \NP4)/NP8
es
NP8
nicht
(S5 \NP6)\(S3 \NP4)
S3 \NP4
>0
S5 \NP6
<0
14 / 23
![Page 35: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/35.jpg)
Introduction Derivation Projection Experiments Conclusions References
Outline
Introduction
Derivation Projection
Experiments
Conclusions
15 / 23
![Page 36: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/36.jpg)
Introduction Derivation Projection Experiments Conclusions References
Training
• Parallel corpus: tatoeba.org
• English parser: EasyCCG trained on CCGrebank
• Word aligments: GIZA++
• Target-language parser: EasyCCG trained on projectedderivations
15 / 23
![Page 37: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/37.jpg)
Introduction Derivation Projection Experiments Conclusions References
Evaluation
• PASCAL challenge on unsupervised grammar induction:Arabic, Czech, Danish, Basque, Dutch, Portuguese, Slovenian,Swedish
• unlabeled dependency f-score
16 / 23
![Page 38: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/38.jpg)
Introduction Derivation Projection Experiments Conclusions References
Training (cont.)
ara ces dan eus nld por slv swe
sentence pairs 20K 11K 21K 2K 44K 161K 835 24Kprojected 7K 4K 11K 590 18K 50K 364 12K
17 / 23
![Page 39: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/39.jpg)
Introduction Derivation Projection Experiments Conclusions References
Baselines
• BH13: CCG induction from raw text + POS tags(Bisk and Hockenmaier, 2013)
• BCH15: CCG induction from raw text(Bisk et al., 2015)
18 / 23
![Page 40: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/40.jpg)
Introduction Derivation Projection Experiments Conclusions References
Results
Language ara ces dan eus nld por slv swe
Monolingual training on PASCALTrain tokens 5K 436K 25K 81K 79K 159K 54K 62KBH13 .651 .507 .585 .450 .544 .629 .464 .669BCH15 .437 .324 .377 .352 .438 .516 .236 .529
Cross-lingual training on TatoebaTrain tokens 20K 11K 21K 2K 44K 161K 835 24Kthis work .468 .449 .630 .290 .614 .678 .350 .637
19 / 23
![Page 41: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/41.jpg)
Introduction Derivation Projection Experiments Conclusions References
Induced Lexicons
eng deu ita nld
SOV (S \NP)\NP - + - +right adj N \N, (N \N)/(N \N) - - + -pro-drop S, S /NP - - + -
20 / 23
![Page 42: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/42.jpg)
Introduction Derivation Projection Experiments Conclusions References
Outline
Introduction
Derivation Projection
Experiments
Conclusions
21 / 23
![Page 43: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/43.jpg)
Introduction Derivation Projection Experiments Conclusions References
Take-home Message
• CCG derivations can be automatically projected along wordalignments
• Cross-lingual supervision helps CCG induction
• Induces linguistically plausible lexicons
• Used for bootstrapping the Parallel Meaning Bank:https://pmb.let.rug.nl
21 / 23
![Page 44: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/44.jpg)
Introduction Derivation Projection Experiments Conclusions References
Bibliography I
Bisk, Y., Christodoulopoulos, C., and Hockenmaier, J. (2015).Labeled grammar induction with minimal supervision. InProceedings of the 53rd Annual Meeting of the Association forComputational Linguistics and the 7th International JointConference on Natural Language Processing (Volume 2: ShortPapers), pages 870–876. Association for ComputationalLinguistics.
Bisk, Y. and Hockenmaier, J. (2013). An HDP model for inducingcombinatory categorial grammars. Transactions of theAssociation for Computational Linguistics, 1:75–88.
22 / 23
![Page 45: Cross-lingual CCG Induction](https://reader033.vdocuments.us/reader033/viewer/2022050504/62719ba4b095a965903985e4/html5/thumbnails/45.jpg)
Introduction Derivation Projection Experiments Conclusions References
Bibliography II
Honnibal, M., Curran, J. R., and Bos, J. (2010). RebankingCCGbank for improved NP interpretation. In Proceedings of the48th Annual Meeting of the Association for ComputationalLinguistics, pages 207–215. Association for ComputationalLinguistics.
Lewis, M. and Steedman, M. (2014). A* CCG parsing with asupertag-factored model. In Proceedings of the 2014 Conferenceon Empirical Methods in Natural Language Processing(EMNLP), pages 990–1000, Doha, Qatar.
Steedman, M. (2001). The Syntactic Process. The MIT Press.
23 / 23