cross-lingual ccg induction
Post on 04-May-2022
7 Views
Preview:
TRANSCRIPT
Introduction Derivation Projection Experiments Conclusions References
Cross-lingual CCG Induction
Kilian Evang@texttheater
University of Dusseldorf
2019-06-04NAACL-HLT
1 / 23
Introduction Derivation Projection Experiments Conclusions References
Outline
Introduction
Derivation Projection
Experiments
Conclusions
2 / 23
Introduction Derivation Projection Experiments Conclusions References
Outline
Introduction
Derivation Projection
Experiments
Conclusions
3 / 23
Introduction Derivation Projection Experiments Conclusions References
Combinatory Categorial Grammar
We
NP
sang
S \NP
S<0
3 / 23
Introduction Derivation Projection Experiments Conclusions References
Combinatory Categorial Grammar
We
NP
saw
(S \NP)/NP
the
NP /N
car
N
that
(N \N)/(S /NP)
John
N
bought
(S \NP)/NP
NP∗
S /(S \NP)T>
S /NP>1
N \N>0
N<0
NP>0
NP>0
S \NP>0
S<0
4 / 23
Introduction Derivation Projection Experiments Conclusions References
Appeal
• coordination
• universal rules
• syntax-semantics interface
5 / 23
Introduction Derivation Projection Experiments Conclusions References
Most CCG Parsers
• trained on large treebanks, or
• hand-crafted
6 / 23
Introduction Derivation Projection Experiments Conclusions References
David Blackwell, CC-BY-NC
What about low-resource languages?
7 / 23
Introduction Derivation Projection Experiments Conclusions References
Unsupervised CCG Induction?
target-language text+
magic=
target-language CCG parser
(Bisk and Hockenmaier, 2013; Bisk et al., 2015)
8 / 23
Introduction Derivation Projection Experiments Conclusions References
Cross-lingual CCG Induction?
English CCG parser+
parallel corpus+
magic=
target-language CCG parser
9 / 23
Introduction Derivation Projection Experiments Conclusions References
Cross-lingual CCG Induction via Derivation Projection
parallel corpus+
English CCG derivations+
word alignments+
derivation projection=
target-language CCG derivations=
target-language training data
10 / 23
Introduction Derivation Projection Experiments Conclusions References
Outline
Introduction
Derivation Projection
Experiments
Conclusions
11 / 23
Introduction Derivation Projection Experiments Conclusions References
Derivation Projection
• project lexical categories along word alignments
• n:1 alignment → merge
• word order difference → flip slash
11 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva tre figli
12 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva tre
N4 /N5
figli
12 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva tre
N4 /N5
figli
N5
12 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva
S2 /NP3
tre
N4 /N5
figli
N5
12 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva
S2 /NP3
tre
N4 /N5
figli
N5
N4
>0
12 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva
S2 /NP3
tre
N4 /N5
figli
N5
N4
>0
NP3
∗
12 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 1/3
He
NP1
had
(S2 \NP1)/NP3
three
N4 /N5
sons
N5
N4>0
NP3 ∗
S2 \NP1
>0
S2<0
Aveva
S2 /NP3
tre
N4 /N5
figli
N5
N4
>0
NP3
∗
S2
>0
12 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una pianta molto decorativa
13 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta molto decorativa
13 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto decorativa
13 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto
(N2 \N3)/(N4 \N5)
decorativa
13 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto
(N2 \N3)/(N4 \N5)
decorativa
N4 \N5
13 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto
(N2 \N3)/(N4 \N5)
decorativa
N4 \N5
N2 \N3
>0
13 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto
(N2 \N3)/(N4 \N5)
decorativa
N4 \N5
N2 \N3
>0
N2
<0
13 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 2/3
a
NP1 /N2
very
(N2 /N3)/(N4 /N5)
decorative
N4 /N5
plant
N3
N2 /N3
>0
N2>0
NP1>0
una
NP1 /N2
pianta
N3
molto
(N2 \N3)/(N4 \N5)
decorativa
N4 \N5
N2 \N3
>0
N2
<0
NP1
>0
13 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau es nicht
14 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau es
NP8
nicht
14 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau es
NP8
nicht
(S5 \NP6)\(S3 \NP4)
14 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau
(S3 \NP4)/NP8
es
NP8
nicht
(S5 \NP6)\(S3 \NP4)
14 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau
(S3 \NP4)/NP8
es
NP8
nicht
(S5 \NP6)\(S3 \NP4)
S3 \NP4
>0
14 / 23
Introduction Derivation Projection Experiments Conclusions References
Example 3/3
Do
(S1 \NP2)/(S3 \NP4)
n’t
(S5 \NP6)\(S1 \NP2)
mess
((S3 \NP4)/PR7)/NP8
it
NP8
up
PR7
(S5 \NP6)/(S3 \NP4)<1
×(S3 \NP4)/PR7
>0
S3 \NP4
>0
S5 \NP6
>0
Versau
(S3 \NP4)/NP8
es
NP8
nicht
(S5 \NP6)\(S3 \NP4)
S3 \NP4
>0
S5 \NP6
<0
14 / 23
Introduction Derivation Projection Experiments Conclusions References
Outline
Introduction
Derivation Projection
Experiments
Conclusions
15 / 23
Introduction Derivation Projection Experiments Conclusions References
Training
• Parallel corpus: tatoeba.org
• English parser: EasyCCG trained on CCGrebank
• Word aligments: GIZA++
• Target-language parser: EasyCCG trained on projectedderivations
15 / 23
Introduction Derivation Projection Experiments Conclusions References
Evaluation
• PASCAL challenge on unsupervised grammar induction:Arabic, Czech, Danish, Basque, Dutch, Portuguese, Slovenian,Swedish
• unlabeled dependency f-score
16 / 23
Introduction Derivation Projection Experiments Conclusions References
Training (cont.)
ara ces dan eus nld por slv swe
sentence pairs 20K 11K 21K 2K 44K 161K 835 24Kprojected 7K 4K 11K 590 18K 50K 364 12K
17 / 23
Introduction Derivation Projection Experiments Conclusions References
Baselines
• BH13: CCG induction from raw text + POS tags(Bisk and Hockenmaier, 2013)
• BCH15: CCG induction from raw text(Bisk et al., 2015)
18 / 23
Introduction Derivation Projection Experiments Conclusions References
Results
Language ara ces dan eus nld por slv swe
Monolingual training on PASCALTrain tokens 5K 436K 25K 81K 79K 159K 54K 62KBH13 .651 .507 .585 .450 .544 .629 .464 .669BCH15 .437 .324 .377 .352 .438 .516 .236 .529
Cross-lingual training on TatoebaTrain tokens 20K 11K 21K 2K 44K 161K 835 24Kthis work .468 .449 .630 .290 .614 .678 .350 .637
19 / 23
Introduction Derivation Projection Experiments Conclusions References
Induced Lexicons
eng deu ita nld
SOV (S \NP)\NP - + - +right adj N \N, (N \N)/(N \N) - - + -pro-drop S, S /NP - - + -
20 / 23
Introduction Derivation Projection Experiments Conclusions References
Outline
Introduction
Derivation Projection
Experiments
Conclusions
21 / 23
Introduction Derivation Projection Experiments Conclusions References
Take-home Message
• CCG derivations can be automatically projected along wordalignments
• Cross-lingual supervision helps CCG induction
• Induces linguistically plausible lexicons
• Used for bootstrapping the Parallel Meaning Bank:https://pmb.let.rug.nl
21 / 23
Introduction Derivation Projection Experiments Conclusions References
Bibliography I
Bisk, Y., Christodoulopoulos, C., and Hockenmaier, J. (2015).Labeled grammar induction with minimal supervision. InProceedings of the 53rd Annual Meeting of the Association forComputational Linguistics and the 7th International JointConference on Natural Language Processing (Volume 2: ShortPapers), pages 870–876. Association for ComputationalLinguistics.
Bisk, Y. and Hockenmaier, J. (2013). An HDP model for inducingcombinatory categorial grammars. Transactions of theAssociation for Computational Linguistics, 1:75–88.
22 / 23
Introduction Derivation Projection Experiments Conclusions References
Bibliography II
Honnibal, M., Curran, J. R., and Bos, J. (2010). RebankingCCGbank for improved NP interpretation. In Proceedings of the48th Annual Meeting of the Association for ComputationalLinguistics, pages 207–215. Association for ComputationalLinguistics.
Lewis, M. and Steedman, M. (2014). A* CCG parsing with asupertag-factored model. In Proceedings of the 2014 Conferenceon Empirical Methods in Natural Language Processing(EMNLP), pages 990–1000, Doha, Qatar.
Steedman, M. (2001). The Syntactic Process. The MIT Press.
23 / 23
top related