Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1143–1154,Lisbon, Portugal, 17-21 September 2015. c©2015 Association for Computational Linguistics.

Parsing English into Abstract Meaning Representation UsingSyntax-Based Machine Translation

Michael Pust, Ulf Hermjakob, Kevin Knight, Daniel Marcu, Jonathan MayInformation Sciences InstituteComputer Science Department

University of Southern California{pust, ulf, knight, marcu, jonmay}@isi.edu

Abstract

We present a parser for Abstract MeaningRepresentation (AMR). We treat English-to-AMR conversion within the frameworkof string-to-tree, syntax-based machinetranslation (SBMT). To make this work,we transform the AMR structure into aform suitable for the mechanics of SBMTand useful for modeling. We introducean AMR-specific language model and adddata and features drawn from semantic re-sources. Our resulting AMR parser signif-icantly improves upon state-of-the-art re-sults.

1 Introduction

Abstract Meaning Representation (AMR) is acompact, readable, whole-sentence semantic an-notation (Banarescu et al., 2013). It includes en-tity identification and typing, PropBank semanticroles (Kingsbury and Palmer, 2002), individual en-tities playing multiple roles, as well as treatmentsof modality, negation, etc. AMR abstracts in nu-merous ways, e.g., by assigning the same concep-tual structure to fear (v), fear (n), and afraid (adj).Figure 1 gives an example.

AMR parsing is a new research problem, withonly a few papers published to date (Flani-gan et al., 2014; Wang et al., 2015) and apublicly available corpus of more than 10,000English/AMR pairs.1 New research problemscan be tackled either by developing new algo-rithms/techniques (Flanigan et al., 2014; Wanget al., 2015) or by adapting existing algo-rithms/techniques to the problem at hand. In thispaper, we investigate the second approach.

The AMR parsing problem bears a strong for-mal resemblance to syntax-based machine transla-tion (SBMT) of the string-to-tree variety, in that

1LDC Catalog number 2014T12

inst

fear-01

die-01

ARG0 ARG1

polarity -

inst

soldier

instARG1

The soldier was not afraid of dying.The soldier was not afraid to die.

The soldier did not fear death.

Figure 1: An Abstract Meaning Representation(AMR) with several English renderings.

a string is transformed into a nested structure inboth cases. Because of this, it is appealing to ap-ply the substantial body of techniques already in-vented for SBMT2 to AMR parsing. By re-usingan SBMT inference engine instead of creating cus-tom inference procedures, we lose the ability toembed some task-specific decisions into a customtransformation process, as is done by Flanigan etal. (2014) and Wang et al. (2015). However, wereap the efficiency gains that come from work-ing within a tested, established framework. Fur-thermore, since production-level SBMT systemsare widely available, anyone wishing to generateAMR from text need only follow our recipe andretrain an existing framework with relevant data toquickly obtain state-of-the-art results.

Since SBMT and AMR parsing are, in fact,distinct tasks, as outlined in Figure 2, to adaptthe SBMT parsing framework to AMR parsing,we develop novel representations and techniques.Some of our key ideas include:

1. Introducing an AMR-equivalent representa-tion that is suitable for string-to-tree SBMTrule extraction and decoding (Section 4.1).

2See e.g. the related work section of Huck et al. (2014).

1143

SBMT AMR parsingTarget tree graphNodes labeled unlabeledEdges unlabeled labeledAlignments words to leaves words to leaves

+ words to edgesChildren ordered unorderedAccuracyMetric

BLEU (Papineniet al., 2002)

Smatch (Cai andKnight, 2013)

Figure 2: Differences between AMR parsing andsyntax-based machine translation (SBMT).

2. Proposing a target-side reordering techniquethat leverages the fact that child nodes inAMR are unordered (Section 4.4).

3. Introducing a hierarchical AMR-specific lan-guage model to ensure generation of likelyparent-child relationships (Section 5).

4. Integrating several semantic knowledgesources into the task (Section 6).

5. Developing tuning methods that maximizeSmatch (Cai and Knight, 2013) (Section 7).

By applying these key ideas, which constitutelightweight changes on a baseline SBMT system,we achieve state-of-the-art AMR parsing results.We next describe our baseline, and then describehow we adapt it to AMR parsing.

2 Syntax-Based Machine Translation

Our baseline SBMT system proceeds as follows.Given a corpus of (source string, target tree,source-target word alignment) sentence translationtraining tuples and a corpus of (source, target,score) sentence translation tuning tuples:

1. Rule extraction: A grammar of string-to-tree rules is induced from training tuples us-ing the GHKM algorithm (Galley et al., 2004;Galley et al., 2006).

2. Local feature calculation: Statistical and in-dicator features, as described by Chiang et al.(2009), are calculated over the rule grammar.

3. Language model calculation: A Kneser-Ney-interpolated 5-gram language model(Chen and Goodman, 1996) is learned fromthe yield of the target training trees.

4. Decoding: A beamed bottom-up chart de-coder (Pust and Knight, 2009; Hopkinsand Langmead, 2010) calculates the optimalderivations given a source string and featureparameter set.

5. Tuning: Feature parameters are optimizedusing the MIRA learning approach (Chiang

Corpus Sentences TokensTraining 10,313 218,021

Development 1,368 29,484Test 1,371 30,263

Table 1: Data splits of AMR 1.0, used in this work.Tokens are English, after tokenization.

et al., 2009) to maximize the objective, typi-cally BLEU (Papineni et al., 2002), associatedwith a tuning corpus.

We initially use this system with no modifica-tions and pretend that English–AMR is a languagepair indistinct from any other.

3 Data and Comparisons

We use English–AMR data from the AMR 1.0 cor-pus, LDC Catalog number 2014T12. In contrastto narrow-domain data sources that are often usedin work related to semantic parsing (Price, 1990;Zelle, 1995; Kuhlmann et al., 2004), the AMR cor-pus covers a broad range of news and web forumdata. We use the training, development, and testsplits specified in the AMR corpus (Table 1). Thetraining set is used for rule extraction, languagemodeling, and statistical rule feature calculation.The development set is used both for parameteroptimization and qualitatively for hill-climbing.The test set is held out blind for evaluation. Wepreprocess the English with a simple rule-basedtokenizer and, except where noted, lowercase alldata. We obtain English–AMR alignments by us-ing the unsupervised alignment approach of Pour-damghani et al. (2014), which linearizes the AMRand then applies the models of Brown et al. (1993)with an additional symmetrization constraint.

All parsing results reported in this work areobtained with the Smatch 1.0 software (Cai andKnight, 2013). We compare our results to those ofFlanigan et al. (2014) on the AMR 1.0 data splits;we run that work’s JAMR software according tothe provided instructions.3 We also compare ourresults to published scores in the recent work ofWang et al. (2015). Their work uses slightly dif-ferent data than that used here 4 but in practice wehave not seen significant variation in results.

3https://github.com/jflanigan/jamr4LDC2013E117, a pre-released version of LDC2014T12

that is not generally available.

1144

inst

fear-01

die-01

ARG0 ARG1

polarity -

inst

soldier

instARG1

(a) The original AMR

inst

fear-01

die-01

ARG0

ARG1

polarity -

inst

soldier

inst

ARG1

*inst

(b) Disconnecting multiple parents of(a)

X

Pfear-01 PARG0 PARG1X X

PsoldierPdie-01 PARG1

fear-01 ARG0

soldier

Ppolarity

polarity

X

-ARG1

die-01 ARG1P*

*

X

(c) Edge labels of (b) pushed to leaves,preterminals added

X

Pfear-01 PARG0

PARG1

X

X

Psoldier Pdie-01 PARG1fear-01 ARG0 soldier

Ppolarity

polarity

X

-ARG1

die-01 ARG1

P*

*

fear-01

fear-01

X

(d) Restructuring (c) with concept la-bels as intermediates

X

Pfear-01 PARG0

PARG1

X

X

Psoldier Pdie-01PARG1fear-01 ARG0 soldier

Ppolarity

polarity

X

-ARG1

die-01 ARG1

P*

*

ARG1

ARG0

X

(e) Restructuring (c) with role labelsas intermediates

X

Pfear-01 PARG0 PARG1X X

PsoldierPdie-01 PARG1

fear-01 ARG0

soldier

Ppolarity

polarity

Spolarity

-ARG1

die-01 ARG1P*

*

X

(f) String preterminal relabeling of (c)

X

Pfear-01 PARG0 PARG1X X

PsoldierPdie-01

PARG1fear-01 ARG0

soldier

Ppolarity

polarity

X

-ARG1

die-01 ARG1P*

*

The soldier was not afraid of dying

X

(g) Original alignment of English to(c)

X

Pfear-01

PARG0

PARG1X

X

Psoldier

Pdie-01

PARG1

fear-01

ARG0

soldier

Ppolarity

polarity

X

-ARG1

die-01

ARG1

P*

*

The soldier was not afraid of dying

X

(h) After reordering of (g)

X

Pfear-01

PARG0PARG1

XX

Psoldier

Pdie-01PARG1

fear-01

ARG0soldier

Ppolarity

polarity

Spolarity

-

ARG1

die-01 ARG1

P*

*

ARG0

polarityX

(i) Restructured, relabeled, and re-ordered tree: (e), (f), and (h)

Figure 3: Transformation of AMR into tree structure that is acceptable to GHKM rule extraction (Galleyet al., 2004; Galley et al., 2006) and yields good performance.

4 AMR Transformations

In this section we discuss various transformationsto our AMR data. Initially, we concern ourselveswith converting AMR into a form that is amenableto GHKM rule extraction and string to tree decod-ing. We then turn to structural transformationsdesigned to improve system performance. Fig-ure 3 progressively shows all the transformationsdescribed in this section; the example we follow isshown in its original form in Figure 3a. We notethat all transformations are done internally; the in-put to the final system is a sentence and the outputis an AMR. We further observe that all transfor-mations are data-driven and language agnostic.

4.1 Massaging AMRs into Syntax-Style Trees

The relationships in AMR form a directed acyclicgraph (DAG), but GHKM requires a tree, sowe must begin our transformations by discardingsome information. We arbitrarily disconnect allbut a single parent from each node (see Figure 3b).

This is the only lossy modification we make toour AMR data. As multi-parent relationships oc-cur 1.05 times per training sentence and at leastonce in 48% of training sentences, this is indeed aregrettable loss. We nevertheless make this mod-ification, since it allows us to use the rest of ourstring-to-tree tools.

AMR also contains labeled edges, unlike theconstituent parse trees we are used to working within SBMT. These labeled edges have informativecontent and we would like to use the alignmentprocedure of Pourdamghani et al. (2014), whichaligns words to edges as well as to terminal nodes.So that our AMR trees are compatible with bothour desired alignment approach and our desiredrule extraction approach, we propagate edge labelsto terminals via the following procedure:

1. For each node n in the AMR tree we createa corresponding node m with the all-purposesymbol ‘X’ in the SBMT-like tree. Outgoingedges from n come in two flavors: concept

1145

edges, labeled ‘inst,’ which connect n to aterminal concept such as fear-01, and roleedges, which have a variety of labels such asARG0 and name, and connect n to anotherinstance or to a string.5 A node has one in-stance edge and zero or more role edges. Weconsider each type of edge separately.

2. For each outgoing role edge we insert two un-labeled edges into the corresponding trans-formation; the first is an edge from m to aterminal bearing the original edge’s role label(a so-called role label edge), and the second(a role filler edge) connects m to the trans-formation of the original edge’s target node,which we process recursively. String targetsof a role receive an ‘X’ preterminal to be con-sistent with the form of role filler edges.

3. For the outgoing concept edge we insert anunlabeled edge connecting m and the con-cept. It is unambiguous to determine whichof m’s edges is the concept edge and whichedges constitute role label edges and theircorresponding role filler edges, as long aspaired label and filler edges are adjacent.

4. Since SBMT expects trees with preterminals,we simply replicate the label identities ofconcepts and role labels, adding a marker (‘P’in Figure 3) to distinguish preterminals.

The complete transformation can be seen in Fig-ure 3c. Apart from multiple parent ancestry, theoriginal AMR can be reconstructed deterministi-cally from this SBMT-compliant rewrite.

4.2 Tree RestructuringWhile the transformation in Figure 3c is accept-able to GHKM, and hence an entire end-to-endAMR parser may now be built with SBMT tools,the resulting parser does not exhibit very good per-formance (Table 3, first line). The trees we arelearning on are exceedingly flat, and thus yieldrules that do not generalize sufficiently. Rules pro-duced from the top of the tree in Figure 3c, suchas that in Figure 4a, are only appropriate for caseswhere fear-01 has exactly three roles: ARG0(agent), ARG1 (patient), and polarity.

We follow the lead of Wang et al. (2010), whoin turn were influenced by similar approaches inmonolingual parsing (Collins, 1997; Charniak,2000), and re-structure trees at nodes with more

5In Figure 3 the negative polarity marker ‘-’ is a string.Disconnected referents labeled ‘*’ are treated as AMR in-stances with no roles.

than three children (i.e. instances with more thanone role), to allow generalization of flat structures.

However, our trees are unlike syntactic con-stituent trees in that they do not have labeled non-terminal nodes, so we have no natural choice ofan intermediate (“bar”) label. We must choose ameaningful label to characterize an instance andits roles. We initially choose the concept label,resulting in trees like that in Figure 3d, in which achain of fear-01 nodes is used to unflatten the root,which has instance fear-01.

This attempt at re-structuring yields rules likethat in Figure 4b, which are general in form butare tied to the concept context in which they wereextracted. This leads to many redundant rules andblows up the nonterminal vocabulary size to ap-proximately 8,000, the size of the concept vocabu-lary. Furthermore, the rules elicited by this proce-dure encourage undesirable behavior such as theimmediate juxtaposition of two rules generatingARG1.

We next consider restructuring with the imme-diately dominant role labels, resulting in trees likethat in Figure 3e and rules like that in Figure 4c.The shape of the structure added is the same as inFigure 3d but the bar nodes now take their labelsfrom their second children. This approach leads tomore useful rules with fewer undesirable proper-ties.

4.3 Tree RelabelingAMR strings have an effective preterminal label of‘X,’ which allows them to compete with full AMRinstances at decode time. However, whether or nota role is filled by a string or an instance is highlydependent on the kind of role being filled. Thepolarity and mode roles, for instance, are nearlyalways filled by strings, but ARG0 and ARG1 arealways filled by instances. The quant role, whichis used for representation of numerical quantities,can be filled by an instance (e.g. for approximatequantities such as ‘about 3’) or a string. To capturethis behavior we relabel string preterminals of thetree with labels indicating role identity and stringsubsumption. This relabeling, replaces, for exam-ple, one ‘X’ preterminal in Figure 3c with “Spo-larity,” as shown in Figure 3f.

4.4 Tree ReorderingFinally, let us consider the alignments betweenEnglish and AMR. As is known in SBMT, non-monotone alignments can lead to large, unwieldy

1146

X

Pfear-01 PARG0 PARG1

x1:X

x2:Xfear-01 ARG0

Ppolarity

polarity

X

-ARG1

The x1 was not afraid x2

(a) A rule extracted from the AMR tree ofFigure 3c. All roles seen in training must beused.

PARG1 x2:X

ARG1

fear-01

x1:fear-01

x1 x2

(b) A rule from the AMR tree ofFigure 3d. Many nearly iden-tical rules of this type are ex-tracted, and this rule can beused multiple times in a singlederivation.

PARG1 x2:X

ARG1

ARG1

x1:ARG0

x1 x2

(c) A rule from the AMR tree of Fig-ure 3e. This rule can be used indepen-dent of the concept context it was ex-tracted from and multiple reuse is dis-couraged.

Figure 4: Impact of restructuring on rule extraction.

rules and in general make decoding more diffi-cult (May and Knight, 2007). While this is oftenan unavoidable fact of life when trying to trans-late between two natural languages with differentsyntactic behavior, it is an entirely artificial phe-nomenon in this case. AMR is an unordered repre-sentation, yet in order to use an SBMT infrastruc-ture we must declare an order of the AMR tree.This means we are free to choose whatever orderis most convenient to us, as long as we keep rolelabel edges immediately adjacent to their corre-sponding role filler edges to preserve conversionback to the edge-labeled AMR form. We thuschoose the order that is as close as possible tothat of the source yet still preserves these con-straints. We use a simple greedy bottom-up ap-proach that permutes the children of each internalnode of the unrestructured tree so as to minimizecrossings. This leads to a 79% overall reduction incrossings and is exemplified in Figure 3g (before)and Figure 3h (after). We may then restructure ourtrees, as described above, in an instance-outwardmanner. The final restructured, relabeled, and re-ordered tree is shown in Figure 3i.

5 AMR Language Models

We now turn to language models of AMRs, whichhelp us prefer reasonable target structures over un-reasonable ones.

Our first language model is unintuitivelysimple—we pretend there is a language calledAMRese that consists of yields of our restruc-tured AMRs. An example AMRese string fromFigure 3i is ‘ARG0 soldier polarity -

fear-01 ARG1 die-01 ARG1 *.’ We thenbuild a standard n-gram model for AMRese.

It also seems sensible to judge the correctness ofan AMR by calculating the empirical probabilityof the concepts and their relations to each other.This is the motivation behind the following modelof an AMR:6

We define an AMR instance i = (c, R), wherec is a concept and R is a set of roles. Wedefine an AMR role r = (l, i), where l is arole label, and i is an AMR instance labeled l.For an AMR instance i let ci be the concept ofi’s parent instance, and li be the label of therole that i fills with respect to its parent. Wealso define the special instance and role labelsROOT and STOP. Then, we define PAMR(i|li, ci),the conditional probability of AMR instance igiven its ancestry as PAMR(i = (c, R)|li, ci) =P (c|li, ci)

∏r∈R

PRole(r|c) × P (STOP|c), where

PRole(r = (l, i)|c) = P (l|c)PAMR(i|l, c).We define P (c|li, ci), P (l|c), and P (STOP|c)

as empirical conditional probabilities, Witten-Bellinterpolated (Witten and Bell, 1991) to lower-order models by progressively discarding contextfrom the right.7 We model exactly one STOPevent per instance. We define the probability ofa full-sentence AMR i as PAMR(i|ROOT) whereROOT in this case serves as both parent conceptand role label.

6This model is only defined over AMRs that can be rep-resented as trees, and not over all AMRs. Since tree AMRsare a prerequisite of our system we did not yet investigatewhether this model could be sufficiently generalized.

7That is, P (c|li, ci) is interpolated with P (c|li) and thenP (c).

1147

System Tune TestAMRese n-gram LM 61.7 59.7

AMR LM 59.1 57.1both LMs 62.3 60.6

Table 2: The effect of AMRese n-gram and AMRLMs on Smatch quality.

As an example, the instance associatedwith concept die-01 in Figure 3b has li =ARG1 and ci = fear-01, so we mayscore it as P (die-01|ARG1,fear-01) ×P (ARG1|die-01) × P (STOP|die-01) ×P (*|ARG1,die-01)

In Table 2 we compare the effect of varyingLMs on Smatch quality. The AMR LM by itselfis inferior to the AMRese n-gram LM, but com-bining the two yields superior quality.

6 Adding External Semantic Resources

Although we are engaged in the task of semanticparsing, we have not yet discussed the use of anysemantic resources. In this section we rectify thatomission.

6.1 Rules from Numerical Quantities andNamed Entities

While the majority of string-to-tree rules in SBMTsystems are extracted from aligned parallel data, itis common practice to dynamically generate addi-tional rules to handle the translation of dates andnumerical quantities, as these follow common pat-terns and are easily detected at decode-time. Wefollow this practice here, and additionally detectperson names at decode-time using the StanfordNamed Entity Recognizer (Finkel et al., 2005).We use cased, tokenized source data to build thedecode-time rules. We add indicator features tothese rules so that our tuning methods can decidehow favorable the resources are. We leave as fu-ture work the incorporation of named-entity rulesfor other classes, since most available named-entity recognition beyond person names is at agranularity level that is incompatible with AMR(e.g. we can recognize ‘Location’ but not distin-guish between ‘City’ and ‘Country’).

6.2 Hierarchical Semantic Categories

In order to further generalize our rules, we mod-ify our training data AMRs once more, this timereplacing the identity preterminals over concepts

X

think

PARG0PARG1

XX

skilled-worker

diePARG1

fear-01

ARG0soldier

Ppolarity

polarity

Spolarity

-

ARG1

die-01 ARG1

P*

*

ARG0

polarityX

Figure 5: Final modification of the AMR data; se-mantically clustered preterminal labels are addedto concepts.

with preterminals designed to enhance the appli-cability of our rules in semantically similar con-texts. For each concept c expressed in AMR, weconsult WordNet (Fellbaum, 1998) and a curatedset of gazetteers and vocabulary lists to identifya hierarchy of increasingly general semantic cat-egories that describe the concept. So as not tobe overwhelmed by the many fine-grained distinc-tions present in WordNet, we pre-select around100 salient semantic categories from the WordNetontology. When traversing the WordNet hierarchy,we propagate a smoothed count8 of the numberof examples seen per concept sense,9 combiningcounts when paths meet. For each selected se-mantic category s encountered in the traversal, wecalculate a weight by dividing the propagated ex-ample count for c at s by the frequency s was pro-posed over all AMR concepts. We then assign cto the highest scoring semantic category s. An ex-ample calculation for the concept computer isshown in Figure 7.

We apply semantic categories to our dataas replacements for identity preterminals ofconcepts. This leads to more general, morewidely-applicable rules. For example, withthis transformation, we can parse correctlynot only contexts in which “soldiers die,” butalso contexts in which other kinds of “skilledworkers die.” Figure 5 shows the addition ofsemantic preterminals to the tree from Figure3i. We also incorporate semantic categoriesinto the AMR LM. For concept c, let sc bethe semantic category of c. Then we reformu-

8We use very simple smoothing, and add 0.1 to the pro-vided example counts.

9Since WordNet senses do not correspond directly toPropBank or AMR senses, we simply use a lexical match andmust consider all observed senses for that match.

1148

System Sec. Tune Testflat trees 4.1 51.6 49.9

concept restructuring 4.2 57.2 55.3role restructuring (rr) 4.2 60.8 58.6

rr + string preterminal relabeling (rl) 4.3 61.3 59.7rr + rl + reordering (ro) 4.4 61.7 59.7rr + rl + ro + AMR LM 5 62.3 60.6

rr + rl + ro + AMR LM + date/number/name rules (dn) 6.1 63.3 61.3rr + rl + ro + AMR LM + dn + semantic categories (sc) 6.2 66.2 64.3

rr + rl + ro + AMR LM + dn + sc + morphological normalization (mn) 6.3 67.3 65.4rr + rl + ro + AMR LM + dn + sc + mn, rule-based alignments 6.4 68.3 66.3

rr + rl + ro + AMR LM + dn + sc + mn, rule-based + unsupervised alignments 6.4 69.0 67.1JAMR (Flanigan et al., 2014) 9 58.8 58.2

dependency parse-based (Wang et al., 2015) 9 N/A 63

Table 3: AMR parsing Smatch scores for the experiments in this work. We provide a cross-reference tothe section of this paper that describes each of the evaluated systems. Entries in bold are improvementsover JAMR (Flanigan et al., 2014). Test entries underlined are improvements over the dependency-basedwork of Wang et al. (2015). Human inter-annotator Smatch performance is in the 79-83 range (Cai andKnight, 2013).

English AMResetigers tigerasbestos asbestosquietly quietnonexecutive executive polarity ‘-’broke up break-up-08

Table 4: Lexical conversions to AMRese formdue to the morphological normalization rules de-scribed in Section 6.3.

late PAMR(i|li, ci) as PAMR(i = (c, R)|li, ci) =P (sc|li, sci

, ci)P (c|sc, li, sci, ci)

∏r∈R

PRole(r|c) ×P (STOP|sc, c), where PRole(r = (l, i)|c) =P (l|sc, c)× PAMR(i|l, c).

6.3 Morphological Normalization

While we rely heavily on the relationships be-tween words in-text and concept nodes expressedin parallel training data, we find this is not suf-ficient for complete coverage. Thus we also in-clude a run-time module that generates AMResebase forms at the lexical level, expressing relation-ships such as those depicted in Table 4. We buildthese dictionary rules using three resources:

1. An inflectional morphological normalizingtable, comprising a lexicon with 84,558 en-tries, hand-written rules for regular inflec-tional morphology, and hand-written lists of

irregular verbs, nouns, and adjectives.2. Lists of derivational mappings (e.g. ‘quietly’→ ‘quiet’, ‘naval’→ ‘navy’).

3. PropBank framesets, which we use, e.g., tomap the morphologically normalized ‘breakup’ (from ‘broke up’) into a sense match,such as break-up-08.

6.4 Semantically informed Rule-basedAlignments

For our final incorporation of semantic resourceswe revisit the English-to-AMR alignments used toextract rules. As an alternative to the unsupervisedapproach of Pourdamghani et al. (2014), we buildalignments by taking a linguistically-aware, super-vised heuristic approach to alignment:

First, we generate a large number of poten-tial links between English and AMR. We attemptto link English and AMR tokens after conver-sion through resources such as a morphologicalanalyzer, a list of 3,235 pertainym pairs (e.g.adj-‘gubernatorial’ → noun-‘governor’), a list of2,444 adverb/adjective pairs (e.g. ‘humbly’ →‘humble’), a list of 2,076 negative polarity pairs(e.g. ‘illegal’→ ‘legal’), and a list of 2,794 knownEnglish-AMR transformational relationships (e.g.‘asleep’ → sleep-01, ‘advertiser’ → personARG0-of advertise-01, ‘Greenwich MeanTime’→ GMT). These links are then culled basedon context and AMR structure. For example, inthe sentence “The big fish ate the little fish,” ini-

1149

Figure 6: BLEU of AMRese and Smatch correlateclosely when tuning.

tially both English ‘fish’ are aligned to both AMRfish. However, based on the context of ‘big’ and‘little’ the spurious links are removed.

In our experiments we explore both replacingthe unsupervised alignments of Pourdamghani etal. (2014) with these alignments and concatenat-ing the two alignment sets together, essentiallydoubling the size of the training corpus. Becausethe different alignments yield different target-sidetree reorderings, it is necessary to build separate5-gram AMRese language models.10 When us-ing both alignment sets together, we also use bothAMRese language models simultaneously.

7 Tuning

We would like to tune our feature weights to max-imize Smatch directly. However, a very con-venient alternative is to compare the AMReseyields of candidate AMR parses to those of ref-erence AMRese strings, using a BLEU objectiveand forest-based MIRA (Chiang et al., 2009). Fig-ure 6 shows that MIRA tuning with BLEU overAMRese tracks closely with Smatch. Note that,for experiments using reordered AMR trees, thisrequires obtaining similarly permuted referencetuning AMRese and hence requires alignments onthe development corpus. When using unsuper-vised alignments we may simply run inferenceon the trained alignment model to obtain devel-opment alignments. The rule-based aligner runsone sentence at a time and can be employed onthe development corpus. When using both sets ofalignments, each approach’s AMRese is used as

10The AMR LM is insensitive to reordering so we do notneed to vary it when varying alignments.

computer

computing device

machine

artefact = 6.1/1143

unit

estimator

person = 0.1/814

causal-agency = 0.1/930

physical-object = 6.2/2218

entity = 6.2/4381

6.1

0.1

0.1

0.10.1

6.1

6.2

6.1

6.1

6.1

Figure 7: WordNet hierarchy for computer.Pre-selected salient WordNet categories areboxed. Smoothed sense counts are propagatedup the hierarchy and re-combined at join points.Scores are calculated by dividing propagatedsense count by count of the category’s prevalenceover the set of AMR concepts. The double boxindicates the selection of artefact as the categorylabel for computer.

a development reference (i.e. each developmentsentence has two possible reference translations).

8 Results and Discussion

Our AMR parser’s performance is shown in Ta-ble 3. We progressively show the incremental im-provements and compare to the systems of Flani-gan et al. (2014) and Wang et al. (2015). Purelytransforming AMR data into a form that is com-patible with the SBMT pipeline yields suboptimalresults, but by adding role-based restructuring, re-labeling, and reordering, as described in Section4, we are able to surpass Flanigan et al. (2014).Adding an AMR LM and semantic resources in-creases scores further, outperforming Wang et al.(2015). Rule-based alignments are an improve-ment upon unsupervised alignments, but concate-nating the two alignments is even better. We com-pare rule set sizes of the various systems in Ta-ble 5; initially we improve the rule set by remov-ing numerous overly brittle rules but then succes-sive changes progressively add useful rules. Theparser is available for public download and use athttp://amr.isi.edu.

1150

System Rulesflat trees 1,430,124

concept restructuring 678,265role restructuring (rr) 660,582

rr + preterminal relabeling (rl) 661,127rr + rl + semantic categories (sc) 765,720

rr + rl + sc + reordering (ro) 790,624rr + rl + sc + ro + rule-based alignments 908,318

rr + rl + sc + ro + both alignments 1,306,624

Table 5: Comparison of extracted rule set size on the systems evaluated in this work. Note that, ascompared to Table 3, only systems that affect the rule size are listed.

9 Related WorkThe first work that addressed AMR parsing wasthat of Flanigan et al. (2014). In that work, mul-tiple discriminatively trained models are used toidentify individual concept instances and then aminimum spanning tree algorithm connects theconcepts. That work was extended and improvedupon by Werling et al. (2015). Recent work byWang et al. (2015) also uses a two-pass approach;dependency parses are modified by a tree-walkingalgorithm that adds edge labels and restructures toresolve discrepancies between dependency stan-dards and AMR’s specification. In contrast tothese works, we use a single-pass approach andre-use existing machine translation architecture,adapting to the AMR parsing task by modify-ing training data and adding lightweight AMR-specific features.

Several other recent works have used a ma-chine translation approach to semantic parsing,but all have been applied to domain data that ismuch narrower and an order of magnitude smallerthan that of AMR, primarily the Geoquery cor-pus (Zelle and Mooney, 1996). The WASP sys-tem of Wong and Mooney (2006) uses hierarchi-cal SMT techniques and does not apply semantic-specific improvements; its extension (Wong andMooney, 2007) incorporates a target-side reorder-ing component much like the one presented inSection 4.4. Jones et al. (2012a) cast semanticparsing as an instance of hyperedge replacementgrammar transduction; like this work they use anIBM model-influenced alignment algorithm and aGHKM-based extraction algorithm. Andreas etal. (2013) use phrase-based and hierarchical SMTtechniques on Geoquery. Like this work, they per-form a transformation of the input semantic repre-sentation so that it is amenable to use in an exist-

ing machine translation system. However, they areunable to reach the state of the art in performance.Li et al. (2013) directly address GHKM’s word-to-terminal alignment requirement by extending thatalgorithm to handle word-to-node alignment.

Our SBMT system is grounded in the theory oftree transducers, which were applied to the task ofsemantic parsing by Jones et al. (2011; 2012b).

Semantic parsing in general and AMR parsingspecifically can be considered a subsumption ofmany semantic resolution sub-tasks, e.g. namedentity recognition (Nadeau and Sekine, 2007), se-mantic role labeling (Gildea and Jurafsky, 2002),word sense disambiguation (Navigli, 2009) and re-lation finding (Bach and Badaskar, 2007).

10 Conclusion

By restructuring our AMRs we are able to converta sophisticated SBMT engine into a baseline se-mantic parser with little additional effort. By fur-ther restructuring our data to appropriately modelthe behavior we want to capture, we are able torapidly achieve state-of-the-art results. Finally, byincorporating novel language models and externalsemantic resources, we are able to increase qualityeven more. This is not the last word on AMR pars-ing, as fortunately, machine translation technologyprovides more low-hanging fruit to pursue.

Acknowledgments

Thanks to Julian Schamper and Allen Schmaltzfor early attempts at this problem. This workwas sponsored by DARPA DEFT (FA8750-13-2-0045), DARPA BOLT (HR0011-12-C-0014), andDARPA Big Mechanism (W911NF-14-1-0364).

1151

ReferencesJacob Andreas, Andreas Vlachos, and Stephen Clark.

2013. Semantic parsing as machine translation. InProceedings of the 51st Annual Meeting of the As-sociation for Computational Linguistics (Volume 2:Short Papers), pages 47–52, Sofia, Bulgaria, Au-gust. Association for Computational Linguistics.

Nguyen Bach and Sameer Badaskar. 2007. A Reviewof Relation Extraction. Unpublished. http://www.cs.cmu.edu/˜nbach/papers/A-survey-on-Relation-Extraction.pdf.

Laura Banarescu, Claire Bonial, Shu Cai, MadalinaGeorgescu, Kira Griffitt, Ulf Hermjakob, KevinKnight, Philipp Koehn, Martha Palmer, and NathanSchneider. 2013. Abstract meaning representationfor sembanking. In Proceedings of the 7th Linguis-tic Annotation Workshop and Interoperability withDiscourse, pages 178–186, Sofia, Bulgaria, August.Association for Computational Linguistics.

Peter F. Brown, Vincent J. Della Pietra, StephenA. Della Pietra, and Robert L. Mercer. 1993.The mathematics of statistical machine translation:parameter estimation. Computational Linguistics,19(2):263–311.

Shu Cai and Kevin Knight. 2013. Smatch: an evalua-tion metric for semantic feature structures. In Pro-ceedings of the 51st Annual Meeting of the Associa-tion for Computational Linguistics (Volume 2: ShortPapers), pages 748–752, Sofia, Bulgaria, August.Association for Computational Linguistics.

Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of the 1st NorthAmerican Chapter of the Association for Computa-tional Linguistics Conference, NAACL 2000, pages132–139, Stroudsburg, PA, USA. Association forComputational Linguistics.

Stanley F. Chen and Joshua Goodman. 1996. An em-pirical study of smoothing techniques for languagemodeling. In Proceedings of the 34th Annual Meet-ing on Association for Computational Linguistics,ACL ’96, pages 310–318, Stroudsburg, PA, USA.Association for Computational Linguistics.

David Chiang, Kevin Knight, and Wei Wang. 2009.11,001 new features for statistical machine trans-lation. In Proceedings of Human Language Tech-nologies: The 2009 Annual Conference of the NorthAmerican Chapter of the Association for Compu-tational Linguistics, pages 218–226, Boulder, Col-orado, June. Association for Computational Linguis-tics.

Michael Collins. 1997. Three generative, lexicalisedmodels for statistical parsing. In Proceedings of the35th Annual Meeting of the Association for Com-putational Linguistics, pages 16–23, Madrid, Spain,July. Association for Computational Linguistics.

Christiane Fellbaum. 1998. WordNet: An ElectronicLexical Database. Bradford Books.

Jenny Rose Finkel, Trond Grenager, and ChristopherManning. 2005. Incorporating non-local informa-tion into information extraction systems by gibbssampling. In Proceedings of the 43rd Annual Meet-ing of the Association for Computational Linguis-tics (ACL’05), pages 363–370, Ann Arbor, Michi-gan, June. Association for Computational Linguis-tics.

Jeffrey Flanigan, Sam Thomson, Jaime Carbonell,Chris Dyer, and Noah A. Smith. 2014. A discrim-inative graph-based parser for the abstract mean-ing representation. In Proceedings of the 52nd An-nual Meeting of the Association for ComputationalLinguistics (Volume 1: Long Papers), pages 1426–1436, Baltimore, Maryland, June. Association forComputational Linguistics.

Michel Galley, Mark Hopkins, Kevin Knight, andDaniel Marcu. 2004. What’s in a translationrule? In Daniel Marcu Susan Dumais and SalimRoukos, editors, HLT-NAACL 2004: Main Proceed-ings, pages 273–280, Boston, Massachusetts, USA,May 2 - May 7. Association for Computational Lin-guistics.

Michel Galley, Jonathan Graehl, Kevin Knight, DanielMarcu, Steve DeNeefe, Wei Wang, and IgnacioThayer. 2006. Scalable inference and training ofcontext-rich syntactic translation models. In Pro-ceedings of the 21st International Conference onComputational Linguistics and 44th Annual Meet-ing of the Association for Computational Linguis-tics, pages 961–968, Sydney, Australia, July. Asso-ciation for Computational Linguistics.

Daniel Gildea and Daniel Jurafsky. 2002. Automaticlabeling of semantic roles. Computational Linguis-tics, 28(3):245–288, September.

Mark Hopkins and Greg Langmead. 2010. SCFG de-coding without binarization. In Proceedings of the2010 Conference on Empirical Methods in NaturalLanguage Processing, pages 646–655, Cambridge,MA, October. Association for Computational Lin-guistics.

Matthias Huck, Hieu Hoang, and Philipp Koehn. 2014.Augmenting string-to-tree and tree-to-string transla-tion with non-syntactic phrases. In Proceedings ofthe Ninth Workshop on Statistical Machine Trans-lation, pages 486–498, Baltimore, Maryland, USA,June. Association for Computational Linguistics.

Bevan Jones, Mark Johnson, and Sharon Goldwa-ter. 2011. Formalizing semantic parsing with treetransducers. In Proceedings of the AustralasianLanguage Technology Association Workshop 2011,pages 19–28, Canberra, Australia, December.

Bevan Jones, Jacob Andreas, Daniel Bauer,Karl Moritz Hermann, and Kevin Knight. 2012a.

1152

Semantics-based machine translation with hy-peredge replacement grammars. In Proceedingsof COLING 2012, pages 1359–1376, Mumbai,India, December. The COLING 2012 OrganizingCommittee.

Bevan Jones, Mark Johnson, and Sharon Goldwater.2012b. Semantic parsing with bayesian tree trans-ducers. In Proceedings of the 50th Annual Meet-ing of the Association for Computational Linguis-tics (Volume 1: Long Papers), pages 488–496, JejuIsland, Korea, July. Association for ComputationalLinguistics.

Paul Kingsbury and Martha Palmer. 2002. From tree-bank to propbank. In In Language Resources andEvaluation.

Gregory Kuhlmann, Peter Stone, Raymond J. Mooney,and Jude W. Shavlik. 2004. Guiding a reinforce-ment learner with natural language advice: Initialresults in robocup soccer. In The AAAI-2004 Work-shop on Supervisory Control of Learning and Adap-tive Systems, July.

Peng Li, Yang Liu, and Maosong Sun. 2013. An ex-tended ghkm algorithm for inducing lambda-scfg.In Marie desJardins and Michael L. Littman, editors,AAAI. AAAI Press.

Jonathan May and Kevin Knight. 2007. Syntacticre-alignment models for machine translation. InJason Eisner and Taku Kudo, editors, Proceedingsof the 2007 Joint Conference on Empirical Meth-ods in Natural Language Processing and Computa-tional Natural Language Learning, pages 360–368,Prague, Czech Republic, June 28 – June 30. Associ-ation for Computational Linguistics.

David Nadeau and Satoshi Sekine. 2007. A surveyof named entity recognition and classification. Lin-guisticae Investigationes, 30(1):3–26, January. Pub-lisher: John Benjamins Publishing Company.

Roberto Navigli. 2009. Word sense disambiguation:A survey. ACM Comput. Surv., 41(2):10:1–10:69,February.

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automaticevaluation of machine translation. In Proceedingsof the 40th Annual Meeting on Association for Com-putational Linguistics, ACL ’02, pages 311–318,Stroudsburg, PA, USA. Association for Computa-tional Linguistics.

Nima Pourdamghani, Yang Gao, Ulf Hermjakob, andKevin Knight. 2014. Aligning English strings withAbstract Meaning Representation graphs. In Pro-ceedings of the 2014 Conference on Empirical Meth-ods in Natural Language Processing (EMNLP),pages 425–429, Doha, Qatar, October. Associationfor Computational Linguistics.

P. J. Price. 1990. Evaluation of spoken language sys-tems: The ATIS domain. In Proceedings of theWorkshop on Speech and Natural Language, HLT’90, pages 91–95, Stroudsburg, PA, USA. Associa-tion for Computational Linguistics.

Michael Pust and Kevin Knight. 2009. Faster mtdecoding through pervasive laziness. In Proceed-ings of Human Language Technologies: The 2009Annual Conference of the North American Chap-ter of the Association for Computational Linguistics,Companion Volume: Short Papers, pages 141–144,Boulder, Colorado, June. Association for Computa-tional Linguistics.

Wei Wang, Jonathan May, Kevin Knight, and DanielMarcu. 2010. Re-structuring, re-labeling, andre-aligning for syntax-based machine translation.Computational Linguistics, 36(2):247–277, June.

Chuan Wang, Nianwen Xue, and Sameer Pradhan.2015. A transition-based algorithm for amr parsing.In Proceedings of the 2015 Conference of the NorthAmerican Chapter of the Association for Computa-tional Linguistics: Human Language Technologies,pages 366–375, Denver, Colorado, May–June. As-sociation for Computational Linguistics.

Keenon Werling, Gabor Angeli, and Christopher D.Manning. 2015. Robust subgraph generation im-proves abstract meaning representation parsing. InProceedings of the 53rd Annual Meeting of theAssociation for Computational Linguistics and the7th International Joint Conference on Natural Lan-guage Processing (Volume 1: Long Papers), pages982–991, Beijing, China, July. Association for Com-putational Linguistics.

I.H. Witten and T.C. Bell. 1991. The zero-frequencyproblem: Estimating the probabilities of novelevents in adaptive text compression. IEEE Trans-actions on Information Theory, 37(4).

Yuk Wah Wong and Raymond Mooney. 2006. Learn-ing for semantic parsing with statistical machinetranslation. In Proceedings of the Human LanguageTechnology Conference of the NAACL, Main Con-ference, pages 439–446, New York City, USA, June.Association for Computational Linguistics.

Yuk Wah Wong and Raymond Mooney. 2007. Learn-ing synchronous grammars for semantic parsingwith lambda calculus. In Proceedings of the 45thAnnual Meeting of the Association of ComputationalLinguistics, pages 960–967, Prague, Czech Repub-lic, June. Association for Computational Linguis-tics.

John M. Zelle and Raymond J. Mooney. 1996. Learn-ing to parse database queries using inductive logicprogramming. In Proceedings of the Thirteenth Na-tional Conference on Artificial Intelligence - Volume2, AAAI’96, pages 1050–1055. AAAI Press.

1153

John Marvin Zelle. 1995. Using Inductive Logic Pro-gramming to Automate the Construction of NaturalLanguage Parsers. Ph.D. thesis, University of Texasat Austin, Austin, TX, USA.

1154