different semantic perspectives for question answering systems

NLP & Semantic Computing Group

Different Semantic Perspectives forHybrid Question Answering Systems

Andre FreitasUniversity of Passau

OKBQA, Jeju, 2016

http://www.slideshare.net/andrenfreitas

These slides:

Outline Multiple Perspectives of Semantic

Representation Lightweight Semantic Representation Knowledge Graph Extraction from Text Answering Queries with

Knowledge Graphs Reasoning Take-away Message

Multiple Perspectives of Semantic Representation

QA & Semantics

• Question Answering is about managing semantic representation, extraction, selection trade-offs.

• And it is about integrating multiple components in a complex approach.

•Semantic best-effort, systems tolerant to noisy, inconsistent, vague, data.

“Most semantic models have dealt with particular types of constructions, and have been carried out under very simplifying assumptions, in true lab conditions.”

“If these idealizations are removed it is not clear at all that modern semantics can give a full account of all but the simplest models/statements.”

Formal World Real World

Baroni et al. 2013

Semantics for a Complex World

Why Not RDF?•Follows a more “database-type” of

representation perspective.

•Gap towards representing text.

Choices of Semantic Representation• Logical

• Frames: verbs | nouns

• Binary relations: binary | n-ary

• Named entities

• Language Models

• Syntactic structures

• Bag-of-words

Concept-level representation

Background knowledge

Extraction complexity

Information Extraction• Logical

• Named entities

• Syntactic Structures & LMs

• Bag-of-words

• Semantic parsing

• Semantic role labeling

• Relation extraction: – closed/open

• Named entity recognition

• Syntactic/N-gram Parsing

• Indexing

Information Extraction• Logical

• Named entities

• Syntactic Structures & LMs

• Bag-of-words

• Semantic parsing

• Semantic role labeling

• Relation extraction: – closed/open

• Named entity recognition

• Syntactic/N-gram Parsing

• Indexing

Use all of them!

Representation focal points•Types of knowledge to focus at the

representation: Facts vs Definitions vs Opinions Temporality Spatiality Modality Polarity Rhetorical structures …

Lightweight Semantic Representation

Objective•Provide a lightweight knowledge representation model which: Can represent textual discourse

information.• Maximizes the capture of textual information.

Is convenient to extract from text. Is convenient to access (query and

browse).13

Lightweight Semantic Representation

Representing Texts as Contextualized Entity-Centric Linked Data Graphs, WebS 2013

Representation Assumptions• Data integration:

Named entities (instances) Abstract classes (unary predicates)

• Rich taxonomical structures.

• Context representation as a first class citizen.

• Open vocabulary.

• Word instead of sense/concept.15

Representation of Complex RelationsGeneral Electric Company, or GE , is an American multinational conglomerate corporation incorporated in Schenectady , New York

Data Integration pointsGeneral Electric Company, or GE , is an American multinational conglomerate corporation incorporated in Schenectady , New York

Named entities are lower entropy integration points Pivot

points18

Named entities are also low entropy entry points for answering queries Pivot

points19

Also abstract classes … Pivot

points20

They are also a very convenient way to represent. Pivot

points21

Taxonomy Extraction Are predicates with more complex compositional patterns

which describe sets.

Parsing complex nominals.

American multinational conglomerate corporation

On the Semantic Representation and Extraction of Complex Category Descriptors, NLDB 2014

multinational conglomerate corporation

corporation

conglomerate corporation

Pivot points

Context RepresentationGeneral Electric Company, or GE , is an American multinational conglomerate corporation incorporated in Schenectady , New York

Reification as a first class representation element

Context RepresentationGeneral Electric Company, or GE , is an American multinational conglomerate corporation incorporated in Schenectady , New York

Temporality, spatiality, modality, rhetorical relations …

Rhetorical Structures using Reification• cause:

e.g. “because scraping the bottom with a metal utensil will scratch the surface.”

• circumstance e.g. “After completing your operating system reinstallation,”

• concession e.g. “Although the hotel is situated adjacent to a beach,”

• condition e.g. “If you can break the $ 1000 dollar investment range,”

• contrast e.g. “but you can do better with 2.4ghz or 900mhz phones.”

• purpose e.g.“in order for the rear passengers to get in the vehicle.”

• …27

Open VocabularyGeneral Electric Company, or GE , is an American multinational conglomerate corporation incorporated in Schenectady , New York

Temporality, spatiality, modality, rhetorical relations …

Open Vocabulary

•Easier to extract but difficult to consume.

•We pay the price at query time.

•How to operate over a large-scale semantically heterogeneous knowledge-graphs?

Words instead of Senses•Motivation: Disambiguation is a tough

problem.

•Sense granularity can be, at many situations, arbitrary (too context dependent).

•We treat a word as a superposition of senses, almost in a “quantum mechanical sense”. 32

Coecke et al. (2010): Category theory and Lambek calculus.

Revisited RDF (for Representing Texts)• Data Model Types: Instance, Class, Property…

• RDFS: Taxonomic representation.

• Reification for contextual relations (subordinations).

• Blank nodes for n-ary relations.

• Triple.

• Labels over URIs.34

Abstract Meaning Representations – AMR,Maximal Use of PropBank Frame Files

Alternative Representations

Distributional Semantics

Distributional Semantic Models Semantic Model with low acquisition effort

(automatically built from text)

Simplification of the representation

Enables the construction of comprehensive commonsense/semantic KBs

What is the cost?

Some level of noise(semantic best-effort)

Limited semantic model37

Distributional Semantics as Commonsense Knowledge

Commonsense is here

leashSemantic Approximation is

I find it rather odd that people are already trying to tie the Commission's hands in relation to the proposal for a directive, while at the same calling on it to present a Green Paper on the current situation with regard to optional and supplementary health insurance schemes.

I find it a little strange to now obliging the Commission to a motion for a resolution and to ask him at the same time to draw up a Green Paper on the current state of voluntary insurance and supplementary sickness insurance.

Beyond Single Word Vector Models: Compositionality

Compositional Semantics Can we extend DS to account for the

meaning of phrases and sentences? Compositionality: The meaning of a

complex expression is a function of the meaning of its constituent parts.

Compositional Semantics

Words in which the meaning is directly determined by their distributional behaviour (e.g., nouns).

Words that act as functions transforming the distributional profile of other words (e.g., verbs, adjectives, …).

Compositional-Distributional Semantics

NLP & Semantic Computing Group Recursive Neural Networks for Structure Prediction

New Model: Recursive Neural Tensor Network•Goal: Function that composes two vectors.•More expressive than any other RNN so far.

44 Socher et al.

Socher et al.

Compositional-distributional model for Categories

Embedding Knowledge Graphs

The vector space is segmented48

Dimensional reduction mechanism!

A Distributional Structured Semantic Space for Querying RDF Graph Data, IJSC 2012

Compositional-distributional model for paraphrases

A Compositional-Distributional Semantic Model for Searching Complex Entity Categories, *SEM (2016)

Knowledge Graph Extraction from Text

Graphene

Graph Extraction Pipeline

Text Transformati

N-ary Relation Extractio

nText Simplificatio

n GraphSerializatio

Taxonomy

Extraction

Storage

RST Classificati

ML-based

Rule-based

ML-based

Minimalistic Text Transformations

Text Transformati

nText Simplificatio

n GraphSerializatio

Taxonomy

Extraction

Storage

RST Classificati

ML-based

Rule-based

ML-based

Minimalistic Text Transformations

•Co-reference Resolution Pronominal co-references.

•Passive We have been approached by the investment

banker. The investment banker approached us.

•Genitive modifier Malaysia's crude palm oil output is estimated

to have risen. The crude palm oil output of Malasia is

estimated to have risen.54

Text Simplification

Text Transformati

nText Simplificatio

n GraphSerializatio

Taxonomy

Extraction

Storage

RST Classificati

ML-based

Rule-based

ML-based

Text Simplification for KG Extraction“Defeating Republican nominee Mitt Romney, Obama, who was the first African American to hold the office, was reelected president in November 2012.”

relations are spread across clauses relations are presented in non-canonical form

Text Simplification for KG Extraction

•Insertion of a text simplification step

Obama was reelected president in November 2012.

Obama was the first African American to hold the office.

Obama was defeating Mitt Romney. Mitt Romney was Republican nominee.

Syntax-driven sentence simplification approachTask:

• Reduce the linguistic complexity of a text while retaining the original information/meaning using a set of syntax-based rewrite operations (deletion, insertion, reordering, sentence splitting).

Idea:• Simplify a sentence by separating out components

that supply only secondary information into simpler stand-alone context sentences, thus yielding one or more reduced core sentences.

Approach• Linguistic analysis of sentences from the English Wikipedia to identify constructs which provide only secondary information:

• non-restrictive relative clauses• non-restrictive and restrictive appositive phrases• participial phrases offset by commas• adjective and adverb phrases delimited by punctuation• particular prepositional phrases• lead noun phrases• intra-sentential attributions• parentheticals• conjoined clauses with specific features• particular punctuation

•Rule-based simplification rules.

Improving Relation Extraction by Syntax-based Sentence Simplification (2016)

N-ary Relation Extraction

Text Transformati

nText Simplificatio

n GraphSerializatio

Taxonomy

Extraction

Storage

RST Classificati

Rule-based

ML-based

OpenIE, University of Washington

Taxonomy Extraction

Text Transformati

nText Simplificatio

n GraphSerializatio

Taxonomy

Extraction

Storage

RST Classificati

Rule-based

ML-based

Representation and Extraction of Complex Category Descriptors, NLDB 2014

RST Classification

Text Transformati

nText Simplificatio

n GraphSerializatio

Taxonomy

Extraction

Storage

RST Classificati

Rule-based

ML-based

Rhetorical Structure Extraction

TEXT-LEVEL RST-STYLE DISCOURSE PARSER (Feng and Hirst, 2012)

Structure classification

Relation classification

Answering Queries with Knowledge Graphs

Now our graph supports semantic approximations as a first-class operation

Approach Overview

Query Planner

Ƭ-Space(embedding

graphs)

WikipediaCommonsense

knowledge

Explicit Semantic Analysis

Core semantic approximation &

composition operations

Query AnalysisQuery Query Features

Query Plan

Core Principles Minimize the impact of Ambiguity, Vagueness,

Synonymy. Address the simplest matchings first (semantic

pivoting).

Semantic Relatedness as a primitive operation.

Distributional semantics models as commonsense knowledge representation.

Lightweight syntactic constraints.67

•Step 2: Query NER Rules-based: POS Tag + IDF

Who is the daughter of Bill Clinton married to?(PROBABLY AN INSTANCE)

Query Pre-Processing (Question Analysis)

•Step 3: Determine answer type Rules-based.

Who is the daughter of Bill Clinton married to? (PERSON)

•Transform natural language queries into a pseudo-logical form.

“Who is the daughter of Bill Clinton married to?”

Bill Clinton

daughter married to

(INSTANCE)

Person

ANSWER TYPE

QUESTION FOCUS71

• Step 5: Determine the query pattern Rules based.• Remove stop words.• Merge words into entities.• Reorder structure from core entity position.

Bill Clinton

daughter married to

(INSTANCE)

Person

(PREDICATE) (PREDICATE) Query Features

• Map query features into a query plan.• A query plan contains a sequence of:

Search operations. Navigation operations.

Query Planning

(INSTANCE) (PREDICATE) (PREDICATE) Query Features

(1) INSTANCE SEARCH (Bill Clinton) (2) DISAMBIGUATE ENTITY TYPE (3) GENERATE ENTITY FACETS (4) p1 <- SEARCH RELATED PREDICATE (Bill Clintion, daughter) (5) e1 <- GET ASSOCIATED ENTITIES (Bill Clintion, p1) (6) p2 <- SEARCH RELATED PREDICATE (e1, married to) (7) e2 <- GET ASSOCIATED ENTITIES (e1, p2) (8) POST PROCESS (Bill Clintion, e1, p1, e2, p2)

Query Plan

Core Entity SearchBill Clinton

daughter married to Person

:Bill_Clinton

Query:

Entity search

Distributional Semantic SearchBill Clinton

:Bill_Clinton

Query:

:Chelsea_Clinton

:child

:Baptists:religion

:Yale_Law_School:almaMater...

(PIVOT ENTITY)

(ASSOCIATED TRIPLES)

:Bill_Clinton

Query:

:Chelsea_Clinton

:child

:Baptists:religion

:Yale_Law_School:almaMater...

sem_rel(daughter,child)=0.054

sem_rel(daughter,child)=0.004

sem_rel(daughter,alma mater)=0.001

Which properties are semantically related to ‘daughter’?

:Bill_Clinton

Query:

:Chelsea_Clinton

:child

:Bill_Clinton

Query:

:Chelsea_Clinton

:child

(PIVOT ENTITY)

:Bill_Clinton

Query:

:Chelsea_Clinton

:child:Mark_Mezvinsky

:spouse

Note the lazy disambiguation

What is the highest mountain?Second Query Example

(CLASS) (OPERATOR) Query Features

mountain - highest PODS

Entity SearchMountain highest

:Mountain

Query:

:typeOf

(PIVOT ENTITY)

Extensional ExpansionMountain highest

:Mountain

Query:

:Everest:typeOf

(PIVOT ENTITY)

:K2:typeOf

Distributional Semantic MatchingMountain highest

:Mountain

Query:

:Everest:typeOf

(PIVOT ENTITY)

:K2:typeOf

:elevation

:location...:deathPlaceOf

Get all numerical valuesMountain highest

:Mountain

Query:

:Everest:typeOf

(PIVOT ENTITY)

:K2:typeOf

:elevation

8848 m

8611 m

Apply operator functional definitionMountain highest

:Mountain

Query:

:Everest:typeOf

(PIVOT ENTITY)

:K2:typeOf

:elevation

8848 m

8611 m

SORTTOP_MOST

Results

StarGraph•Open source NoSQL platform for building

and interacting with large and sparse knowledge graphs.

•Semantic approximation as a built-in operation.

•Scalable query execution performance.

Heuristics for the selection of the semantic pivot is critical!•Discussed here just superficially:

Information-theoretical justification.

How hard is the Query? Measuring the Semantic Complexity of Schema-Agnostic Queries, IWCS (2015).

Schema-agnositc queries over large-schema databases: a distributional semantics approach, PhD Thesis (2015).

On the Semantic Mapping of Schema-agnostic Queries: A Preliminary Study, NLIWoD (2015).

Indra Multilingual platform for experimentation

with different word vector models.

"Indra's net" is the net of the Vedic god Indra, whose net hangs over his palace on Mount Meru, the axis mundi of Hindu cosmology and Hindu mythology. Indra's net has a multifaceted jewel at each vertex, and each jewel is reflected in all of the other jewels.

In the Avatamsaka Sutra, the image of "Indra's net" is used to describe the interconnectedness of the universe.

Bridging Structured & Unstructured Data•NER + Text + Passage Retrieval Ranking

Simple and powerful QA basis.

•Lazy disambiguation.

Treo Answers Jeopardy Queries (Video)

Ranking Candidate Answers•But what if there are multiple candidate answers!

Q: Who was Queen Victoria’s second son?•Answer Type: Person

• Passage:The Marie biscuit is named after Marie Alexandrovna, the daughter of Czar Alexander II of Russia and wife of Alfred, the second son of Queen Victoria and Prince Albert

96Dan Jurafky’s slides

Ranking Candidate Answers•But what if there are multiple candidate answers!

Q: Who was Queen Victoria’s second son?•Answer Type: Person

• Passage:The Marie biscuit is named after Marie Alexandrovna, the daughter of Czar Alexander II of Russia and wife of Alfred, the second son of Queen Victoria and Prince Albert

97Dan Jurafky’s slides

Feature Engineering

The Marie biscuit is named after Marie Alexandrovna, the daughter of Czar Alexander II of Russia and wife of Alfred, the second son of Queen Victoria and Prince Albert

followed by a ‘,’ followed by an apposition

Who was Queen Victoria’s second son?

contains an entity in the query

has a four-word overlap

type = PERSON

matches AnswerType

Propositionalisation

e0 followedBy(,) followedByAppositionContainingQueryEntities() answer …

Alfred true true true… … …

passage

entity (e0)

entity (en)

The Marie biscuit is named after Marie Alexandrovna, the daughter of Czar Alexander II of Russia and wife of Alfred, the second son of Queen Victoria and Prince Albert

answer

Reasoning for Text Entailment

Beyond Word Vector Models

give birth mother

Distributional semantics can give us a hint about the concepts’ semantic proximity...

...but it still can’t tell us what exactly the relationship between them is

give birth

mother???

Beyond Word Vector Models

give birth

mother???

give birth

mother???

Beyond Word Vector Models: Intensional Reasoning

Representing structured intensional-level knowledge.

Creation of an intensional-level reasoning model.

Commonsense Reasoning

Selective (focussed) reasoning - Selecting the relevant facts in the

context of the inference

Reducing the search space.Scalability

Extended WordNet (XWN)

NLP & Semantic Computing Group Commonsense Data (ConceptNet)

http://conceptnet5.media.mit.edu/

Distributional semantic relatedness as a Selectivity Heuristics

Distributional heuristics

target

source answer

target

source answer

target

source answer

John Smith

EngineerInstance-level

occupation

Does John Smith have a degree?

A Distributional Semantics Approach for Selective Reasoning on Commonsense Graph Knowledge Bases, NLDB (2015).

Bringing it into the Real World

Semeval 2017

Take-away Message• Choosing the sweet-spot in terms of semantic

representation is critical for the construction of robust QA systems.

Work at a word-based representation instead of a sense representation.

Text simplification/clausal disembedding critical for relation extraction.

Need for a standardized semantic representation for relations extracted from texts.

Take-away Message•Text entailment:

Intensional-level reasoning. Natural logic. Distributional semantics.

•Distributional semantics: Robust, language-agnostic semantic

matching. Selective reasoning over commonsense

different semantic perspectives for question answering systems

Science

a framework for web based language independent semantic...

amuse: multilingual semantic parsing for question...

semantic web - eu perspectives

learning multilingual semantic parsers for question...

towards answering geography questions in gaokao: a hybrid...

using semantic-based approach to manage perspectives of...

syntactic and semantic models and algorithms in question...

answering visuo-semantic queries with imgpedia · answering...

question answering benchmark and semantic...

long-answer question answering and rhetorical-semantic...

answering questions with complex semantic constraints on...

image question answering: a visual semantic embedding...

question-answering by a semantic network of parallel...

uncc biomedical semantic question answering systems. … ·...

question answering as global reasoning over semantic...

expressive query answering for semantic wikis (20min)

question-answering by a semantic network of parallel...

rewriting-based query answering for semantic data...

perspectives on design principles – semantic invariants...

semantic interpretation of user query for question answering...