Selecting Contextual Peripheral Information for Answer Presentation: TheNeed for Pragmatic Models

Rivindu Perera, Parma NandSchool of Computer and Mathematical Sciences

Auckland University of TechnologyAuckland, New Zealand

{rivindu.perera, parma.nand}@aut.ac.nz

Abstract

This paper explores the possibility of pre-senting additional contextual information as amethod of answer presentation Question An-swering. In particular the paper discusses theresult of employing Bag of Words (BoW) andBag of Concepts (BoC) models to retrievecontextual information from a Linked Dataresource, DBpedia. DBpedia provides struc-tured information on wide variety of entitiesin the form of triples. We utilize the QALDquestion sets consisting of a 100 instances inthe training set and another 100 in the testingset. The questions are categorized into singleentity and multiple entity questions based onthe number of entities mentioned in the ques-tion. The results show that both BoW (syn-tactic models) and BoC (semantic models) arenot capable enough to select contextual infor-mation for answer presentation. The resultsfurther reveals that pragmatic aspects, in par-ticular, pragmatic intent and pragmatic infer-ence play a crucial role in contextual informa-tion selection in the answer presentation.

1 Introduction

Answer Presentation is the final step in QuestionAnswering (QA) which focuses on generating an an-swer which closely resemble with a human providedanswer (Perera, 2012b; Perera, 2012a; Perera andNand, 2014a). There is also a requirement to asso-ciate the answer with additional contextual informa-tion when presenting the answer.

This paper focus of exploring methods to extractadditional contextual information to present with the

extracted factoid answer. We provide a classifica-tion if questions based on the type of the answer re-quired and the number of entities that mentioned inthe questions. The question classification is illus-trated in Fig. 1. Firstly, question can be categorizedbased on the information need where questions mayrequire a definition as the answer or a factoid an-swer which is an information unit (Perera, 2012a;Perera and Nand, 2014a). The definitional questionsneed definitions which include both direct and re-lated background information and there is no need tofurther expand the answer with contextual informa-tion. So far the way factoid questions presentationinvolved only the answer itself without contextualinformation. Recently, Mendes and Coheur (2013)argued that even factoid questions need to presentadditional information. An advantage of presentingcontextual information is that answer is justified bythe information provided, so that users can concludethat the answer that is acquired by the system is onethat they are searching for.

The rest of the paper is structured as follows. Sec-tion 2 explores BoW and BoC models to rank con-textual information. Section 3 focuses on presentingthe experimental framework. Section 5 presents in-formation on related work and we conclude the pa-per in Section 6.

2 Content selection using weighted triples

This section presents models to rank triples focusingon open domain questions as communicative goals.Open domain questions require knowledge from dif-ferent domains to be aggregated which making itmore challenging compared to simply generating a

PACLIC 29

19729th Pacific Asia Conference on Language, Information and Computation: Posters, pages 197 - 205

Shanghai, China, October 30 - November 1, 2015Copyright 2015 by Rivindu Perera and Parma Nand

Figure 1: Classification of common question types

content for given single theme topic. Our objectiveis to select a set of triples which can be used to gen-erate a more informative answer for a given ques-tion.

We investigate the problem from two perspec-tives; as a Bag of Words (BoW) and as a Bag ofConcepts (BoC). In the following sections, we dis-cuss the strategy used for ranking and the details onthe supporting utilities including domain corpus, ref-erence corpus, triple retrieval, and threshold basedselection.

The high level design of the framework used toexperiment BoW and BoC is shown in the Fig. 2.The model utilizes two corpora (domain and refer-ence) and selectively used based on the requirement.The domain corpus is constructed using search snip-pets collected from the web by using informationfrom the question and answer as query terms. Thereference corpus represents knowledge about gen-eral domain. The model also has utility functionsto retrieve triples using SPARQL queries, filter theduplicates, and to perform basic verbalization.

2.1 The problem as a Bag of Words

We utilized token similarity, Term Frequency - In-verse Document Frequency (TF-IDF), and Resid-ual Inverse Document Frequency (RIDF) in twoflavours which are widely used in information re-trieval tasks. The following sections describes thesemodels in detail.

2.1.1 Token similarityToken similarity ranks the triples based on the ap-

pearance of the terms in triple and the question be-ing considered. In particular we employ the cosine

similarity (1) to calculate the similarity between thetokenized and stopwords removed question/answerand the triple.

simcosine(−→Q,−→T ) =

−→Q ·−→T

|−→Q ||−→T |

=Σni=1QiTi√

Σni=1Q

2i

√Σni=1T

2i

(1)

Here, Q and T represent the question and the triplerespectively.

2.1.2 Term Frequency – Inverse DocumentFrequency (TF-IDF)

The TF-IDF (2) is used to rank term (t) from thequestion and answer present in the triple (T). A tripleis then associated with a weight which is the sum ofthe weights assigned to the triple terms.

TF − IDF (Q,T ) =∑i∈Q,T

tfi.idfi

=∑i∈Q,T

tfi.log2N

dfi

(2)

Where tf represents the term frequency, N standsfor number of documents in the collection and dfis the number of documents with the correspondingterm. Q represents the question, however in our ex-periment we tested the possibility of utilizing a do-main corpus instead of the original question or thequestion with the answer.

PACLIC 29

198

Figure 2: The schematic representation of the content selection framework

2.1.3 Okapi BM25Okapi ranking is an extension to the TF-IDF that

is based on the probabilistic retrieval framework.The Okapi ranking function can be defined as fol-

lows:

Okapi(Q,T ) =∑i∈Q,T

[log

N

dfi

].

(k1 + 1) tfi,T

k1

((1− b) + b

(LTLave

))+ tfi,T

.(k3 + 1) tfi,Qk3 + tfi,Q

(3)

Where, LT and Lave represent the length of thetriple and average of length of a triple respectively.The Okapi also uses set of parameters where b isusually set to 0.75 and k1 and k3 range between 1.2and 2.0. The k1 and k3 can be determined throughoptimization or can be set to range within 1.2 and 2.0in the absence of development data (Manning andSchutze, 1999).

2.1.4 Residual Inverse Document Frequency(RIDF)

The idea behind the RIDF is to find content wordsbased on actual IDF and predicted IDF. The widelyused methods to IDF prediction is Poisson and Kmixture. However, K mixture tends to fit very wellwith content terms. On the other hand, Poisson de-viates from the IDF remarkably and provides non-content words. Given term frequencies in triple col-lection, predicted IDF can be used to measure theRIDF for a triple as follows:

RIDF =∑i∈T

(idfi − idfi

)=∑i∈T

(idfi − log

1

1− P (0;λi)

) (4)

Where λi represents the average number of oc-currences of term and P (0;λi) represents the Pois-son prediction of df where term will not be found ina document. Therefore, 1 − P (0;λi) can be inter-preted as finding at least one term and can be mea-sured using:

P (k;λi) = e−λiλkik!

(5)

Based on the same RIDF concept, we can mod-erate this to work with term distribution models thatfits well with actual df such as K mixture. The defi-nition of the K-mixture is given below.

P (k;λi) = (1− α)δk,0 +α

β + 1

(β

β + 1

)k(6)

In K-mixture based RIDF we interpreted the devi-ation from predicated df to mark the term as a non-content term.

2.2 The problem as a Bag of Concepts

This section explains two BoC models which canrank triples utilizing the semantic representationof the triple collection. In particular, we employ

PACLIC 29

199

two widely accepted BoC models; Latent Seman-tic Analysis (LSA) and adoption of Log LikelihoodDistance (LLD) using two corpora. The followingsections describe them in detail.

2.2.1 Latent Semantic Analysis

This method analysed how triples in the collec-tion can be ranked concept-wise and retrieved re-lated to the question and answer where triples arerepresented in a semantic space. Such a ranking canexpose the original semantic structure of the spaceand its dimensions (Manning and Schutze, 1999).In particular, we employed the Latent Semantic In-dexing (LSI) for each collection of triples associatedwith the question.

2.2.2 Corpus based Log Likelihood Distance(LLD)

The idea behind the implementation of thismethod is to identify domain specific concepts(compared to the general concepts) and rank tripleswhich contain such concepts. For this we employeda domain corpus (see Section 2.3) and a general ref-erence corpus (see Section 2.4). The model extractsconcepts which are related to the domain on the ba-sis of their frequency in domain corpus and generalreference corpus. A term that is more frequentlyseen in a domain corpus compared to the general ref-erence corpus implies that the term is a concept thatis used in the domain being considered (Perera andNand, 2014b; Perera and Nand, 2014c). We utilizedthe log likelihood distance (He et al., 2006; Gelbukhet al., 2010) to measure the importance as mentionedbelow:

Wt = 2×((

fdomt × log

(fdomt

f expdomt

))+(

f reft × log

(f reft

f expreft

)))(7)

where, fdomt and f reft represent frequency of term(t) in domain corpus and reference corpus respec-tively. Expected frequency of a term (t) in domain(f expdomt ) and reference corpora (f expreft ) werecalculated as follows:

f expdomt = sdom ×

(fdomt + f reft

sdom + sref

)(8)

f expreft = sref ×

(fdomt + f reft

sdom + sref

)(9)

where, sdom and sref represent total number oftokens in domain corpus and reference corpus re-spectively. Next, we can calculate the weight of atriple (〈subject, predicate, object〉) by summing upthe weight assigned to each term of the triple

2.3 Domain CorpusThe domain corpus is a collection of text related tothe domain of the question being considered. How-ever, finding a corpus which belongs to the same do-main as the question is challenge on its own. Toovercome this, we have utilized an unsupervised do-main corpus creation based on a web snippet extrac-tion. The input to this process is a set of extractedkey phrases from a question and its answers.

2.4 Reference CorpusThe reference corpus is an additional resource uti-lized for the LLD based contextual information se-lection. We used the British National Corpus (BNC)as the reference corpus. The selection is influencedby the language used in the DBpedia, British En-glish. However, what is important for the LLD cal-culation is a term frequency matrix. We have firstperformed stopword filtering on the BNC and thisoperation reduced the original size of BNC (100 mil-lion words) to 52.3 million words. Next, the termfrequency matrix is built using a unigram analysis.

2.5 Triple retrievalThe model employs the Jena RDF framework forthe triple retrieval. We have implemented a Java li-brary to query and automatically download neces-sary RDF files from DBpedia.

2.6 Threshold based selectionAfter associating each triple with a calculatedweight, we then need to limit the selection based ona particular cut-off point as the threshold (θ). Due

PACLIC 29

200

Table 1: Dataset statistics. Invalid questions are thosethat are already marked by dataset providers as invalidand questions where for which triples cannot be retrievedfrom DBpedia

Training Test

All questions 100 100Invalid questions 5 10Single entity questions 47 42Multiple entity questions 48 48

to the absence of knowledge to measure the θ at thisstage, it is considered as a factor that needs to betuned based on experiments. Further discussion onselecting the θ can be found in Section 4.

3 Experimental framework

3.1 DatasetWe used the QALD-2 training and test datasets, butremoved questions which marked as “out of scope”by dataset providers and those for which DBpediatriples did not exist. Table 1 provides the statistics ofthe dataset, including the distribution of questions intwo different question categories, single entity andmultiple entity questions.

We have also built a gold triple collection for eachquestion for the purpose of evaluation. These goldtriples were selected by analysing community pro-vided answers for the questions in our dataset. Table1 shows the statistics for both training and testingdatasets.

3.2 Results and discussionThe evaluation is carried out using gold triples as de-scribed in Section 3.1. The definitions of precision(P), recall (R) and F-score (F*) are given below:

P =|triplesselected ∩ triplesgold|

|triplesselected|(10)

R =|triplesselected ∩ triplesgold|

|triplesgold|(11)

F ∗ =2PR

P +R(12)

The threshold (θ) (measure as a percentage fromthe total triple collection) value for the ranked tripleswas experimentally chosen to using the training

Table 2: Statistics related to the gold triple percentagein total triple collection in training dataset. The µ rep-resents the mean percentage of gold triples included inthe total collection. The σ shows the standard deviation.The Max% and Min% represent maximum and minimumpercentage of gold triples from the total collection respec-tively.

µ σ Max% Min%

Single entitytype

68.89 4.28 78.79 63.58

Multiple entitytype

30.43 3.88 37.06 22.93

dataset. This threshold value was then used to selecttriples which were relevant for the testing dataset.The value was experimentally determined by usinga combination of precision and recall value formthe training data. For an accurate model, the pre-cision is expected to remain constant until it startsselecting the irrelevant triples after which the preci-sion will gradually decrease. Correspondingly, therecall value will increase until the threshold pointafter which the model will start selecting irrelevanttriples, which will start pushing the recall valuedown. Hence the optimum θ value will be the pointat the maximum point for both recall and precisionwhich is the maximum score.

Using the θ identified from training set, we canthen test the model using testing dataset. When mea-suring the θ based on the training dataset it is alsoimportant to measure the proportion of gold triplescompared to the number of total triples. A set ofstatistics related to this calculation is shown in Table2.

65 70 75 80 85 90 95 100

0.65

0.7

0.75

Threshold (θ)

F-Sc

ore

CosineTFIDFOkapiRIDF-PoissonRIDF-Kmixture

Figure 3: F-score gained by Bag of Words models plottedagainst threshold for questions with single entity type

According to statistics shown in Table 2 it is clearthat the mean percentage of gold triples percentages

PACLIC 29

201

30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105

0.65

0.7

0.75

0.8

Threshold (θ)

F-Sc

ore

LSILLD

Figure 4: F-score gained by Bag of Concepts modelsplotted against threshold for questions with single entitytype

20 30 40 50 60 70 80 90 100

0.4

0.45

0.5

Threshold (θ)

F-Sc

ore

CosineTFIDFRIDF-Poisson

Figure 5: F-score gained by Bag of Words models plottedagainst threshold for questions with multiple entity types.Okapi and K-mixture based ranking methods completelyfailed to identify relevant triples.

30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105

0.45

0.46

0.47

0.48

Threshold (θ)

F-Sc

ore

LLD

Figure 6: F-score gained by Bag of Concepts modelsplotted against threshold for questions with multiple en-tity types. The LSI method failed completely in identify-ing relevant triples.

are 68.89% for single entity types and 30.43% fromthe multiple entity types. Furthermore, the maxi-mum and minimum percentages are also near val-ues to the receptive mean values. This encompassesthat there is a possibility to find a threshold value forboth single and multiple entity types questions. Fig.3 depicts the evaluation performed on the single en-tity question category in training dataset using fiveBoW models under investigation. The results showthat the maximum F-score obtained when θ is set to100%. This shows that these models unable to accu-rately differentiate between the relevant and relevanttriples. The BoW models consider only the words

Table 3: Performance of LLD on single entity type ques-tion test dataset with 78% threshold

Precision Recall F-score

LLD on TestDataset (SingleEntity)

0.72 0.84 0.76

as features and therefore every entity is assigned thesame importance.

The corresponding evaluation performed on thesingle entity question category in training dataset us-ing BoC models is shown in Fig. 4. Latent Seman-tic Indexing (LSI) has performed poorly and has notmanaged to identify a global maximum. However,the Log Likelihood Distance (LLD) has identified aglobal maximum with a θ value of 78%. Further-more, it has also shown the expected behaviour withan increase in θ value. When this threshold valuewas used to extract triples from the test dataset theresults were encouraging. Table 3 shows that LLDhas achieved F-score of 0.76 for the testing datasetwith a 0.72 precision value. The LLD model out-performs the other models in this context mainlybecause it also incorporates the domain knowledge(provided through a domain corpus as explained inSection 2.2.2).

Fig. 5 and Fig. 6 depict the evaluation performedon the multiple entity question category from train-ing dataset, for both Bag of Words and Bag of Con-cepts models. RIDF-Kmixture and Okapi have com-pletely failed without any success in identifying therelevant triples. The Cosine, TF-IDF, and RIDF-Poisson have also not identified the optimum thresh-old (see Fig. 5). From the BoC models, the LSImethod has also failed entirely. The LLD mode hasidentified a local maximum at θ = 48, however themodel has not behaved as expected. Furthermore,the global maximum identified at θ = 100 impliesthat the model can identify all relevant triples onlywhen the total triple collection is retrieved. Thisconfirms that although a Bag of Concepts modelsuch as LLD performed well in the single entitytype questions, none of the models performed wellin contextual information selection for multiple en-tity type questions.

Analysis of the erroneous triples for this experi-

PACLIC 29

202

ment revealed that for multiple entity type questionsit is important to identify the intent entity from thequestion. The information from the intent of thequestion can be used to factor in a weight correctionfor the triples. This leads to the study the Bag ofNarrative (Cambria and White, 2014) model whichis essentially based on pragmatic aspects of the lan-guage. Section 4 discusses this aspect in detail.

4 Pragmatic aspect in contextualinformation selection

We introduce two pragmatic based concepts thatneed to be studied in contextual information selec-tion approaches, derived from psycholinguistics.

• Pragmatic intent (Byram and Hu, 2013) of aquestion in the perspective of contextual infor-mation and,

• Pragmatic inferences (Byram and Hu, 2013;Tomlinson and Bott, 2013) that can be drawnbased on already known information.

4.1 Pragmatic Intent

The pragmatic intent of a question in our problemcan be defined as the entity that a user is actually in-tending to know more about. This concept deviatesfrom the two early approaches in query classifica-tion; Broder’s taxonomy (Broder, 2002) (classifyingqueries as informational, transactional, and naviga-tional) and question typologies (Li and Roth, 2006)(determining the answer type for a question).

Consider the examples given in Table 4, where ineach question multiple entities are mentioned. Theentities are numbered and pragmatic intent is tagged(with code :i).

In Q1, Marc Mezvinsky is the pragmatic intentof the question which is also the expected answer.The same rule applies for Q2, Q3, and Q4. How-ever, in Q5 and Q6 the pragmatic intent is a part ofthe question ([MI6] and [Natalie Portman]), but notthe answer. This variation makes it difficult to iden-tify the pragmatic intent of a question compared tothe question target identification. When presentingcontextual information to the user, the informationrelated to the pragmatic intent together with infor-mation that is shared by pragmatic intent and otherentities need to be given the priority.

Table 5: Example question to illustrate the pragmatic in-ference used in the information elimination

Q7 Which river does the Brooklyn Bridgein New York cross?

Answer East RiverTriple 〈East River, flow through, New York〉

4.2 Pragmatic inferenceThe pragmatic inference is the interpreting informa-tion based on the context that it operates on. For ex-ample, a storyteller will not mention every incidentor fact that happened in a narrative, thus some partsmay be left for the reader to interpret using com-mon sense knowledge, open domain knowledge, andknowledge that is already mentioned in the narrative.Applying this well-established psycholinguistic the-ory in our approach, we noticed several scenarioswhere we can improve the contextual informationby eliminating information that can be pragmaticallyinferred and prioritizing information that needs forthe context.

Consider the question (Q7), answer and the tripleprovided in Table 5. Using the question and its an-swer we can infer the following two facts encodedin the triples.

• F1: Brooklyn Bridge, located in, New York

• F2: Brooklyn Bridge, crosses, East River

As humans, we can infer that if Brooklyn Bridgeis located in New York and if it crosses the EastRiver, then East river must flow through New York,hence it is co-located. Therefore, the triple in Table5 becomes unimportant for the context because it isalready inferred by F1 and F2 which can be derivedfrom question and its answer.

The pragmatic inference can also be used to pri-oritize the information using semantic relations thatentities contain. For example consider the two sce-narios illustrated in Table 6 where important contex-tual information can be inferred based on the seman-tic relationship of the pragmatic intent and entities.

In Q8 the relation between the entity “VirginGroup” and its co-founders (“Richard Branson” and“Nik Powell”) is in the form of launching a new or-ganization. This makes the information such as cur-

PACLIC 29

203

Table 4: Example questions to illustrate the pragmatic intent variation in different questions. Entities are numberedand intent is marked with code i

# Question Answer

Q1 Who is the daughter of [Bill Clinton]1 married to? [Marc Mezvinsky]i2Q2 Which river does the [Brooklyn Bridge]1 in [New York]2 cross? [East river]i3Q3 Which bridge located in [New York]1 is opened on 19th March 1945? [Brooklyn Bridge]i2Q4 What is the highest place in [Karakoram]1? [K2]i2Q5 In which [UK]1 city is the headquarters of the [MI6]i2? [London]3Q6 Was [Natalie Portman]i1 born in the [United States]2? No

Table 6: Examples illustrate the use of pragmatic infer-ence in information prioritization

# Question Answer

Q8 Who is the founderof [Virgin Group]1?

[Richard Branson]i2and [Nik Powell]i3

Q9 How often was[Michael Jordan]i1divorced?

2

rent positions which are held by its co-founders tobe prioritized over other information which is notstrongly related to the context of the question. Next,Q9 is related to Michael Jordan’s marriage. Whenretrieving contextual information for this questionthe basic information about his wives such as per-sonal names becomes more important for the contextof the question.

5 Related work

Benamara and Dizier (2003) present the coopera-tive question answering approach which generatesnatural language responses for given questions. Inessence, a cooperative QA system moves a few stepsfurther from ordinary question answering systemsby providing an explanation of the answer. How-ever, this research lacks the investigation to the in-formation needs of different questions and the pro-cess of utilizing cohesive information for the expla-nation, without redundant text.

Bosma (2005) incorporates the summarization asa method of presenting additional information in QAsystems. He coins the term, an intensive answerto refer to the answer generated from the system.The process of generating an intensive answer is

based on the summarization using rhetorical struc-tures. Several other summarization based methodsfor QA such as Demner-Fushman and Lin (2006)and Yu et al. (2007) also exist with slightly vary-ing techniques.

Vargas-Vera and Motta (Vargas-Vera and Motta,2004) present an ontology based QA system,AQUA. Although AQUA is primarily aimed at ex-tracting answers from a given ontology, it also con-tributes to answer presentation by providing an en-riched answer. The AQUA system extracts ontologyconcepts from the entities mentioned in the questionand present those concepts in aggregated natural lan-guage.

6 Conclusion

This study has examined the role and effectivenessof syntactic and semantic models in contextual in-formation selection for answer presentation. The re-sults showed that the semantic models (e.g., LLD)performed the best for single entity based questions,however the performance dropped for multiple en-tity questions. An analysis of the multi-entity ques-tions showed that in order to improve performancethere is a need to integrate pragmatic aspects intothe ranking framework. Further work needs to bedone to establish a framework to model pragmaticaspects in contextual information selection. We havealready launched the development of the pragmaticframework as discussed in Section 4. Future workwill introduce diverse methods of answer presenta-tion in question answering system utilizing contex-tual information (Perera and Nand, 2015b; Perera etal., 2015; Perera and Nand, 2015a; Perera and Nand,2015c).

PACLIC 29

204

ReferencesFarah Benamara and Patrick Saint Dizier. 2003. Dy-

namic generation of cooperative natural language re-sponses in webcoop. In 9th European Workshop onNatural Language Generation, Budapest, Hungary.ACL.

Wauter Bosma. 2005. Extending answers using dis-course structure. In Recent Advances in Natural Lan-guage Processing, Borovets, Bulgaria. Association forComputational Linguistics.

Andrei Broder. 2002. A Taxonomy of Web Search. ACMSIGIR Forum, 36(2):3–10.

Michael Byram and Adelheid Hu. 2013. RoutledgeEncyclopedia of Language Teaching and Learning.Routledge, Taylor & Francis Group, London, UK.

Erik Cambria and Bebo White. 2014. Jumping NLPCurves: A Review of Natural Language ProcessingResearch. IEEE Computational Intelligence Maga-zine, 9(2):48–57, May.

Dina Demner-Fushman and Jimmy Lin. 2006. Answerextraction, semantic clustering, and extractive summa-rization for clinical question answering. In Proceed-ings of the 21st International Conference on Compu-tational Linguistics and the 44th annual meeting of theACL - ACL ’06, pages 841–848, Morristown, NJ, USA,July. Association for Computational Linguistics.

Alexander Gelbukh, Grigori Sidorov, and Liliana Lavin-Villa, Eduardo Chanona-Hernandez. 2010. AutomaticTerm Extraction Using Log-Likelihood Based Com-parison with General Reference Corpus. In NaturalLanguage Processing and Information Systems, pages248–255. Springer Berlin Heidelberg.

Tingting He, Xiaopeng Zhang, and Ye Xinghuo. 2006.An Approach to Automatically Constructing DomainOntology. In 20th Pacific Asia Conference on Lan-guage, Information and Computation, pages 150–157,Wuhan. Association for Computational Linguistics.

Xin Li and Dan Roth. 2006. Learning question classi-fiers: the role of semantic information. Natural Lan-guage Engineering, 12(3):229.

Christopher D. Manning and Hinrich Schutze. 1999.Foundations of statistical natural language process-ing. Massachusetts Institute of Technology.

Ana Christina Mendes and Luisa Coheur. 2013. Whenthe answer comes into question in question-answering:survey and open issues. Natural Language Engineer-ing, 19(01):1–32, January.

Rivindu Perera and Parma Nand. 2014a. Interaction his-tory based answer formulation for question answering.In International Conference on Knowledge Engineer-ing and Semantic Web (KESW), pages 128–139.

Rivindu Perera and Parma Nand. 2014b. Real text-cs- corpus based domain independent content selection

model. In IEEE International Conference on Toolswith Artificial Intelligence (ICTAI), pages 599–606.

Rivindu Perera and Parma Nand. 2014c. The role oflinked data in content selection. In Pacific Rim In-ternational Conference on Artificial Intelligence (PRI-CAI), pages 573–586.

Rivindu Perera and Parma Nand. 2015a. Generat-ing lexicalization patterns for linked open data. InSecond Workshop on Natural Language Processingand Linked Open Data collocated with 10th RecentAdvances in Natural Language Processing (RANLP),pages 2–5.

Rivindu Perera and Parma Nand. 2015b. A multi-strategy approach for lexicalizing linked open data. InInternational Conference on Intelligent Text Process-ing and Computational Linguistics (CICLing), pages348–363.

Rivindu Perera and Parma Nand. 2015c. Realtext-asg: Amodel to present answers utilizing the linguistic struc-ture of source question. In 29th Pacific Asia Confer-ence on Language, Information and Computation.

Rivindu Perera, Parma Nand, and Gisela Klette. 2015.Realtext-lex: A lexicalization framework for linkedopen data. In 14th International Semantic Web Con-ference.

Rivindu Perera. 2012a. Ipedagogy: Question answer-ing system based on web information clustering. InIEEE Fourth International Conference on Technologyfor Education (T4E).

Rivindu Perera. 2012b. Scholar: Cognitive ComputingApproach for Question Answering. Honours thesis,University of Westminster.

John Tomlinson and Lewis Bott. 2013. How intonationconstrains pragmatic inference. In 35th Annual Con-ference of the Cognitive Science Society, Berlin, Ger-many. Cognitive Science Society.

M Vargas-Vera and E Motta. 2004. Aqua-ontology-based question answering system. In Mexican Inter-national Conference on Artificial Intelligence, MexicoCity, Mexico. Springer-Verlag.

Hong Yu, Minsuk Lee, David Kaufman, John Ely,Jerome A. Osheroff, George Hripcsak, and JamesCimino. 2007. Development, implementation, and acognitive evaluation of a definitional question answer-ing system for physicians. Journal of Biomedical In-formatics, 40:236–251.

PACLIC 29

205