Advantages of Query Biased Summaries in Information

Retrieval

by A. Tombros and M. Sanderson

Presenters:Omer Erdil Albayrak

Bilge Koroglu

OutlineBrief Introduction to Typical Information Retrieval System

Why query biased summaries needed?

How a query biased summary generated?

Experimental EnvironmentExperimental ResultsConclusion and Future Works

2/13

Introduction

3/13

Typical IR System: input: information needoutput: ranked document list

Figure 1. Screenshot from an IR System

Introduction (con’t...)Deciding whether the retrieved document

is worth to further investigationExamining the static summariesChecking the whole content

Weak content indicators: static summaries

Time consuming job: refering full text nearly all times

Aiming to minimise reaching whole content Generating query biased summaries

4/13

Generating Query Biased SummariesQuery-specific summaries for each retrieved

documentClassical approach to summarization

Extracting sentencesAssigning scores to sentencesSelection of best-scoring sentences

Some modification on score assignmentExtra importance to titles and subtitlesMore weights to sentences with clusters of terms

Additional points to sentences includes query terms5/13

Experimental Environment

6/13

TREC test collection: Wall Street Journal news50 queries of which relevant documents are known

2 groups of 10 postgraduate students to find relevant docs for 50 queriesOne group with static summariesAnother is to use query biased summaries

50 retrieved docs per each query5 queries per student5 minutes allocated per queryIdentical computers in hardware/software

aspects

Experimental Environment (con’t...)

7/13 Figure 2. The subjects performing on query biased & static summaries in the experiments

Experimental Results

8/13

Recall: the ratio of total number of relevant documents for query to the number of retrieved relevant documents

Precision: the ratio of total number of retrieved documents for query to the number of retrieved relevant documents

Figure 4. Recall values of 2 groups

Figure 5. Precision values of 2 groups

Experimental Results (con’t...)

9/13

Speed of evaluators’ judgments of relevancy2.62% on 20 documents corresponds 13%

increase of average number of examined documents

Figure 6. The number of doc percentage for 2 group

Experimental Results (con’t...)

10/13

The need for checking full textAverage number of full text reference per

query: ‘query biased summary group’ : 0.3 ‘static summary group’: 4.74

Figure 7. Average number of references to the full text of documents per query

Conclusion and Future Works

11/13

Effective method: employing query biased summaries in IR SystemsEasily identifiable more relevant documentsDecreasing the need to check whole

content of documentApplicable to web search engines

Expensive to retrieve the documenst from slow, not reliable, and remote servers

Requiring to manage an index fileMore experiments on different summarization

methods with different datasets

12/13

Thank you...