introduction to information retrieval - homepages of uva...

Introduction toInformation RetrievalChristof Monz and Maarten de Rijke

Spring 2002

Introduction to Information Retrieval, Spring 2002, Week 1 Copyright c© Christof Monz & Maarten de Rijke 1

Today’s Program


Today’s Program

• What’s Information Retrieval?


Today’s Program


• Some administrative stuff


Today’s Program


• Some administrative stuffI Overview of the course


Today’s Program


• Some administrative stuffI Overview of the courseI Grading, homework etc.


Today’s Program



• How to represent information


Today’s Program



• How to represent information

• Our first retrieval model: boolean retrieval


What is Information Retrieval?



• Finding relevant information in large collections of data




• In such a collection you may want to find:




• In such a collection you may want to find:I ‘Give me information on the history of the Kennedys’





An article about the Kennedys





An article about the Kennedys (text retrieval)





An article about the Kennedys (text retrieval)I ‘What does a brain tumor look like on a CT-scan’






A picture of a brain tumor






A picture of a brain tumor (image retrieval)






A picture of a brain tumor (image retrieval)I ‘It goes like this: hmm hmm hahmmm . . . ’







A certain song







A certain song (music retrieval)


Text Retrieval


telnet://opc.uva.nl

http://www.altavista.com

http://www.google.com

http://www.ilse.nl

Text Retrieval

• Online library catalogs (OPAC)


telnet://opc.uva.nl



http://www.ilse.nl

Text Retrieval


• Internet search engines, such as

AltaVista, Google, Ilse


telnet://opc.uva.nl



http://www.ilse.nl

Text Retrieval




• Specialized systems (aka vendors):


telnet://opc.uva.nl



http://www.ilse.nl

Text Retrieval




• Specialized systems (aka vendors):I MEDLINE (medical articles)


telnet://opc.uva.nl



http://www.ilse.nl

Text Retrieval




• Specialized systems (aka vendors):I MEDLINE (medical articles)I Lexis-Nexis (legal, business, academic, . . . )


telnet://opc.uva.nl



http://www.ilse.nl

Text Retrieval




• Specialized systems (aka vendors):I MEDLINE (medical articles)I Lexis-Nexis (legal, business, academic, . . . )I Westlaw (legal articles)


telnet://opc.uva.nl



http://www.ilse.nl

Text Retrieval




• Specialized systems (aka vendors):I MEDLINE (medical articles)I Lexis-Nexis (legal, business, academic, . . . )I Westlaw (legal articles)I Dialog (business information)


telnet://opc.uva.nl



http://www.ilse.nl

Retrieval vs. Browsing



• Popular Web Directories:



• Popular Web Directories:I Yahoo!, Open Directory Project (dmoz)




• The user has to ‘guess’ the ‘right’ directories to find

the information





the informationI The user has to adapt to the designers’

conceptualization of the directory







• The goal of information retrieval is to provide

immediate random access to the data







• The goal of information retrieval is to provide

immediate random access to the dataI The user can specifiy his information need


IR vs. Database Querying



• IR is not the same thing as querying a database




• Database querying assumes that the data is in a

standardized format





standardized format

• Transforming all information, news articles, web sites

into a database format is difficult and impossible for

large data collections





standardized format

• Transforming all information, news articles, web sites

into a database format is difficult and impossible for

large data collections

• Text retrieval can work with plain, unformatted data


Relevance as Similarity



• A fundamental idea within IR is:




‘A document is relevant to a query

if they are similar ’






• Similarity can be defined as






• Similarity can be defined asI string matching/comparison






• Similarity can be defined asI string matching/comparisonI similar vocabulary






• Similarity can be defined asI string matching/comparisonI similar vocabularyI same meaning of text


The Ubiquity of IR


The Ubiquity of IR

• Information filtering


The Ubiquity of IR

• Information filteringI E-mail routing


The Ubiquity of IR

• Information filteringI E-mail routingI Text categorization


The Ubiquity of IR


• Detecting information structure


The Ubiquity of IR


• Detecting information structureI Hyperlink generation


The Ubiquity of IR


• Detecting information structureI Hyperlink generationI Topic/Information detection/screening


The Ubiquity of IR


• Detecting information structureI Hyperlink generationI Topic/Information detection/screeningI Portal development and maintenance


The Ubiquity of IR


• Detecting information structureI Hyperlink generationI Topic/Information detection/screeningI Portal development and maintenance

• Question Answering


Some Research Groups in IR



• Industrial IR research:



• Industrial IR research: AT&T, NEC, Sun

Microsystems, Microsoft, G&E Research, Sabir

Research, NTT, AltaVista, Xerox, Q-Go, GO.com

(Infoseek), Lexiquest, Answers.com, AnswerLogics,

Google, Ask-Jeeves, Lucent Technologies, IBM . . .








• Academic IR Groups:








• Academic IR Groups: Cornell, Massachusetts, Twente,

Glasgow, Sheffield, Dortmund, Dublin, Stanford,

Syracruse, Virginia Tech, Pisa . . .











• Other:











• Other: CIA, DARPA, ERCIM, Mitre, NIST . . .


History of IR


History of IR

• 1950: Calvin N. Moors coins the term ‘Information Retrieval’


History of IR


• 1959: Luhn describes statistical retrieval


History of IR



• 1960: Maron and Kuhns define a probabilistic model of IR


History of IR




• 1966: Cranfield project defines evaluation measures


History of IR





• 1968: Gerard Salton’s first book about the SMART retrieval

system


History of IR






system

• 1972: Lockheed introduces DIALOG as commercial online service


History of IR






system

• 1972: Lockheed introduces DIALOG as commercial online service

• Late 1980’s: First PC systems incorporate retrieval


History of IR


History of IR

• Early 1990’s: Cheap disks lead to the information storage

revolution


History of IR


revolution

• 1992: Westlaw is the first large-scale information service using

probabilistic retrieval


History of IR


revolution



• Mid 1990’s: Multi-media databases


History of IR


revolution




• 1994: The internet and web explosion


History of IR


revolution




• 1994: The internet and web explosion

• 1995: IR techniques are incorporated in all kinds of information

management applications


Overview of the Course



• Basic IR models (week 1 & 2)




• Evaluating the quality of IR methods (week 3)





• Text representation (week 4)






• Components of an IR system (week 5 & 6)







• Improving effectiveness and efficiency (week 6 & 7)








• Web-based IR (week 8 & 9)








• Web-based IR (week 8 & 9)

• Current research themes (week 10)


Objectives of the Course



• At the end of the course you will be able to. . .



• At the end of the course you will be able to. . .I Exploit web specific information when searching



• At the end of the course you will be able to. . .I Exploit web specific information when searchingI Understand the core components of modern IR

systems




systemsI Understand the potential of IR techniques for today’s

information society





information societyI Build your own search engine (in principle)





information societyI Build your own search engine (in principle)I Make some serious dough


Grading etc.


Grading etc.

• Prerequisites:


Grading etc.

• Prerequisites:I Computer literacy (including an account on gene plus

the ability to use the unix command line interface)


Grading etc.



• Assessment:


Grading etc.



• Assessment:I Weekly reading assignments (1 or 2 papers per week)


Grading etc.



• Assessment:I Weekly reading assignments (1 or 2 papers per week)I (3-5) assignments


Grading etc.



• Assessment:I Weekly reading assignments (1 or 2 papers per week)I (3-5) assignmentsI Final exam


Grading etc.



• Assessment:I Weekly reading assignments (1 or 2 papers per week)I (3-5) assignmentsI Final examI Final mark is obtained as the average of the final

exam (60%), assignments (30%) and reading (10%)


Web Site of the Course


http://www.science.uva.nl/~christof/course/ir/


• URL: www.science.uva.nl/∼christof/courses/ir/





• Features of the web site:





• Features of the web site:I Some of the reading material is available online





• Features of the web site:I Some of the reading material is available onlineI Links to universities, companies and people relevant

to IR






to IRI Printer-friendly versions of the transparancies






to IRI Printer-friendly versions of the transparancies

• Fill out the online form to be added to the mailing list

for this course (important!)



Retrieval Models


Retrieval Models

• A retrieval model is an idealization or abstraction of an

actual retrieval process


Retrieval Models



• Conclusions derived from a model depend on whether

the model is a good approximation of the retrieval

situation


Retrieval Models



• Conclusions derived from a model depend on whether

the model is a good approximation of the retrieval

situation

• Note that a retrieval model is not the same thing as a

retrieval implementation


Retrieval Models


Retrieval Models

display documentsto the user

queryformulation

identify relevantinformation

document

User

representations


Components of a Retrieval Model



• The user:



• The user:I Search expert (e.g., librarian) vs. non-expert



• The user:I Search expert (e.g., librarian) vs. non-expertI Backgound of the user (knowledge of the topic)



• The user:I Search expert (e.g., librarian) vs. non-expertI Backgound of the user (knowledge of the topic)I In-depth searching vs. ‘just-wanna-get-an-idea’

searching




searching

• The documents:




searching

• The documents:I Different languages




searching

• The documents:I Different languagesI Semi-structured (e.g. HTML or XML) vs. plain


Document Representation



• Meta-descriptions



• Meta-descriptionsI Field information (author, title, date)



• Meta-descriptionsI Field information (author, title, date)I Key words




- Predefined




- Predefined

- Manually extracted (by author/editor)




- Predefined

- Manually extracted (by author/editor)

• Content: automatically identifying what the document

is about



Manual Automatic

Controlled

Vocabulary

Free Text



Manual Automatic

Controlled Current indexing

Vocabulary practice

Free Text



Manual Automatic

Controlled Current indexing Text categorization

Vocabulary practice ‘intelligent’ IR

Free Text



Manual Automatic



Current indexingFree Text practice



Manual Automatic



Current indexing Text search enginesFree Text practice ‘statistical’ IR


Controlled Vocabularies


http://www.acm.org/class/1998/overview.html

http://www.nlm.nih.gov/mesh/MBrowser.html

http://www.yahoo.com


• Examples are:






• Examples are:I ACM Computing Classification System







An article on Web search engines would (probably)

be classified as H.3.5 where:









- H: Information Systems










- H.3: Information Storage and Retrieval











- H.3.5: Online Information Services











- H.3.5: Online Information ServicesI NLM Medical Subject Headings (MeSH)











- H.3.5: Online Information ServicesI NLM Medical Subject Headings (MeSH)I Yahoo!





Manual vs. Automatic Indexing



• Pros of manual indexing:




+ Human judgements are most reliable





+ Searching controlled vocabularies is more efficient






• Cons of manual indexing:







− Time consuming







− Time consuming

− The person using the retrieval system has to be

familiar with the classification system







− Time consuming

− The person using the retrieval system has to be

familiar with the classification system

− Classification systems are sometimes incoherent


Automatic Content Representation



• Using natural language understanding?



• Using natural language understanding?I Computationally too expensive in real-world settings



• Using natural language understanding?I Computationally too expensive in real-world settingsI Coverage



• Using natural language understanding?I Computationally too expensive in real-world settingsI CoverageI Language dependence



• Using natural language understanding?I Computationally too expensive in real-world settingsI CoverageI Language dependenceI The resulting representations may be too explicit to

deal with the vagueness of a user’s information need



• Using natural language understanding?I Computationally too expensive in real-world settingsI CoverageI Language dependenceI The resulting representations may be too explicit to

deal with the vagueness of a user’s information need

• Alternative: a document is simply an unstructured set

of words appearing in it: bag of words


Bag-of-Words Approach


Bag-of-Words Approach• A document is an unordered list of words



Grammatical information is lost




• Tokenization: What is a word?




• Tokenization: What is a word?Is ‘White House’ one or two words?





• Case folding





• Case folding

‘President Bush’ becomes ‘president’ , ‘bush’





• Case folding


• Stemming or lemmatization





• Case folding



Morphological information is thrown away





• Case folding




‘agreements’ becomes ‘agreement’ (lemmatization)





• Case folding




‘agreements’ becomes ‘agreement’ (lemmatization)

or even ‘agree’ (stemming)


Example Bag of Words



Scientists have found compelling new evidence of possible

ancient microscopic life on Mars, derived from magnetic

crystals in a meteorite that fell to Earth from the red planet,

NASA announced on Monday.



Scientists have found compelling new evidence of possible

ancient microscopic life on Mars, derived from magnetic

crystals in a meteorite that fell to Earth from the red planet,

NASA announced on Monday.

a, ancient, announced, compelling, crystals, derived, earth,

evidence, fell, found, from (2×), have, in, life, magnetic,

mars, meteorite, microscopic, monday, nasa, new, of,

on (2×), planet, possible, red, scientists, that, the, to


What is this about?


What is this about?

?

added, al, an, and, ballots, been, completed, count,

county (2×), even, former, gore, ground, had, hand,

have (2×), he, if, in (2×), independent, lost, many, miami-

dade, might, new, not, of, president, presidential, requested,

shows, study, that, the, vice, votes, would


What is this about?

?

added, al, an, and, ballots, been, completed, count,

county (2×), even, former, gore, ground, had, hand,

have (2×), he, if, in (2×), independent, lost, many, miami-

dade, might, new, not, of, president, presidential, requested,

shows, study, that, the, vice, votes, would

=

An independent study shows former Vice President Al Gore

would not have added many new votes in Miami-Dade County

and might even have lost ground in that county, if the

hand count of presidential ballots he requested had been

completed.


Boolean Retrieval


Boolean Retrieval

• Boolean operators are: AND (NEAR), OR, NOT


Boolean Retrieval


• The semantics of the Boolean operators:


Boolean Retrieval


• The semantics of the Boolean operators:I t1 AND t2 =


Boolean Retrieval


• The semantics of the Boolean operators:I t1 AND t2 = {d | t1 ∈ r(d)} ∩ {d | t2 ∈ r(d)}


Boolean Retrieval



Documents whose representation contains t1 and t2


Boolean Retrieval



Documents whose representation contains t1 and t2I t1 OR t2 =


Boolean Retrieval



Documents whose representation contains t1 and t2I t1 OR t2 = {d | t1 ∈ r(d)} ∪ {d | t2 ∈ r(d)}


Boolean Retrieval




Documents whose representation contains t1 or t2


Boolean Retrieval




Documents whose representation contains t1 or t2I NOT t1 =


Boolean Retrieval




Documents whose representation contains t1 or t2I NOT t1 = {d | t1 6∈ r(d)}


Boolean Retrieval




Documents whose representation contains t1 or t2I NOT t1 = {d | t1 6∈ r(d)}

Documents whose representation doesn’t contain t1


Boolean Retrieval


Boolean Retrieval

• Information need: President Bill Clinton


Boolean Retrieval


• Boolean query: clinton AND (bill OR president)


Boolean Retrieval


• Boolean query: clinton AND (bill OR president)

bill

clinton

president


Boolean Retrieval in Action



1President George W. Bush on Tuesday makes his first address to a jointsession of Congress and has promised a ”to the point” speech laying outhis plans for tax cuts and spending priorities.




2

While he was still president, Bill Clinton telephoned the chief executive oftelevision network CBS seeking to help two old friends in a million-dollarbilling dispute, The Wall Street Journal reported in its online editionTuesday.




2

While he was still president, Bill Clinton telephoned the chief executive oftelevision network CBS seeking to help two old friends in a million-dollarbilling dispute, The Wall Street Journal reported in its online editionTuesday.

3The White House press office did return calls seeking President Bush’sposition on the bill, but Bell, from the national partnership, said she isoptimistic that paid family leave will become a reality under a Republicanadministration.


Zipf’s Law


Zipf’s Law

no

. occ

urr

ence

s

words (sorted by freq.)


Zipf’s Law

no

. occ

urr

ence

s


• only a few words occur many times


Zipf’s Law

no

. occ

urr

ence

s


• only a few words occur many times

• a lot of words occur only once (hapax legomina)


Searching the Collection



• Finding a word by linear search can be inefficient




• Solution: Construct a matrix which indexes documents

and words





and wordsI Matrix can be extremely large





and wordsI Matrix can be extremely largeI The matrix will be sparse (Zipf’s law)






• Better: Construct an inverted index






• Better: Construct an inverted indexI A word points to the documents in which it occurs






• Better: Construct an inverted indexI A word points to the documents in which it occurs

• This is an implementational (not a modeling) issue!


Pros and Cons of Boolean Retrieval



• Pros of Boolean Retrieval:




+ Clean and simple formalism





+ Firm grip on query formulation






• Cons of Boolean Retrieval:







− Most non-experts cannot handle boolean expressions,

and query formulation may be time consuming









− No ranking of retrieved documents









− No ranking of retrieved documents

− Exact matching may lead to too few or too many

retrieved documents


Alternatives to Boolean Retrieval



• Vector Space Retrieval:



• Vector Space Retrieval:I Users can enter free text



• Vector Space Retrieval:I Users can enter free textI Documents are ranked



• Vector Space Retrieval:I Users can enter free textI Documents are rankedI Best match instead of exact match



• Vector Space Retrieval:I Users can enter free textI Documents are rankedI Best match instead of exact match

• Probalistic Retrieval


What We’ve Done Today



• What’s Information Retrieval




• What’s a retrieval model





• Documents and their representations






• Boolean retrieval







• Searching words







• Searching words

• Discussion of boolean retrieval


Homework


http://www.science.uva.nl/~christof/courses/ir/hw/

Homework

• Read Chaper 1 of Salton & McGill: Modern

Introduction to Information Retrieval (library)



Homework



• Read Jansen et al.: Real life, real users, and real needs:

A study and analysis of user queries on the web

(available online)



Homework



• Read Jansen et al.: Real life, real users, and real needs:

A study and analysis of user queries on the web

(available online)

• Give an intuitive ranking for 7 documents

For details go to:

www.science.uva.nl/∼christof/courses/ir/

