Indexing by Latent Semantic Analysis

Written by Deerwester, Dumais, Furnas, Landauer, and Harshman

(1990)

Reviewed by Cinthia Levy

Latent Semantic Indexing

Term-matchingMost retrieval systems match words of a query

(keywords) with words of a document.

ProblemWhat if users want to retrieve information

based upon conceptual content?

Latent Semantic Indexing

Expressing a concept in keywords is complicated and unreliable

Synonymy: many ways to define a concept. Results in ‘poor recall’.

Polysemy: most words have multiple meanings. Results in ‘poor precision’.

Latent Semantic Indexing

Three factors contribute to the failure that IR systems have in overcoming problems associated w/synonymy & polysemy:

1. Identification of index terms is incomplete2. No automatic method adequately addresses

polysemy3. Technical: the way current IR systems work

Latent Semantic Indexing

Goal

...to build an IR system that predicts what terms “really” are implied by a query or what terms “really” apply to a document (i.e. the latent semantics).

Latent Semantic Indexing

Choosing a model

Proximity model:

similar items are put near each other in some space or structure.

Latent Semantic Indexing

Existing proximity models include:

Hierarchical, partition & overlapping clusterings

Ultrametric & additive trees Factor-analytic & multidimensional distance

models

Latent Semantic Indexing

Alternate model was considered, based on the following criteria:

1. Adjustable representational richness2. Explicit representation of both terms and

documents3. Computational tractability for large datasets

Latent Semantic Indexing

Singular value decomposition (SVD) or two-mode factor analysis,

satisfied all three criteria!

SVD: a fully automatic statistical method used to determine associations among terms in a large document collection, and to create a semantic or concept space.

Latent Semantic Indexing

Basis of LSI:

Documents are condensed to contain only “content words” w/semantic meaning

Patterns of word distribution (co-occurrence) are analyzed across a collection of documents.

Latent Semantic Indexing

Basis of LSI:

Document collection is examined as a whole

Documents with many words in common are semantically close.

Documents with few words in common are semantically distant.

Latent Semantic Indexing

Steps of LSI:

Format document: stop words removed, punctuation removed, no capitalization.

Select content words: words with no semantic value are removed using stop list.

Apply Stemming*: reduces words to root form.

*(not applied in Deerwester, et al.)

Latent Semantic Indexing

Result: List of content words

The list of content words is used to generate a term-document matrix.

Latent Semantic Indexing

Term-document matrix

Latent Semantic Indexing

Term-document matrix:

• Term weighting* is applied to each value• SVD algorithm is applied to the matrix• Matrix represents vectors in a multi-

dimensional space

*(not applied in Deerwester, et al.)

Latent Semantic Indexing

Visual representation of a three-dimensional space:Content words form three orthogonal axes (mutually

perpendicular)

eggsbaconcoffee

Latent Semantic Indexing

“If you draw a line from the origin of the graph to each of these points, you obtain a set of vectors in 'bacon-eggs-and-coffee' space. The size and direction of each vector tells you how many of the three key items were in any particular order, and the set of all the vectors taken together tells you something about the kind of breakfast people favor on a Saturday morning.”

Retrieved from:

http://javelina.cet.middlebury.edu/lsa/out/lsa_explanation.htm

Latent Semantic Indexing

Retrieved fromhttp://lsi.research.telcordia.com/lsi-bin/lsiQuery

Latent Semantic Indexing

Romans 1:22 Professing themselves to be wise, they became fools…

Romans 16:6 Greet Mary, who bestowed much labour on us.

Matthew 24:22 And except those days should be shortened, there should no flesh be saved: but for the elect's sake those days shall be shortened.

John 3:17 For God sent not his Son into the world to condemn the world; butthat the world through him might be saved.

Latent Semantic Indexing

(Deerwester…)

System compared to:• Straight term matching• Voorhees• SMART

Using:1. collection of medical abstracts (MED)

2. information science abstracts (CISI)

Latent Semantic Indexing

Summary of analyses

• LSI performed better than or equal to simple term matching

• LSI was shown to be superior to system described by Voorhees

• LSI performed better than or equal to SMART

Latent Semantic Indexing

Conclusion

LSI represents both terms and documents in the same space which provides for the retrieval of relevant information.

LSI does not rely on literal matching thus retrieves more relevant information than other methods.

LSI offers an adequate solution to the problem of synonymy but only a partial solution to the problem of polysemy.