Synchronicity

Real Time Recovery ofMissing Web Pages

Martin Kleinmklein@cs.odu.edu

Introduction to Digital LibrariesWeek 14

CS 751 Spring 201104/12/2011

2

Who are you again?

• Ph.D. student w/ MLN since 2005• Diagnostic exam in 2006, dissertation proposal

in 2008• 17 publications to date• Outstanding RA award CS dept • CoS dissertation fellowship• 3 ACM SIGWEB + 2 misc travel grants• CS595 (S10) & CS518 (F10)

3

The Problem

http://www.jcdl2007.org

http://www.jcdl2007.org/JCDL2007_Program.pdf

4

The Problem

• Web users experience 404 errors• expected lifetime of a web page is 44 days [Kahle97]

• 2% of web disappears every week [Fetterly03]

• Are they really gone? Or just relocated?• has anybody crawled and indexed it?• do Google, Yahoo!, Bing or the IA have a copy of

that page?• Information retrieval techniques needed to

(re-)discover content

Web Infrastructure (WI) [McCown07]

• Web search engines (Google, Yahoo!, Bing) and their caches

• Web archives (Internet Archive)• Research projects (CiteSeer)

5

The Environment

Digital preservation happens in the WI

6

Refreshing and Migration in the WI

Google Scholar

CiteSeerX

Internet Archivehttp://waybackmachine.org/*/http:/techreports.larc.nasa.gov/ltrs/PDF/tm109025.pdf

1same URI maps to same or very similar content at a later time

2

same URI maps to different content at a later time

3

different URI maps to same or very similar content at the same or at a later time

4

the content can not be found at any URI

7

URI – Content Mapping Problem

U1

C1

U1

C1

timeA B

U1

C2

U1

C1

timeA B

U2

C1

U1

C1

U1

404

timeA B

U1

???

U1

C1

timeA B

Content Similarity

8

JCDL 2005http://www.jcdl2005.org/

July 2005http://www.jcdl2005.org/

Today

Content Similarity

9

Hypertext 2006http://www.ht06.org/

August 2006http://www.ht06.org/

Today

Content Similarity

10

PSP 2003http://www.pspcentral.org/events/annual_meeting_2003.html

August 2003http://www.pspcentral.org/events/archive/annual_meeting_2003.html

Today

Content Similarity

11

ECDL 1999http://www-rocq.inria.fr/EuroDL99/

October 1999http://www.informatik.uni-trier.de/~ley/db/conf/ercimdl/ercimdl99.html

Today

Content Similarity

12

Greynet 1999http://www.konbib.nl/infolev/greynet/2.5.htm

1999Today

? ?

LS

RemovalHit

RateProxyCache

GoogleYahooBing

• First introduced by Phelps and Wilensky [Phelps00]

• Small set of terms capturing “aboutness” of a document, “lightweight” metadata

13

Lexical Signatures (LSs)

ResourceAbstract

• Following TF-IDF scheme first introduced by Spaerck Jones and Robertson [Jones88]

• Term frequency (TF):– “How often does this word appear in this

document?”• Inverse document frequency (IDF):

– “In how many documents does this word appear?”

14

Generation of Lexical Signatures

• “Robust Hyperlink”• 5 terms are suitable• Append LS to URL

http://www.cs.berkeley.edu/~wilensky/NLP.html?lexical-signature=texttiling+wilensky+disambiguation+subtopic+iago

• Limitations:1. Applications (browsers) need to be modified to

exploit LSs2. LSs need to be computed a priori3. Works well with most URLs but not with all of

them 15

LS as Proposed by Phelps and Wilensky

• Park et al. [Park03] investigated performance of various LS generation algorithms

• Evaluated “tunability” of TF and IDF component

• Weight on TF increases recall (completeness)• Weight on IDF improves precision (exactness)

16

Generation of Lexical Signatures

Rank/Results URL LS

1/243 http://endeavour.cs.berkeley.edu/ endeavour 94720-1776 achieve inter-endeavour amplifiesSearch

1/1,930 http://www.jcdl2005.org jcdl2005 libraries conference cyberinfrastructure jcdl Search

1/25,900 http://www.loc.gov celebrate knowledge webcasts kluge librarySearch

17

Lexical Signatures -- Examples

18

Synchronicity

404 error occurs while browsing look for same or older page in WI (1)if user satisfied return page (2)else generate LS from retrieved page (3) query SEs with LS if result sufficient return “good enough” alternative page (4) else get more input about desired content (5) (link neighborhood, user input,...) re-generate LS && query SEs ... return pages (6)

The system may not return any results at all

19

Synchro…What?

Synchronicity• Experience of causally unrelated events

occurring together in a meaningful manner• Events reveal underlying pattern, framework

bigger than any of the synchronous systems• Carl Gustav Jung (1875-1961)

• “meaningful coincidence”• Deschamps – de Fontgibu plum

pudding example

picture from http://www.crystalinks.com/jung.html

20

404 Errors

21

404 Errors

22

“Soft 404” Errors

23

“Soft 404” Errors

A Comparison of Techniques for Estimating IDF Values to Generate

Lexical Signatures for the Web(WIDM 2008)

• LSs are usually generated following the TF-IDF scheme

• TF rather trivial to compute• IDF requires knowledge about:

• overall size of the corpus (# of documents)• # of documents a term occurs in

• Not complicated to compute for bounded corpora (such as TREC)

• If the web is the corpus, values can only be estimated

The Problem

25

• Use IDF values obtained from 1. Local collection of web pages2. ``screen scraping‘‘ SE result pages

• Validate both methods through comparison to baseline

• Use Google N-Grams as baseline• Note: N-Grams provide term count (TC)

and not DF values – details to come

The Idea

26

27

Accurate IDF Values for LSs

Screen scraping the Google web interface

28

The Dataset

Local universe consisting of copies of URLs from the IAbetween 1996 and 2007

Same as above, follows Zipf distribution

10,493 observations254,384 total terms16,791 unique terms

The Dataset

29

Total terms vs new terms

The Dataset

30

Based on all 3 methodsURL: http://www.perfect10wines.comYear: 2007Union: 12 unique terms

LSs Example

31

1. Normalized term overlap• Assume term commutativity• k-term LSs normalized by k

2. Kendall Tau• Modified version since LSs to compare

may contain different terms3. M-Score

• Penalizes discordance in higher ranks

Comparing LSs

32

Top 5, 10 and 15 terms

LC – local universe

SC – screen scraping

NG – N-Grams

Comparing LSs

33

• Both methods for the computation of IDF values provide accurate results• compared to the Google N-Gram baseline

• Screen scraping method seems preferable since• similaity scores slightly higher• feasible in real time

Conclusions

34

Correlation of Term Count and Document Frequency for Google N-Grams

(ECIR 2009)

• Need of a reliable source to accurately compute IDF values of web pages (in real time)

• Shown, screen scraping works but• missing validation of baseline (Google N-

Grams)• N-Grams seem suitable (recently created,

based on web pages) but provide TC and not DF what is their relationship?

The Problem

36

37

Background & Motivation

• Term frequency (TF) – inverse document frequency (IDF) is a well known term weighting concept• Used (among others) to generate lexical signatures (LSs)

• TF is not hard to compute, IDF is since it depends on global knowledge about the corpus When the entire web is the corpus IDF can only be estimated!

• Most text corpora provide term count values (TC)

D1 = “Please, Please Me” D2 = “Can’t Buy Me Love”D3 = “All You Need Is Love” D4 = “Long, Long, Long”

TC >= DF but is there a correlation? Can we use TC to estimate DF?

Term All Buy Can’t Is Love Me Need Please You Long

TC 1 1 1 1 2 2 1 2 1 3

DF 1 1 1 1 2 2 1 1 1 1

• Investigate relationship between:• TC and DF within the Web as Corpus (WaC)• WaC based TC and Google N-Gram based TC

• TREC, BNC could be used but:• they are not free• TREC has been shown to be somewhat dated

[Chiang05 ]

The Idea

38

• Analyze correlation of list of terms ordered by their TC and DF rank by computing:• Spearman‘s Rho• Kendall Tau

• Display frequency of TC/DF ratio for all terms• Compare TC (WaC) and TC (N-Grams)

frequencies

The Experiment

39

40

Experiment Results

Investigate correlation between TC and DFwithin “Web as Corpus” (WaC)

Rank similarity of all terms

41

Experiment Results

Investigate correlation between TC and DFwithin “Web as Corpus” (WaC)

Spearman’s ρ and Kendall τ

42

Experiment Results

Rank WaC-DF WaC-TC Google N-Grams1 IR IR IR IR2 RETRIEVAL RETRIEVAL RETRIEVAL IRSG3 IRSG IRSG IRSG RETRIEVAL4 BCS IRIT CONFERENCE BCS5 IRIT BCS BCS EUROPEAN6 CONFERENCE 2009 GRANT CONFERENCE7 GOOGLE FILTERING IRIT IRIT8 2009 GOOGLE FILTERING GOOGLE9 FILTERING CONFERENCE EUROPEAN ACM

10 GRANT ARIA PAPERS GRANT

Google: screen scraping DF values from the Google web interface

Top 10 terms in decreasing order of their TF/IDF valuestaken from http://ecir09.irit.fr

U = 14∩ = 6

Strong indicator that TC can be used to estimate DF for web pages!

Integer ValuesTwo Decimals One Decimal

Frequency of TC/DF Ratio Within the WaC

Experiment Results

43

44

Experiment Results

Show similarity between WaC based TC andGoogle N-Gram based TC

TC frequencies

N-Grams have a threshold of 200

• TC and DF Ranks within the WaC show strong correlation

• TC frequencies of WaC and Google N-Grams are very similiar

• Together with results shown earlier (high correlation between baseline and two other methods) N-Grams seem suitable for accurate IDF estimation for web pages Does not mean everything correlated to TC can be used as DF substitude!

Conclusions

45

Inter-Search EngineLexical Signature Performance

(JCDL 2009)

Inter-Search EngineLexical Signature Performance

Martin Klein Michael L. Nelson

{mklein,mln}@cs.odu.edu

http://en.wikipedia.org/wiki/ElephantElephantTusksTrunkAfricanLoxodonta

Elephant, Asian, AfricanSpecies, TrunkElephant, African, Tusks

Asian, Trunk

48

Revisiting Lexical Signatures to(Re-)Discover Web Pages

(ECDL 2008)

50

How to Evaluate the Evolution of LSs over Time

Idea: • Conduct overlap analysis of LSs generated

over time• LSs based on local universe mentioned above

• Neither Phelps and Wilensky nor Park et al. did that• Park et al. just re-confirmed their findings after 6

month

51

Dataset

Local universe consisting of copies of URLs from the IAbetween 1996 and 2007

10-term LSs generated forhttp://www.perfect10wines.com

LSs Over Time - Example

52

53

LS Overlap Analysis

Rooted:overlap between the LS of the year of the first observation in the IA and all LSs of the consecutive years that URL has been observed

Sliding:overlap between two LSs of consecutive years starting with the first year and ending with the last

54

Evolution of LSs over Time

Results:• Little overlap between the early years and more recent ones• Highest overlap in the first 1-2 years after creation of the LS• Rarely peaks after that – once terms are gone do not return

Rooted

55

Evolution of LSs over Time

Results:• Overlap increases over time• Seem to reach steady state around 2003

Sliding

56

Performance of LSs

Idea: • Query Google search API with LSs• LSs based on local universe mentioned above• Identify URL in result set

• For each URL it is possible that:1. URL is returned as the top ranked result2. URL is ranked somewhere between 2 and 103. URL is ranked somewhere between 11 and 1004. URL is ranked somewhere beyond rank 100

considered as not returned

57

Performance of LSs wrt Number of Terms

Results:• 2-, 3- and 4-term LSs perform poorly• 5-, 6- and 7-term LSs seem best

• Top mean rank (MR) value with 5 terms• Most top ranked with 7 terms• Binary pattern: either in top 10 or undiscovered

• 8 terms and beyond do not show improvement

58

Performance - Number of Terms

• Lightest gray = rank 1

• Black = rank 101 and beyond

• Ranks 11-20, 21-30,… colored proportionally

• 50% top ranked, 20% in top 10, 30% black

Rank distribution of 5 term LSs

Performance of LSs wrt Number of Terms

59

Performance of LSs

Scoring:• normalized Discounted Cumulative Gain (nDCG)• Binary relevance: 1 for match, 0 otherwise

60

nDCG for LSs consisting of 2-15 terms(mean over all years)

Performance of LSs wrt Number of Terms

61

Performance of LSs over Time

Score for LSs consisting of 2, 5, 7 and 10 terms

• LSs decay over time• Rooted: quickly after generation• Sliding: seem to stabilize

• 5-, 6- and 7-term LSs seem to perform best• 7 – most top ranked• 5 – fewest undiscovered• 5 – lowest mean rank

• 2..4 as well as 8+ terms insufficient

Conclusions

62

Evaluating Methods to Rediscover Missing Web Pages from theWeb Infrastructure

(JCDL 2010)

64

The Problem

Internet Archive - Wayback Machine

64

www.aircharter-international.comhttp://web.archive.org/web/*/http://www.aircharter-international.com

Lexical Signature(TF/IDF)Charter Aircraft Cargo Passenger Jet Air Enquiry

TitleACMI, Private Jet Charter, Private Jet Lease, Charter Flight Service: Air Charter International

59 copies

The Problem

65

The Problem

65

www.aircharter-international.com

Lexical Signature(TF/IDF)Charter Aircraft Cargo Passenger Jet Air Enquiry

The Problem

66

The Problem

www.aircharter-international.com

TitleACMI, Private Jet Charter, Private Jet Lease, Charter Flight Service: Air Charter International

The Problem

67

The Problem

If no archived/cached copy can be found...

Tags

C?

B

A

Link Neighborhood (LNLS)

The Problem

68

The ProblemThe Problem

69

Contributions

• Compare performance of four automated methods to rediscover web pages

1. Lexical signatures (LSs) 3. Tags

2. Titles 4. LNLS

• Analysis of title characteristics wrt their retrieval performance

• Evaluate performance of combination of methods and suggest workflow for real time web page rediscovery

Contributions

70

Experiment - Data Gathering

• 500 URIs randomly sampled from DMOZ

• Applied filters

– .com, .org, .net, .edu domains

– English Language

– min. of 50 terms [Park]

• Results in 309 URIs to download and parse

Data Gathering

71

Experiment - Data Gathering

• Extract title– <Title>...</Title>

• Generate 3 LSs per page– IDF values obtained from Google, Yahoo!, MSN Live

• Obtain tags from delicious.com API (only 15%)

• Obtain link neighborhood from Yahoo! API (max. 50 URIs)– Generate LNLS

– TF from “bucket” of words per neighborhood

– IDF obtained from Yahoo! API

Data Gathering

72

LS Retrieval Performance

5- and 7-Term LSs

•Yahoo! returns most URIs top ranked and leaves least undiscovered

•Binary retrieval pattern, URI either within top 10 or undiscovered

LS Retrieval Performance

73

Title Retrieval Performance

Non-Quoted and Quoted Titles

•Results at least as good as for LSs

•Google and Yahoo! return more URIs for non-quoted titles

•Same binary retrieval pattern

Title Retrieval Performance

74

Tags Retrieval Performance

•API returns up to top10 tags - distinguish between # of tags queried

•Low # of URIs

•More later…

Tags Retrieval Performance

75

LNLS Retrieval Performance

•5- and 7-term LNLSs

•< 5% top ranked

•More later…

LNLS Retrieval Performance

76

Query LNLS

Combination of Methods

Can we achieve better retrieval performance if we combine 2 or more methods?

Done

Done

Done

Query Tags

Query Title

Query LS

Combination of Methods

77

Combination of Methods

Top Top10 UndisLS5 50.8 12.6 32.4LS7 57.3 9.1 31.1TI 69.3 8.1 19.7TA 2.1 10.6 75.5 Top Top10 Undis

LS5 67.6 7.8 22.3LS7 66.7 4.5 26.9TI 63.8 8.1 27.5TA 6.4 17.0 63.8Top Top10 Undis

LS5 63.1 8.1 27.2LS7 62.8 5.8 29.8TI 61.5 6.8 30.7TA 0 8.5 80.9

Google

Yahoo!

MSN Live

Combination of Methods

78

Combination of Methods

Google Yahoo! MSN Live

LS5-TI 65.0 73.8 71.5

LS7-TI 70.9 75.7 73.8

TI-LS5 73.5 75.7 73.1

TI-LS7 74.1 75.1 74.1

LS5-TI-LS7 65.4 73.8 72.5

LS7-TI-LS5 71.2 76.4 74.4

TI-LS5-LS7 73.8 75.7 74.1

TI-LS7-LS5 74.4 75.7 74.8

LS5-LS7 52.8 68.0 64.4

LS7-LS5 59.9 71.5 66.7

Top Results for Combination of Methods

Combination of Methods

79

•Length varies between 1 and 43 terms

•Length between 3 and 6 terms occurs most frequently and performs well [Ntoulas]

Title Characteristics

Length in # of Terms

Title Characteristics

80

•Length varies between 4 and 294 characters

•Short titles (<10) do not perform well

•Length between 10 and 70 most common

•Length between 10 and 45 seem to perform best

Title Characteristics

Length in # of Characters

Title Characteristics

81

•Title terms with a mean of 5,6,7 characters seem most suitable for well performing terms

•More than 1 or 2 stop words hurts performance

Title Characteristics

Mean # of Characters, # of Stop Words

Title Characteristics

82

Concluding Remarks

Lexical signatures, as much as titles, are very suitable as search engine queries to rediscover missing web pages. They return 50-70% URIs top ranked.

Tags and link neighborhood LSs do not seem to significantly contribute to the retrieval of the web pages.

Titles are much cheaper to obtain than LSs.The combination of primarily querying titles and 5-term LSs as a second option returns more than 75% URIs top ranked.

Not all titles are equally good.Titles containing between 3 and 6 terms seem to perform best. More than a couple of stop words hurt the performance.

Conclusions

Is This a Good Title?(Hypertext 2010)

86

The Problem

86

www.aircharter-international.com

Lexical Signature(TF/IDF)Charter Aircraft Cargo Passenger Jet Air Enquiry

The Problem

87

The Problem

www.aircharter-international.com

TitleACMI, Private Jet Charter, Private Jet Lease, Charter Flight Service: Air Charter International

The Problem

88

The Problem

http://www.drbartell.com/

Lexical Signature(TF/IDF)Plastic Surgeon Reconstructive Dr Bartell Symbol University

???

The Problem

89

The Problem

http://www.drbartell.com/

TitleThomas Bartell MD Board-Certified - Cosmetic Plastic Reconstructive Surgery

The Problem

90

The Problem

90

www.reagan.navy.mil

Lexical Signature(TF/IDF)Ronald USS MCSN Torrey Naval Sea Commanding

The Problem

91

The Problem

TitleHome Page ???

www.reagan.navy.mil

Is This a Good Title?

The Problem

92

Contributions

• Discuss discovery performance of web pages titles (compared to LSs)

• Analysis of discovered pages regarding their relevancy

• Display title evolution compared to content evolution over time

• Provide prediction model for title’s retrieval potential

Contributions

93

Experiment - Data Gathering

• 20k URIs randomly sampled from DMOZ

• Applied filters– English language – min. of 50 terms

• Results in 6,875 URIs

• Downloaded and parsed the pages

• Extract title and generate LS per page (baseline).com .org .net .edu sum

Original 15289 2755 1459 497 20000Filtered 4863 1327 369 316 6875

Data Gathering

94

Title (and LS) Retrieval Performance

Titles 5- and 7-Term LSs

•Titles return more than 60% URIs top ranked

•Binary retrieval pattern, URI either within top 10 or undiscovered

Title and LS Retrieval Performance

95

???

Relevancy of Retrieval Results

•Distinguish between discovered (top 10) and undiscovered URIs

•Analyze content of top 10 results

•Measure relevancy in terms of normalized term overlap and shingles between original URI and search result by rank

Do titles return relevant results besides the original URI?

Relevancy of Retrieval Results

96

Relevancy of Retrieval Results

Term OverlapDiscovered Undiscovered

High relevancy in the top rankswith possible aliases and duplicates.

Relevancy of Retrieval Results

97

Relevancy of Retrieval Results

ShinglesDiscovered Undiscovered

More optimal shingles values than top ranked URIs - possible aliases and duplicates.

Relevancy of Retrieval Results

98

1998-01-27Sun Software Products Selector Guides - Solutions Tree

1999-02-20Sun Software Solutions

2002-02-01Sun Microsystems Products

2002-06-01Sun Microsystems - Business & Industry Solutions

2003-08-01Sun Microsystems - Industry & Infrastructure Solutions Sun Solutions

Title Evolution - Example I

2004-02-02Sun Microsystems – Solutions

2004-06-10Gateway Page - Sun Solutions

2006-01-09Sun Microsystems Solutions & Services

2007-01-03Services & Solutions

2007-02-07Sun Services & Solutions

2008-01-19Sun Solutions

www.sun.com/solutions

Title Evolution – Example I

99

2000-06-19DataCity of Manassas Park Main Page

2000-10-12DataCity of Manassas Park sells Custom Built Computers & Removable Hard Drives

2001-08-21DataCity a computer company in Manassas Park sells Custom Built Computers & Removable Hard Drives

Title Evolution - Example II

2002-10-16computer company in Manassas Virginia sells Custom Built Computers with Removable Hard Drives Kits and Iomega 2GB Jaz Drives (jazz drives) October 2002 DataCity 800-326-5051 toll free

2006-03-14Est 1989 Computer company in Stafford Virginia sells Custom Built Secure Computers with DoD 5200.1-R Approved Removable Hard Drives, Hard Drive Kits and Iomega 2GB Jaz Drives (jazz drives), introduces the IllumiNite; lighted keyboard DataCity 800-326-5051 Service Disabled Veteran Owned Business SDVOB

www.datacity.com/mainf.html

Title Evolution – Example II

100

•Copies from fixed size time windows per year

•Extract available titles of past 14 years

•Compute normalized Levenshtein edit distance between titles of copies and baseline(0 = identical; 1 = completely dissimilar)

How much do titles change over time?

Title Evolution Over TimeTitle Evolution Over Time

101

Title Evolution Over Time

Title edit distance frequencies

•Half the titles of available copies from recent years are (close to) identical

•Decay from 2005 on (with fewer copies available)

•4 year old title:40% chance to be unchanged

Title Evolution Over Time

102

Title Evolution Over Time

Title vs Document

•Y: avg shingle value for all copies per URI

•X: avg edit distance of corresponding titles

•overlap indicated by:green: <10red: >90

•Semi-transparent: total amount of points plotted

[0,1] - over 1600 times

[0,0] - 122 times

Title Evolution Over Time

103

Title Performance Prediction

•Quality prediction of title by

•Number of nouns, articles etc.

•Amount of title terms, characters ([Ntoulas])

•Observation of re-occurring terms in poorly performing titles - “Stop Titles”

home, index, home page, welcome, untitled document

The performance of any given title can be predicted as insufficient if it consists to 75% or more of a “Stop Title”!

[Ntoulas]A. Ntoulas et al. “Detecting Spam Web Pages Through Content Analysis” In Proceedings of WWW 2004, pp 83-92

Title Performance Prediction

104

Concluding Remarks

The “aboutness” of web pages can be determined from either the content or from the title.

More than 60% of URIs are returned top ranked when using the title as a search engine query.

Titles change more slowly and less significantly over time than the web pages’ content.

Not all titles are equally good. If the majority of title terms are Stop Titles its quality can be predicted poor.

Conclusions

Find, New, Copy, Web, Page -Tagging for the (Re-)Discovery of Web Pages

(submitted for publication)

106

The Problem

We have seen that we have a good chance to rediscover missing pages with

• Lexical signatures• Titles

BUT

What if no archived/cached copy can be found?

The Problem

107

The ProblemThe Solution?

ConferencesDigitallibrariesConferenceLibraryJcdl2005

Search

108

The Problem

• What is a good length for a tag based query string?• 5 or 7 tags like lexical signatures?

• Can we improve retrieval performance when combining tags w/ title- and/or lexical signature-based queries?

• Do tags contain information about a page that is not in the title/content?

The Questions

109

The Problem

• URIs with tags rather sparse in previously created corpora

• Creation of new, tag centered corpus• query Delicious for 5k unique URIs

• eventually obtain:• 4,968 URIs• 11 duplicates• 21 URIs w/o tags

The Experiment

110

The ProblemThe Experiment

• Tags queried against the Yahoo! BOSS API• Same four retrieval cases introduced earlier• nDCG w/ same relevance scoring• Mean Average Precision

111

The ProblemThe Experiment

• JaroWinkler distance between URIs• Dice similarity between contents

112

The ProblemThe Experiment

Combining methods

113

The Problem

• Fact:• ~50% of tags do not occur in page

• “Secret”:• ~50% of tags do not occur in current version of page

• ergo: How about previous versions?

The Experiment

114

The Problem

• 3,306 URIs w/ older copies• 66.3% of our tags do not occur in page • 4.9% of tags occur in previous version of page – Ghost Tags• represent a previous version better than the current one

• But what kind of tags are these?• Are they important to the document? To the Delicious user?

Ghost Tags

115

The ProblemGhost Tags

Document importance:TF rank

User importance:Delicious rank

Normalized rank:0 - top1 - bottom

116

Concluding Remarks

Tags can be used for search!

We can improve the retrieval performance by combining tags based search with titles and lexical signatures.

Ghost Tags exist! One out of three important terms better describes a previous than the current version of a page.

How old are Ghost Tags?When do tags “ghostify”? Wrt importance/change of page?

Conclusions

Rediscovering Missing Web Pages Using Link Neighborhood Lexical Signatures

(JCDL 2011)

118

The Problem

We have seen that we have a good chance to rediscover missing pages with

• Lexical signatures• Titles

BUT

What if no archived/cached copy can be found?Plan A: Tags

The Problem

119

The ProblemThe Solution?

Plan B: Link neighborhood Lexical Signatures

120

The ProblemThe Questions

• What is a good length for a neighborhood based lexical signature?• 5 or 7 terms like lexical signatures?• 5..8 terms like tag-based queries?

• How many backlinks do we need?• Is the 1st level of backlinks sufficient?• From where in the linking page should we draw the candidate terms?

121

The ProblemThe Radius Question

Paragraph

Entire page

Anchor text

122

The Dataset

• Same as for JCDL 2010 experiment• 309 URIs• 28,325 first level & 306,700 second level backlinks• Filter for language, file type, content length, HTTP

response code, “soft 404s” => 12% discarded• Lexical signature generation

• IDF values from Yahoo!• 1..7 and 10 terms

123

The ProblemThe Results

level-radius-rank

Anchor text

124

The ProblemThe Results – Backlink Level

level-radius-rank

Anchor text

±5 words

125

The ProblemThe Results – Backlink Level

level-radius-rank

Anchor text

±10 words

126

The ProblemThe Results – Backlink Level

level-radius-rank

Anchor text

±10 words

127

The ProblemThe Results – Radius

level-radius-rank

All Radii

128

The ProblemThe Results – Backlink Rank

level-radius-rank

Anchor,Ranks

10,100,1000

129

The ProblemThe Results – In Numbers

1-anchor-1000

1-anchor-10

WINNER

• 4 terms• first backlink level only• top 10 backlinks only• anchor text only

130

Concluding Remarks

Link neighborhood based lexical signatures can help rediscover missing pages.

It is a feasible “Plan C” due to the high success rate of cheaper methods (titles, tags, lexical signatures).

Fortunately smallest parameters perform best (anchor, 10 backlinks, 1st level backlinks)

Can we find an optimum for the number of backlinks? (10/100/1000 leaves a big margin)Can we identify “Stop Anchors” e.g. click here, acrobat, etc

Conclusions