msr-bing image retrieval challenge ,written by win

Text

Image Retrieval Challenge -Enhance relationships between query and image

Instructor： MeiChen Yeh ChenLin Yu, ChiungWei Hsu

VIPLAB

Outline

1. Proposed method

2. Evaluation Metric

3. Experiment Result

4. Finding and Difficulty

5. Demo

6. Conclusion

7. Future work

Proposed Method

Query

Natural Language Processing

Tokenization

POSt

QE by WordNet QE by WikipediaWordNet Wikipedia

Click_count ranking Top candidatesUser Clicklog from

MSR dataset

Apple apple apples

an apple ….

Query Processing

1. Stop word and removal

2. Tokenization

3. Stemming and Lemmatization

4. Part-of-speech Tagging

5. Wiki-suggestion (Misspelled words)

6. Expansion (wordnet and wikipeia)

Apple apple apples

an apple ….

Ranking Tablelog candidate count image

apple 1890 QYQtQsx9lH1KwA

apple 503 QJ4gfSPJYhbw0A

… … …

apple mac 490 PvfGna70qGiBIA

Click-count Ranking

MSR dataset provide real world data for user query log.

With this, generated homemade searching table by“Click-count”.

“Max click count rule”

Log data 1,000,000 (only 1/20)

We can make sure that candidate pictures are most popular.

Apple apple apples

an apple ….

Ranking Tablelog candidate count image

apple 1890 QYQtQsx9lH1KwA

apple 503 QJ4gfSPJYhbw0A

… … …

apple mac 490 PvfGna70qGiBIA

Evaluation Metric

MSR vs DIY Method

!

!

[rel]={Excellent=3,Good=2,Bad=0}

X

✔

Experiment Result

Prepare and WorkOff-line:

NLTK to process user query log

Build Ranking table (1,000,000)

Include image(base64) to Database(800,000)

On-line:

NLTK to process query input

Query expansion by word net and wikipedia

Large-scale database query processing

Single unit-query'president','frank','mars','chinese','taiwan','dargon','crash','bird','France','Eiffel','president','tony','frank','mars','chinese','taiwan','London','Mexican','ydney',

'google','yahoo','jessica','microsoft','amazon','windows','apple','line','linux','android',

'world','iphone','bacteria','cat','basketball','dog','micky','tom','jerry','christmas','table',

Test : 32 queries Acc:87.5 %

Compound word-querybook store, picture frame, the lost and bewildered tourist, ice cream, cell phone, apple pie, a story as old as time, a cool wet afternoon, many cases of infectious disease

swimming pool, the senlie old man,pencil box , long and winding road, tiddy bear , hot dog, jennifer love hewitt, some cookie shaped like stars

hello kitty coloring page, kelly osbourne drinking, micky mouse, a wet amd stinky dog

Test : 20 queries Acc:42.28 %

Finding and Difficulty

Spelling correctly can improve retrieval accuracy.

Query expansion can find more related images

!

A ambiguous query can be difficult to used.

The gap exists between users and result images, because the word is polysemic.

The user query still has a semantic problem.

Finding

In a compound word query, the relationship

between previous and next word is very

important.

Query semantic is still a challenge.

Large-scale data processing is a big problem.

How to speed up search performance?

Difficulty

Demo

Conclusion

Enhance relationships between query and image

Find relationships between query and image

Future Work

Query

Natural Language Processing

Tokenization

POSt

QE by WordNet QE by WikipediaWordNet Wikipedia

Click_count ranking Top candidates

Named Entity Recognition

User Clicklog from MSR dataset

Enhance

–ChenLin Yu, ChiungWei Hsu

“Thank you”