advisor: hsin-hsi chen reporter: chi-hsin yu date: 2010.08.05

Advisor: Hsin-Hsi Chen

Reporter: Chi-Hsin Yu

Date: 2010.08.05

Query Log Related Topics

IntroductionTopic A: Relevance Assignment

Using Query Log to Assign Actual Relevance of Documents for User Communities

Topic B: Knowledge Transfer (cloud wisdom in QL)Transfer Learning in Query Log

Topic C: Query UnderstandingSemantic Relation of Query Terms in Query

LogDiscussions

Outlines

Query Log

Introduction (1/3)

From: Intent Based Clustering of Search Engine Query Log, 2009

IssuesPerceived relevance v.s. Actual relevance Clicks

Click bias in positions Relevance

Query – documentUser intent/goal – queries – documents User community – queries – documents

Cost Editorial judgments v.s. model predicted judgments

Introduction (2/3)

Introduction (3/3)

From: SIGIR 2010 Tutorials

Task (original)Assign relevance judgment for a q-d pair

Relevance Assignment (1/4)

21121

Actual Relevance

From: Intent Based Clustering of Search Engine Query Log, 2009

Applications of the predicted relevance judgments (pr)As meta-features

As actual relevanceLow cost

Relevance Assignment (2/4)

Samples(Matrix)

pr

RankingAlgorithms

Performance in dataset (editorial judgment)

ButA Dynamic Bayesian

Network Click Model for Web Search Ranking (WWW2009, Track: Data

Mining/Session: Click Models)

ExperimentsPredicting click-through ratePredicted relevance as a

ranking featureLearning a ranking function

with predicted relevance

Relevance Assignment (3

From: (WWW2009, Track: Data Mining/Session: Click Models)

Task (Revised) Assign relevance judgment for a ((user community, q), d) pair

Not q-d pair Pseudo-query: (user community, q)

Models GA DBN: same as proposed click models in WWW09 papers

Difficulties Pseudo-query generation (include user information) User clustering/classification

Evaluation Joint training (as in the WWW09 papers)

Application For detail analysis of personalized search

because we can use predicted relevance to substitute the editorial judgment

Relevance Assignment (4/4)

Query log = cloud wisdom

Task: Mining/leverage cloud wisdom in QLUse transfer learningUse QL to learn meta-features

Knowledge Transfer (1/3)

Task Leverage useful structure/knowledge in QL to boost

performance of existing datasets (human judgment)Algorithm

SCL: structure correspondence learning Difficulties

Selection of extended feature s in QLEvaluation

As common IR evaluation metricsExpected results (planned experiments)

Can improve performance when use whole training datasetCan improve performance when using small training

dataset

Knowledge Transfer (2/3)

SCL ACL 2005, EMNLP 2006Domain Adaptation with Structural

Correspondence Learning (EMNLP 2006)

Knowledge Transfer (3/3)

From: EMNLP 2006

From Google search suggestions

Interpretation “machine learning wiki/amazon” Concept + in site “machine learning stanford” concept + in organization “machine learning tutorial/tool/ppt/journal” concept + in

topic/resources “machine learning kernel” concept + topic

Semantic Relations of Query Terms (1/6)

Compare to compound noun semanticsDiarmuid ´O S´eaghdha, 2008

Semantic Relations of Query Terms (2/6)

From: Diarmuid ´O S´eaghdha, 2008

Beyond static semantic relations

Dynamic semantic relations recognition What is the patterns in the process of query

reformulation?Is this useful to identify user goal in a session? Can we build new click model based on semantic

relation?

Semantic Relations of Query Terms (3/6)

Pseudo-session A 1. Apple2. Apple ipod3. Apple ipod discount

Pseudo-session A 1. <concept>2. <organization> <product>3. <organization> <product>

<topic/concept>

Current works (incomplete)

Introduction – Semantic Relations of Query Terms (4/6)

From: SIGIR 2010 Tutorials

Research planDefinition of semantic relations in QL

Use Google query suggestions to study the types of semantic relations

Segmentation of query termsMapping segmented query terms to ontologyClassification of semantic relation in QLMining important statistics from QLApplications

Ranking strategies based on SRsClick models based on SRs

Semantic Relations of Query Terms (5/6)

Task Study of semantic relations of query terms

AlgorithmQuery Segmentation, classification, statistics

miningDifficulties

Depends ...Evaluation

Depends ...Expected results

New problem in NLP and in IR

Semantic Relations of Query Terms (6/6)

Thanks for Your Attention. Discussions