orthogonal query recommendation (recsys 2013)
TRANSCRIPT
![Page 1: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/1.jpg)
Orthogonal Query
Recommendation
(Puya) Hossein Vahabi 1, Margareta Ackerman 2, David Loker 3
Ricardo Baeza-Yates 4, Alejandro Lopez-Ortiz 3
1 Microblr Ltd, UK & Italy
2 CalTech, USA
3 University of Waterloo, Canada
4 Yahoo! Research Lab, Barcelona, Spain
ACM RecSys
2013
The 7th ACM Recommender Systems
Conference
![Page 2: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/2.jpg)
Outline
Motivation
Orthogonal Queries
How to find Orthogonal Queries efficiently
?
Recommendation algorithm and
caching schemes
Evaluation
Conclusions
2
![Page 3: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/3.jpg)
3
MOTIVATION
![Page 4: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/4.jpg)
Motivation4
If a user is not satisfied with the
answers, a query recommendation
can be useful.
Query Recommendation
To click or
not to click ?
![Page 5: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/5.jpg)
Motivation5
Sometimes users’ know what
they are looking for, but they don’t
know which keywords to use: “Daisy Duke” but looking for “Catherine Bach”
“diet supplement” but looking for “body building
supplements”
Traditional query
recommendation algorithms fails:
Why?
![Page 6: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/6.jpg)
Why traditional query recommendations
fails? 7
Query Recommendation
Because they are looking for
highly related queries, while I’m
looking for FAA (Federal Aviation
Administration)
![Page 7: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/7.jpg)
Why traditional query recommendations
fails? 8
Query Recommendation
No results for queries in the long-
tail
NO RESULTS.
How to deal with poorly formulated
or long-tail queries?
![Page 8: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/8.jpg)
9
ORTHOGONAL QUERIES
![Page 9: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/9.jpg)
Definition: Orthogonal Queries10
«Orthogonal queries are related
queries that have (almost) no
common terms with the user's
query.»
![Page 10: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/10.jpg)
Definition: Orthogonal Queries11
![Page 11: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/11.jpg)
12
HOW TO FIND ORTHOGONAL
QUERIES EFFICIENTLY?
![Page 12: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/12.jpg)
How to find Orthogonal
Queries?13
![Page 13: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/13.jpg)
How to find Orthogonal
Queries?14
![Page 14: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/14.jpg)
How to find Orthogonal
Queries?15
4
Term Overlap Result Overlap
![Page 15: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/15.jpg)
Examples: Orthogonal Queries16
Result overlap and term overlap
considering 100 results per query
![Page 16: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/16.jpg)
Examples: Orthogonal Queries17
Same keywords: result overlap of 0.408.
![Page 17: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/17.jpg)
Examples: Orthogonal Queries18
Different keywords: result overlap of 0.02.
![Page 18: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/18.jpg)
Range Identification19
![Page 19: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/19.jpg)
ResultOverlap vs Average
TermOverlap21
Pearson correlation coefficient: 0.567
indicating significant positive correlation
![Page 20: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/20.jpg)
TermOverlap vs Average
ResultOverlap22
Pearson correlation coefficient: 0.686
indicating a strong positive correlation
![Page 21: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/21.jpg)
Range Selection23
We have found out that:
there is strong positive correlation
between TermOverlap
ResultOverlap.
Based on the resultOverlap we
want to know whether two
queries have (almost) no
common terms
![Page 22: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/22.jpg)
Range Selection24
![Page 23: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/23.jpg)
Range selection:
resultOverlap vs avg.
termOverlap25
![Page 24: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/24.jpg)
Range Selection26
![Page 25: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/25.jpg)
Orthogonal Queries: Algorithm27
![Page 26: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/26.jpg)
28
RECOMMENDATION
ALGORITHM
AND CACHING SCHEMES
![Page 27: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/27.jpg)
Orthogonal Query(OQ)
Recommendation29
OQ Efficient Computation minHashtechnique plus Caching
OQ Recommendation We rank the orthogonal queries based on the cache policy priority
Cache
![Page 28: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/28.jpg)
Learning to Cache30
How to best fill the cache? MCQ policy, Most Clicks Query
MFFQS policy, Most Frequent Final Query in Session
MFQ policy, Most Frequent Query
MRQ policy, Most Recent Query
S-xmin policy, Test session of x minutes.
We want to learn what is the best
cache policy No complicated
machine learning
techniques!
![Page 29: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/29.jpg)
Satisfied Sessions with Retype31
Click
![Page 30: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/30.jpg)
Dataset32
x min. Test Sessions
![Page 31: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/31.jpg)
Learning to Cache33
Best cache policy has the
highest probability of having the
last query of the test sessions
What is the best policy based on
cache size?
What is the best policy for shorter
or longer test sessions?
![Page 32: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/32.jpg)
Normalized Hits vs Cache Size34
MCQ has the highest hit ratio!
Test sessions of 30 min.
![Page 33: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/33.jpg)
Hit Ratio (%) 35
MCQ has the highest hit ratio!
Test Sessions
Cache Policy
Cache size: 80k
![Page 34: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/34.jpg)
Learning to Cache36
We did optimization also to learn
which was the best caching policy
specifically for OQ, and MCQ
again was the best !
![Page 35: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/35.jpg)
37
EVALUATION
![Page 36: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/36.jpg)
How to evaluate query
recommenders?41
A recommender system should
predict the last query of the
satisfied sessions with retype
based on the first one!
...
![Page 37: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/37.jpg)
How to evaluate query
recommenders?42
...
![Page 38: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/38.jpg)
Comparison43
CG Cover Graph
UF-IQF User Frequency-Inverse Query
Frequency
OQ Orthogonal Query
Recommendation
SC Short Cut
QFG Query Flow Graph
TQG Term-Query Graph
SQ Similar Query
We compared OQ recommendation with 6 others query recommendation algorithms
![Page 39: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/39.jpg)
Comparison44
![Page 40: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/40.jpg)
Results, S@10 as a percentage45
Cache size:
80k
![Page 41: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/41.jpg)
Results, S@10 as a percentage46
We are the best for the long-tail
UF-IQF seems to be better than us, but this is due to the fact that in many cases the last query of a session is just a reformulation of the initial query or is just an spelling error
Is combining OQ with other query recommendation algorithms useful?
![Page 42: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/42.jpg)
Successful Results Overlap:
S@1047
Overlap of the successful
orthogonal recommendations
w.r.t. the successful baseline
query recommendations
OQ SUCCEEDS WHERE
OTHERS FAIL!
![Page 43: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/43.jpg)
User Study48
For each algorithm: generate 5 recommendations
For each initial query: Mix recommendations of different
algorithms
Ask 10 assessors to evaluate the best recommendations
![Page 44: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/44.jpg)
Results: User Study on top-5
Recommendations49
In 45% of cases
OQ is judged to be
useful or
somewhat useful.
OQ is the best for
queries in the
long-tail.
![Page 45: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/45.jpg)
Example50
![Page 46: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/46.jpg)
51
CONCLUSIONS
![Page 47: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/47.jpg)
Conclusions & Future Work52
We have defined Orthogonal
Queries as queries with (almost)
no term overlap with the user’s
query
We have presented how to
efficiently find Orthogonal Queries
by recurring to the result set
YOU MIGHT FIND ANOTHER
WAY
![Page 48: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/48.jpg)
Conclusions & Future Work53
We have presented how to learn
from data which is the best
caching policy
We have designed a simple way
to recommend orthogonal queries
Future work is to find more
good techniques and variants of
this
![Page 49: Orthogonal query recommendation (RecSys 2013)](https://reader030.vdocuments.us/reader030/viewer/2022032616/55a708ba1a28ab1b0b8b457c/html5/thumbnails/49.jpg)
54
THANK YOU