Shuguang Han1, Daqing He1, Zhen Yue2 & Peter Brusilovsky1

1. University of Pittsburgh, 2. Yahoo! Labs

1

}  Task-centered personalized search ◦  Ahn, J.-w., Brusilovsky, P., He, D., Grady, J., and Li, Q.

(2008) Personalized Web Exploration with Task Models. In: Proceedings of the 17th international conference on World Wide Web, WWW '08, Beijing, China, April 21-25, 2008, ACM, pp. 1-10

}  Social navigation for social search ◦  Farzan, R. (2009). A Study of social navigation support

under different situational and personal factors (Ph.D. Thesis). University of Pittsburgh

}  What is relevant on a page? ◦  Loboda, T. D., Brusilovsky, P., and Brunstein, J. (2011)

Inferring Word Relevance from Eye-movements of Readers. In: Proceedings of 2011 International Conference on Intelligent User Interfaces,IUI 2011, Palo Alto, California, USA, 2011, pp. 175-184

2

4/23/08  WWW  2008   3  

4

}  Cross-device web search ◦  A common search condition in commercial search engines

(Wang et al., 2013) ◦  Refers to the case that a user initializes an information need

on one device but complete it in another (Montañez et al., 2014) �  Often observed for exploratory and complex search tasks

}  Automatic support of cross-device web search is an important research topic ◦  Support = better ranking of relevant documents ◦  Re-rank relevant documents based on users’ search history

(query history and click-through)

5

}  Support queries in the current session using search history from the previous session ◦  Using click-through as relevance feedback (Han et al., 2015) ◦  Re-rank the default Google search results

}  Two problems ◦  Using the full-text of click-through documents is too noisy ◦  Click-through history in mobile may be too few

6

q1 q2 q3 q4 q5 q6 q7 q8

To-be-supported queries

Search context modeling and document re-ranking

}  Mobile touch interaction (MTI) can be used to locate the most relevant subdocument content ◦  Basic idea: low touch speed may indicate users’ interests on

the corresponding subdocument content (Han et al., 2015) ◦  Applying this method in click-through documents can model

users’ search context more accurately

}  MTIs from other users can be used to vote for the most relevant content ◦  Social search based on social navigation (Brusilovsky, et al., 2004)

}  Our social navigation MTI approach considers both of the two advantages

7

8

Goal: Rank relevant documents of query qi for U3

Ranking relevant documents based on: 1: pure query keywords 2: query + the full-text of P1 3: query + subdocument chunk (C6) inferred from self MTI 4: query + subdocument chunks (C1, C3 & C6) inferred from social navigation MTIs

Note: We will talk later (in slide 10) to illustrate the way of extracting the subdocument chunks.

}  Two conditions ◦  Mobile-to-desktop (M-D) search: 1st session on mobile, 2nd

session on desktop ◦  Desktop-to-desktop (D-D) search: 1st session on desktop, 2nd

session on desktop ◦  We also explore the possibility of applying MTIs from the 1st

session of M-D to the 2nd sessions of both M-D and D-D.

}  Three task types ◦  Product (PD) ◦  People (PE) ◦  News (NE) ◦  Total six tasks with two for each type

9

10

}  Procedure ◦  Each participant goes through the same six tasks ◦  Three on M-D and three on D-D ◦  Latin square for task and search condition orders ◦  Each participant is asked to rate the relevance for saved

documents

}  Data collection ◦  24 participants. 961 queries. 1,790 visited pages ◦  3,286 MTIs: drag down/up (71%) are dominated MTIS

}  Ground truth and evaluation ◦  We aggregate relevance scores from all users (with Bayesian

smoothing, see Han et al., 2015) for ground-truth ◦  nDCG@20 for evaluation

11

Session1 (8 minutes / Task) Session 2 (7 minutes / Task)

Mobile Desktop Desktop Desktop

PD1, PE1, NE1 PD2, PE2, NE2 PD1, PE1, NE1 PD2, PE2, NE2

M-D (1st session) D-D (1st session) M-D (2nd session) D-D (2nd session)

One example of experiment procedure (task order and device order were rotated in both two sessions)

Experiment system interfaces for desktop search (left) and mobile search (right)

12

q1 q2 q3 q4 q5 q6 q7 q8

To-be-supported queries

Search context modeling and document re-ranking

}  Using context-sensitive retrieval model ◦  Infer search context

�  from the full-text of click-through �  from the self-MTI based subdocument chunks �  from the social navigation-MTI based subdocument chunks

◦  Re-rank documents using the learning-to-rank algorithm ◦  Evaluate new ranking list based on the ground-truth

Default search results

13

MTI position P MTI speed S Inactive time I after MTI

Reading zone z = [P – M, P + M]. M = 15 words in this paper.

If S ≤ certain speed and I ≥ certain seconds, z is treated as a relevant document chunk.

14

}  Two experiments ◦  Applying social navigation-based MTIs on M-D ◦  Applying social navigation-based MTIs on D-D

�  Using MTIs from M-D for desktop click-through documents

}  Four models to compare ◦  Pure google results (G) ◦  Re-rank G using full-text of click-through (G + HF) ◦  Re-rank G using self-MTI based document chunks (G + HMTI-S) ◦  Re-rank G using social navigation MTI based document

chunks (G + HMTI-SN)

}  Task effects ◦  PD, PE and NE tasks

15

Table 1. nDCG of different runs on M-D. ~HX means a significance comparing to HX

    nDCG@20   Sig. (~HF)   Sig. (~HMTI-S)  

G   0.3974   ↓, p<0.001   ↓, p<0.001  

G+HF   0.4298   -   ↓, p=0.004  

G+HMTI-S   0.4421   ↑, p=0.004   -  

G+HMTI-SN   0.4497   ↑, p<0.001   ↑, p=0.003  

Experiment results : •  MTIs help locate more relevant content: both G+HMTI-S and

G+HMTI-SN are significantly better than G+HF. •  Social navigation MTIs (G+HMTI-SN) further improve the

search performance over the self-MTIs.

16

Table 2. Task effects. Numbers in bold (italics) indicates p<0.05 compare with HF (HMTI-S) using Wilcoxon signed-rank test

    PD   PE   NE  

G   0.4012↓   0.4648↓   0.3286↓  

G+HF   0.4300   0.4835   0.3773  

G+HMTI-S   0.4276   0.5077↑   0.3902  

G+HMTI-SN   0.4249   0.5197↑   0.4019↑  

Experiment results: •  PE and NE show the same trend as overall results, while PD

has no significance. •  The necessity of considering task differences when

applying search context-based retrieval model

17

Table 3. nDCG of different runs on D-D. ~HX means a significance comparing to HX

    nDCG@20   Sig. (~HF)   Sig. (~HMTI-SN)  

G   0.4068   ↓, p<0.001   ↓, p<0.001  

G+HF   0.4452   -   ↓, p=0.610  

G+HMTI-SN   0.4491   ↑, P=0.610   -  

Experiment results : •  Social navigational MTIs do not improve the performance.

o  This may indicate that users have different intentions when they visit the same web page from different devices.

•  Only using the Social Navigation MTIs under the similar search queries (G+HMTI-SN-Q), we observe the significant performance boost.

G+HMTI-SN-Q   0.4549   ↑, P=0.021   ↑, P=0.006  

18

Table 4. Task effects. Numbers in bold (italics) indicates p<0.05 compare with HF (HMTI-S) using Wilcoxon signed-rank test

    PD   PE   NE  

G+HF   0.3612   0.5420   0.4308  

G+HMTI-SN   0.3575   0.5462   0.4498  

G+HMTI-SN-Q   0.3608   0.5529↑(p=0.06)   0.4576↑  

Experiment results: •  Same as M-D, PE and NE show the same trend as overall

results, while PD has no significance. This indicates the necessity of considering task differences when applying search context-based retrieval model

19

}  Conclusions ◦  MTI-based subdocument content gives more fine-grained

relevant information ◦  Social navigation MTIs achieve better performance because of

its ability of aggregating users’ voting for relevance ◦  Social navigation MTIs can be applied not only in M-D but also

in D-D, which could benefit commercial search engines who have both mobile and desktop search logs ◦  Generalizing social navigation MTIs across different search

contexts should be careful

}  Future work ◦  Combing social navigation-based MTI with self-based MTIs ◦  Studying the utility of social navigation MTIs in a natural

search setting (rather than control lab study)

20

More details: Han, S., He, D., Yue, Z., and Brusilovsky, P. (2015) Supporting Cross-Device Web Search with Social Navigation-Based Mobile Touch Interactions. In: F. Ricci, K. Bontcheva, O. Conlan and S. Lawless (eds.) Proceedings of 23nd Conference on User Modeling, Adaptation and Personalization (UMAP 2015), Dublin, Ireland, June 29 - July 3, 2015, Springer Verlag, pp. 143-155, also available at http://link.springer.com/chapter/10.1007/978-3-319-20267-9_12.