SLOW SEARCHJaime Teevan, Kevyn Collins-Thompson, Ryen White, Susan Dumais and Yubin Kim

Slow Movements

Speed Focus in Search Reasonable

Not All Searches Need to Be Fast

• Long-term tasks• Long search sessions• Multi-session searches

• Social search• Question asking

• Technologically limited• Limited connectivity• Mobile devices• Search from space

Making Use of Additional Time

TIME + SEARCHUnderstand how time influences the search experience

Time in Search: Micro Scale

• Impact of sub-second changes in response time• Study using large-scale query log analysis• Natural variation exists within a single query• Measure interaction as f(response time)

• Time to click• Abandonment rate

Small Increases Big Changes

500 600 700 800 900 1000 1100 1200 1300 1400 15000.9

1

1.1

1.2

1.3

1.4

Page Load Time (ms)

Rel

ativ

e T

ime

To

Cli

ck

500 600 700 800 900 1000 1100 1200 1300 1400 15000.9

1

1.1

1.2

1.3

1.4

NavigationalInformational

1 10 100 10000.9

1.2

Time Spent Searching (mins)

500 600 700 800 900 1000 1100 1200 1300 1400 15000.9

1.2

Page Load Time (ms)

Impact of Sub-Second Changes

• Imperceptible changes impact behavior• Impact of slower page load time

• Slower time to click• Increased abandonment

• Some queries more sensitive to time than others• Time impacts navigational queries > informational• Informational queries less impacted by very slow results

Time in Search: Macro Scale

• Impact of longer changes in response time• Two user surveys with 1476 participants in total

• Detailed survey with 141 MSFT employees• Online survey with 1335 people

• Asked about the participant’s last search• Willingness to wait for results as f(response time)

1 10 100 10000.9

1.2

Time Spent Searching (mins)

Last Search Took Minutes or Hours

1 10 100 10000.0

0.2

0.4

0.6

0.8

1.0

Time Spent Searching (mins)

Pro

bab

ilit

y o

f S

earc

hin

g t

hat

Lo

ng

Time Constraints in Search

• Reported spending more time on some searches• Important tasks• Tasks with a deadline• Tasks with bad search results

• Time constraints• 34% of tasks needed results urgently• 39% of tasks needed results by a deadline• 27% of tasks needed results whenever

Willingness to Wait

• Time willing to wait < actual time spent searching• Participants do not trust the search engine

• 28% people said they want “fast results always”

• For important tasks• Spent more time searching than less important• Willing to wait less than less important

Willing to Wait for Quality Results

1 10 100 10000.0

0.2

0.4

0.6

0.8

1.0

PerfectMuch BetterBetterSlightly Better

Wait Time (mins)

Pro

bab

ilit

y o

f S

earc

hin

g t

hat

Lo

ng

Impact of Longer Intervals of Time

• People do not want to wait, but…• Report spending a lot of time on important tasks

• 90% of the important tasks took more than 1 minute• 23% of respondents found their results unacceptable

• Are willing wait in some cases• When the results obtained were poor• When a perfect answer is sought

Summary

QUESTIONS?Jaime Teevan teevan@microsoft.com Yubin Kim yubink@cmu.edu