clipping lists & change borders: improving multitasking efficiency with peripheral information...

Clipping Lists & Change Borders: Improving Multitasking Efficiency with

Peripheral Information Design

Mary Czerwinski

George Robertson

Desney Tan

Microsoft Research

Tara MatthewsU.C. Berkeley

2

Peripheral Info can Help Multitaskers

Information workers balance many tasks and interruptions and need help to maintain task flow know when to resume tasks more easily reacquire tasks

We explore peripheral information design to support these needs

3

Study: compare abstraction techniques

Change detection signals when a change has

occurred Relevant task information

pulling out and showing the most relevant content

Scaling shrunken version of all the

content

Which will most improve multitasking efficiency?

4

Results: most relevant task info…

+ benefits task flow, resumption timing, and reacquisition

+ improves multitasking performance more than either change detection or scaling

Clipping Lists performed best: like WinCuts, cuts out a relevant

portion of each task window like Scalable Fabric, arranges

clippings in the periphery

5

Outline

System design Scalable Fabric, Clipping Lists, & Change Borders

User study Results Design implications & future work

6

Our Designs: Video

7

Study Design

no Clippings Clippings

no Change Borders

Change Borders

8

Comparing Tradeoffs

no Clippings Clippings

no Change Borders

+ spatial layout

– no legible content

+ most relevant task info

– detailed visuals / text

Change Borders

+ spatial layout

+ simple visual cue for change

– limited info



9

User Study: Participants

26 users from the Seattle area (10 female) moderate to high experience using computers

and Microsoft Office-style applications

10

User Study: Tasks

Four tasks designed to mimic real world tasks Quiz - wait for modules to load Uploads - wait for documents to upload Email - wait for quiz answers and upload

task documents to arrive Puzzle - high-attention task done while

waiting

12

User Study: Tasks



waiting

13

Uploads

14

User Study: Tasks



waiting

15

User Study: Tasks



waiting

16

Puzzle

17

User Study: Tasks



waiting

18

User Study Setup

left monitor right monitor

19

Metrics & Key Results

Metrics Overall performance Ability to maintain task flow Knowing when to resume a task Ease of reacquiring tasks User satisfaction

Key results Clipping Lists perform significantly better for all metrics Change Borders further improved performance for most

metrics, on average

20

Results: overall performance

Clipping Lists faster performanceChange Borders no significant improvement

Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds

SF Clippings + Change

ClippingsSF + Change

Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



21



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



22



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



23



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



24



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Average T

im

e in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



Average Task Times

540

560

580

600

620

640

660

680

700

Avera

ge T

ime in S

econds



25

Results: task flow

Clipping Lists better task flowChange Borders worse task flow for SF

Average Task Switches

40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches



26

Results: task flow



40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches



27

Results: task flow



40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches



28

Results: task flow



40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Average # of S

witches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches




40

42

44

46

48

50

52

54

56

Avera

ge #

of

Sw

itches



29

Results: knowing when to resume

Clipping Lists trend toward resuming the Quiz task at more opportune times

Average Time to Resume Quiz

0

10

20

30

40

50

60

70

80

90

Ave

ra

ge

T

im

e in

S

eco

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

ra

ge

T

im

e in

S

eco

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

rag

e T

ime

in

Se

co

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

rag

e T

ime

in

Se

co

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

ra

ge

T

im

e in

S

eco

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

ra

ge

T

im

e in

S

eco

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

rag

e T

ime

in

Se

co

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

rag

e T

ime

in

Se

co

nd

s



30

Results: knowing when to resume

Clipping Lists trend toward resuming the Quiz task at more opportune times


0

10

20

30

40

50

60

70

80

90

Ave

ra

ge

T

im

e in

S

eco

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

ra

ge

T

im

e in

S

eco

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

rag

e T

ime

in

Se

co

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

rag

e T

ime

in

Se

co

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

ra

ge

T

im

e in

S

eco

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

ra

ge

T

im

e in

S

eco

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

rag

e T

ime

in

Se

co

nd

s




0

10

20

30

40

50

60

70

80

90

Ave

rag

e T

ime

in

Se

co

nd

s



31

Results: ease of reacquiring tasks

Clipping Lists easier to reacquire tasksChange Borders no significant improvement

Average Quiz Window Switches

16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches



Average Upload Window Switches

0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches



32




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches



33




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Average # of S

witches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




16.5

17

17.5

18

18.5

19

19.5

20

20.5

21

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Average # of S

witches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches




0

24

68

10

1214

1618

20

Avera

ge #

of

Sw

itches



34

Results: user satisfaction

Clipping List UIs rated better than those without

Change Border UIs rated better than those without

Preferred UI 17 – Clipping Lists + Change Borders 4 – Scalable Fabric + Change Borders 2 – Clipping Lists 2 – Scalable Fabric

35





36





37





38

Results Summary Clipping Lists were most effective for all metrics

Overall performance Ability to maintain task flow Knowing when to resume a task Ease of reacquiring tasks User satisfaction

Improvements are cumulative, adding up to a sizeable impact on daily multitasking productivity Clipping Lists

29 seconds faster on average Clipping Lists + Change Borders

44 seconds faster on average

39

Peripheral Design Implications

Why were Clipping Lists more effective? Provided most relevant task info

Why is this surprising? Clippings are higher in detail than Change Borders – require

more attention

Why were Change Borders relatively less effective? Change detection didn’t provide enough task info

Change Borders + Clipping Lists were most effective together

40

Peripheral Design Implications

Show enough relevant task information Even if this means more detailed visuals Combining relevant task info with simple visuals is

best

41

Future Work

Glanceable peripheral interfaces Relevant task info does not require excessive details Easy to perceive & interpret visuals should perform better Presumably, simpler visuals will be easier to perceive What is the sweet spot between simple design and

conveying relevant info?

Automatic selection of content relevant to user’s tasks

42

Summaryno Clippings Clippings

no Change Borders

+ spatial layout




Change Borders

+ spatial layout


– limited info



43


no Change Borders

+ spatial layout




Change Borders

+ spatial layout


– limited info



44


no Change Borders

+ spatial layout




Change Borders

+ spatial layout


– limited info



45

Questions?

Contact Tara [email protected]

Download Scalable Fabrichttp://research.microsoft.com/research/downloads/

46

Extra Slides

47

Summary

Results: showing relevant task info via Clippings best enabled users to

maintain task flow know when to resume tasks more easily reacquire tasks

Peripheral design implications: providing enough relevant task info may be more important

than very simplistic designs together, relevant task info and simplistic visuals improve

performance even more

48

Our Designs

49

Scalable Fabric Study compares Scalable Fabric and two variations of it

Clipping Lists & Change Borders We use SF since tasks span multiple applications and SF

keeps track of tasks across applications

50

Scaling: baseline condition

Previous study: Scalable Fabric performed as well as

Windows & was qualitatively preferred

Scaling benefits: convey window’s spatial layout, major

color schemes, & graphics may enable easy recognition of

windows for reacquiring tasks large updates are visible

…but virtually none of the content is legible…

51

Clipping Lists: semantic content extraction

SF scaled windows are replaced by clippings

Clippings: live window cuts, manually created by user

Task: list of clippings Benefits:

shows most relevant portion of window more readable than scaled window

Disadvantages: may provide too much info, increasing

cognitive overhead removes spatial window layout

52

Change Borders: change detection

Red Change Border Green Change BorderRed Change Border Green Change Border

red = change in progress green = change is complete

53

Change Borders: change detection

Benefits: cognitive overhead is low knowing about changes can help users know when

to resume a paused task, e.g., when waiting for important email documents to upload files to be checked in to a

version control system a compiler build to complete a large file or Web page to load

Disadvantages: change doesn’t always require a

task switch, e.g., moving Web page ads spam email

clipping lists & change borders: improving multitasking efficiency with peripheral information...

Documents

attention task

upload task documents

task flowknowing

help tomaintain task

arrivepuzzle high

tasksfour tasks

quiz answers

real world tasksquiz