Digital Reading in Academic Environments

Usability Test for Potential Use of e-Readers in UniversitiesDana Kianfar, Evgeniya Zabolotnya

E-books become increasingly popular among students, instructors, and academic advisors. Can they be recommended in academic environments?

Research Question:

→ How well can students perform essential tasks needed for academic purposes on e-book readers?→ Given a set of different e-book reader which is the one preferred by the students and why?

Devices:

• iPad 2 (full-sized tablet)• Nexus 7 (mini tablet)• Kindle (e-Ink screen)

Nexus• Pros• Ergonomics, battery life

• Cons• Small screen, glare• Confusing touch-sensitivity

iPad• Pros• Large screen, aesthetics• Intuitive interface

• Cons• Glare• Heavy

Kindle• Pros• Very ergonomic, No glare• Amazon Kindle store

• Cons• Interface

Observations: pros and cons

Requirements

• Based on previous exploratory study• Simplicity• Response Time & screen refresh rate• Touch-sensitivity• Weight, comfortability

• Results convey a relatively high impatience and intolerance to errors• In an academic setting

1. Pre-test questionnaire (~ 2 minutes)• Warm-up• Collecting background information• Ten close-ended questions

2. Usability test (~ 15 minutes)• 15 fixed set of tasks mining for quantitative data• Order of tasks to measure learnability and memorability• Primary metrics for evaluation: errors per task per device, time per

device• Think-Aloud protocol

3. Post-test questionnaire (~ 5 minutes)• Cool-down• Allow participants to reflect on their experience• 29 open-ended questions aiming at qualitative data

Method: Procedure

1. Pre-test• Demographics: age, gender, nationality, mother tongue• Technology: comfortability, previous experience with readers

2. Post-test• Likert scales on overall device evaluation• For each device

• Positive/negative on reading experience• Intuitive interface and easy to use• Response-time

• Overall thoughts, favorites and suggestions

Method: Questionnaires

• From flipping pages to searching the web• Easy but very common operations, covering most of what the device

offers• Same procedure for each device and each participant• Devices distributed at random to prevent learning effect

Method: Tasks

  iPad Nexus KindleSergei 3 2 1axxxx 2 3 1

ST 1 2 3Berry 1 3 2

Viktoria 2 3 1Uzu 3 2 1

Anais 2 1 3Abdul A. 1 3 2

(1) Find the book “The Picture of Dorian Gray” in the native application(2) Navigate to page 24(3) Highlight two paragraphs(4) Write a note on the same page: note by [alias-name](5) Bookmark page 20(6) View notes on page 100(7) Return to bookmarked page(8) Search for word 'horrid' in the text(9) Define 'horrid' using the dictionary(10) Look for “Oscar Wilde” on Wikipedia (using web browser)(11) Navigate to first link you find(12) Return to the previous page(13) Navigate to Chapter 3(14) Change the font size(15) Find the note you made earlier

Method: Tasks

Results - Participants

• 50% of the participants described themselves as technologically savvy

• Average total time spent on test was 12.8 minutes

• Average time of those who did not consider themselves as technologically

savvy was 17.5 minutes

ID Age Nationality Gender

Sergei 20-24 European Maleaxxxx 25-29 Asian FemaleST 20-24 European MaleBerry 20-24 European MaleViktoria <20 European FemaleUzu 20-24 Asian FemaleAnais 20-24 European FemaleAbdul Aziz 20-24 Asian Male

Results – Time spent on device

1. Completion of tasks took the longest with the Kindle 9 minutes

2. The Nexus 7’ mean was 4.8 minutes

3. The iPad was the least time consuming with on average 3.6 minutes

iPad21%

Nexus27%

Kindle52%

Average time spent on reader

Error distribution• 43% of errors occurring on the Kindle

• 38% closely followed by the Nexus 7

• 21% of total errors on the iPad only

Results – Most erroneous tasks• Task #2, navigate to page 24, was the most erroneous with 22% of the total

amount of errors→ First task, participants may have benefited from an introductory exploration, Kindle does not display page numbers. On Nexus 7 it was difficult to stop page changing.

• Task# 6, asking to find a note on page X has caused 14% of the total errors. → On Kindle to be found under the ‘Go to’ menu. On the Nexus notes are embedded in the menu bar having a great resemblance with other icons, difficult to remember/confuse. Menu bar would disappear/ responds slowly.

• Task#9, asking to define the word horrid via the dictionary, received 13% of all errors

Task 1

3%Task 222%

Task 39%

Task 45%

Task 55%

Task 614%

Task 75%

Task 81%

Task 913%

Task 105%

Task 113%

Task 124%

Task 135%

Task 144% Task 15

3%→ 63% of the 15 errors occurred on the Kindle, bad interface design, difficulty using the physical keyboard. On the Nexus 7, did not find such application, had to leave the book to search web

Results – Participant Evaluation

• The Kindle was referred to as interesting, complicated, non-intuitive

• The iPad was described as being easy, fun, intuitive, and fast.

• Nexus 7 was referred to as annoying when jumping pages, or waiting for

the menu to appear.

• LCD screen as fun, enjoyable, and colourful

• But, for academic purposes an e-ink reader is less tiring to the eyes.

iPad Nexus 7 Kindle012345

4.75

3.375 3.125

Reader EvaluationLikert: 1:bad, 5:excellent

Conclusion

• Students were able to perform essential academic tasks most efficiently with the use of

the iPad

→ shortest time spent

→ smallest number of errors

• Great difficulties with the use of the Kindle

• Participants were able to perform academic tasks on all devices, but with great

difference in time and ease

→ iPad is the only device that can be recommended for academic studies

• To be considered:

iPad interface has greatest resemblance to a PC

Popularity of Apple products among students

Possible distraction through applications (social media)