Slide 1

IMSE: 8810 Human Factor Instructor: Dr. Linsey Baker Student: Qiuyue Zhao

Slide 2

Outlines 1. Introduction of usability What is usability? Usability in mobile phone design Why is usability important? How do we do usability? 2. Discussion of papers Paper 1: A framework for evaluating the usability of mobile phones Paper 2: Systematic evaluation methodology for cell phone user interfaces Paper 3: Chinese character entry for mobile phones 3. Final conclusion

Slide 3

Usability: What is it?

Slide 4

Usability is a measure of quality Its defined by six quality components: Effectiveness Learnability Efficiency Memorability Error Prevention Satisfaction Jakob Nielsen Ben Shneiderman

Slide 5

Main mobile problems Three features or limitations of mobile phone Small screens Awkward input, especially for typing Processing power and available memory

Slide 6

Usability in Mobile phone design Very heavy and awkward Smaller and more portable The first mobile phone with internal antenna Full QWERTY keyboard Multi-touch

Slide 7

Why is usability important in Mobile phone design? Usability is often associated with the functionalities of the product If its difficult to use, people wont buy If users get lost, they wont buy If its hard to read or doesnt answer users key question, they wont buy

Slide 8

How do we do usability? Early Focus on Users and Tasks Know who the users are Know what tasks the users will perform Know Which are most important Empirical Measurement (Evaluation methods) Empirical Measurement Before starting the new design, test the old design. Test your competitors' designs Iterative Design a cyclic process of prototyping, testing, analyzing, refining a product or process

Slide 9

Usability Evaluation Methods(UEM) Analytic methods Claims analysis Usability inspection Heuristic evaluation Cognitive walkthrough Empirical methods Think aloud Questionnaires User interview User observation

Slide 10

Paper 1 :A framework for evaluating the usability of mobile phone What is this paper about? Propose a structuralized usability evaluation model in order to help practitioners identify and organize critical usability problems in a systematic way, understand the relationships among usability factors, and generate better design ideas. Why? Pervious studies only focus on what could constitute the usability factors, seldom organize them in a systematically. Most concerned with software products, not mobile phone. So do not reflect the feature of mobile phone appropriately.

Slide 11

Something about Usability evaluation 1. Importance of usability evaluation 2. Usability design problems: Interface features design (task-independent) LUI: logistical user interface GUI: Graphic user interface PUI: Physical user interface Users tasks design (task-dependent) 3. Usability Evaluation Methods(UEM)

Slide 12

Section 1: Research method Step 1: collection and analysis of usability problems First: classify usability problems into 2 categories Task-independent problems (interface design features) Task-dependent problems (user tasks) Second: typically choose 28 mobile phone tasks Third: associate usability problems with mobile phone tasks

Slide 13

Step 2: Identification of design features to be evaluated What interface features to evaluate Step 3: Evaluation strategy How to evaluate interface feature LUI evaluation GUI evaluation PUI evaluation Task-based evaluation Section 1: Research method Choose from four kinds of evaluations

Slide 14

Section 2:Proposed evaluation framework Hierarchical model of usability factors Aim to evaluate Usability indicator Usability criteria Usability property property

Slide 15

Usability indicators Visual support of task goals Support of cognitive iteration Support of efficient interaction Functional support of user needs Ergonomic support Physical aspect of interaction Cognitive interaction affective

Slide 16

Partial list of Usability criteria

Slide 17

Criteria checklist

Slide 18

Section3:Usability evaluation framework Visual support of task goals

Slide 19

Conclusion This study developed an evaluation framework for supporting usability practitioners to test the suability of mobile phones in an analytical way. A hierarchical model of usability Four sets of checklists A quantification method An evaluation process Limitation Mapping relationship between evaluation areas and usability indicators and its consequent evaluation items of the checklists. The goal-mean relationship need to be further examined

Slide 20

Paper 2: Systematic evaluation methodology for cell phone user interfaces Object Develop a Systematic Evaluation Methodology for Cell Phone User interface(SEM-CPU). SEM-CPU is a specifically designed to integrate five empirical methods into a laboratory-based test in order to evaluate cell phone UIs.

Slide 21

Three stages comprised to construct SEM-CPU Stage 1 Stage 2 Stage 3

Slide 22

Stage1: Data collection in SEM-CPU Five method Scenario-based task performance Benchmark tasks: tasks that reflect the realistic context of use found outside of the laboratory Voice Activated Dialing (VAD) Short Messaging Service(SMS) Phone Book(PB) Questionnaires (six usability attributes: icons clarity, text label, ease of use, ease of locating functions, quality of feed back and overall satisfaction.) Retrospective think-aloud instead of concurrent think aloud User observation for critical incidents Post-task interview

Slide 23

Procedures of data collection

Slide 24

Stage 2: Data analysis Step 1: quantitative analysis Effectiveness Efficiency Satisfaction Learnability Step 2: qualitative data transformation Qualitative Data logging(QDL) template step 3:navigation flow diagram analysis Step 4:verbal protocol Task completion time(s) Error rates Task completion time and task completion ratio at repeated trails Ratings of satisfaction Task completion ratio(%)

Slide 25

An example of the table with the reorganized results of verbal protocol/critical analysis

Slide 26

Step 3: Navigation flow diagram analysis No apparent disability Cognitive disability Physical disability Advantage: can identify the cause of usability problems

Slide 27

Stage 3: Data integration Triangulation strategy

Slide 28

Example of design specification

Slide 29

Conclusion This paper present a systematic methodology, called SEM- CPU, to evaluation cell phone UIs in laboratory-based testing. SEM-CPU guides usability engineers integrating five empirical methods to discover valid usability problems, and generate proper design specifications. Efficiently conduct laboratory-based testing with multiple empirical methods Collect and measure necessary usability attributes Identify determinants of usability problems Generate proper solution Limitation Quality analysis relies heavily on experts knowledge.

Slide 30

Paper 3: Chinese Character entry for mobile phones Object A longitudinal experiment to evaluate character entry performance using both objective and subjective measures for a new design and the existing cell phone keypad.

Slide 31

Background Knowledge Chinese is an iconographic language Two primary methods for entering Chinese Characters Pronunciation-based methods (Pinyin) Stroke-based method Stroke: each stroke is written with a single action Chinese Characters: the minimum functional unit. Most characters are composed of two or more strokes. character strokes

Slide 32

Current stroke input method A single stroke is used as the legend for each key to represent the corresponding group of strokes Problems: Failed to convey the necessary information to all users to understand which strokes could be entered using each key. Select the wrong key Separate a single stroke into two or more pieces and attempt to enter each piece separately Try to enter two or more strokes using a single key

Slide 33

A new stroke-based solution New: combining abstract symbols and concrete stroke examples

Slide 34

Evaluation method Participants: born and raised in China, lived in the US for more than 4 years. 24 volunteers(12 males, 12 females ) Tasks Text entry tasks: enter 5 sentences that were based on headlines from a Chinese news website each of the 6 days Distracter task Questionnaire Procedures Pre-test on the first day 6 days survey Nine-point scale Post-test on the last day

Slide 35

Dependent variables Entry speed Character-level failures Stroke-level accuracy Overall satisfaction Further usage Effectiveness of two keypad design How well participants understood the system

Slide 36

Result Entry speed increased by 32% eliminate 80% Character-level errors for the first time users higher stroke-level accuracy rates than the original design Overall satisfaction increased More interested in using the new system The author uses ANCOVA to prove the interaction between the keypad design and the changes of variables.

Slide 37

Thank you !