wonsup lee, kihyo jung, heecheon...
TRANSCRIPT
![Page 1: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/1.jpg)
Wonsup Lee, Kihyo Jung, Heecheon You
Department of Industrial and Management Engineering, POSTECH, South Korea
17th IEA World Congress
![Page 2: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/2.jpg)
Contents
2
![Page 3: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/3.jpg)
Background
Getting importance of user-friendly product development⇒ Required easy of use, comfortable and satisfactory interface
(Jordan, 1996; Abras et al., 2004; Courage and Baxter, 2005)
Short-term lifecycle of product development & high competition⇒ Required quick, simple, and comprehensive evaluation
on product development companies (Hartson et al., 1996; Butler, 1996)
⇒ for quick evaluation: Subjective testing > Objective testing⇒ for simple evaluation: Quantitative testing > Qualitative testing
3
![Page 4: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/4.jpg)
Classification of usability evaluation methods
Type of information collection• Qualitative evaluation vs. Quantitative evaluation
Type of collected data• Subjective evaluation vs. Objective evaluation
Scope of evaluation• Comprehensive evaluation vs. partial evaluation
![Page 5: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/5.jpg)
Type of information collection
Qualitative evaluation • Descriptive information of opinions or insights• Methodologies: interview, heuristic evaluation, observation,
questionnaires, think aloud
Quantitative evaluation • Quantified opinions• Methodologies: measurement, scaling (e.g. Borg’s CR-10),
scoring
![Page 6: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/6.jpg)
Type of collected data
Subjective evaluation • Participants’ opinions, experimenters’ insights• Methodologies Subjective and qualitative: interview, questionnaires Subjective and quantitative: Borg’s CR-10 scale, 5-scale
Objective evaluation • Participants’ performance or ability (e.g. performance time,
accuracy, error rate, motion, force)• Methodologies: ergonomic experiment with equipment
![Page 7: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/7.jpg)
Scope of evaluation
Comprehensive evaluation • Summative evaluation• Usually conducted on the final stage of product development
phase• Evaluated most of elements or characteristics
Partial evaluation • Formative evaluation• Usually conducted on the earlier stage or on special purpose• Evaluated focused elements, functions or benchmark tasks• e.g. keypad size of mobile phone
![Page 8: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/8.jpg)
Better manners for practitioners
Type of information collection• Qualitative evaluation vs. Quantitative evaluation
Type of collected data• Subjective evaluation vs. Objective evaluation
Scope of evaluation• Comprehensive evaluation vs. partial evaluation
for quick, simple, and comprehensive evaluation for practitioners
![Page 9: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/9.jpg)
Product-User Interaction Model
Force- Muscular strength- Force distribution
Posture/Motion- Joint angle- Joint motion range
Grip fit
Dust ormaterials
Furniture
Tasksequence
Typeof task
Efficiency
Comfort
Fatigue
Fitness ofgrasp
Spatiallimitations
Joints
Muscles
Anthropometry
Product Task User
Type ofthe floor
Physical UI
Environment
Need to consider 4 categories and those preferences
9
![Page 10: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/10.jpg)
Objectives
Developed a systematic protocol for comprehensive testing and analysis on product usability
Conducted subjective and quantitative usability evaluation
Applied to canister-type vacuum cleaner to examine its effectiveness
10
![Page 11: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/11.jpg)
Approach
11
![Page 12: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/12.jpg)
Approach
12
![Page 13: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/13.jpg)
Step 1. Analysis of product characteristics
Force- Muscular strength- Force distribution
Posture/Motion- Joint angle- Joint motion range
Grip fit
Dust ormaterials
Furniture
Tasksequence
Typeof task
Efficiency
Comfort
Fatigue
Fitness ofgrasp
Spatiallimitations
Joints
Muscles
Anthropometry
Product Task User
Type ofthe floor
Physical UI
Environment
Product-User Interaction Model Analysis of each categories and those preferences
13
![Page 14: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/14.jpg)
Hierarchy of product component (illustrated for vacuum cleaner)
Category Component Category Component
Body
Body case
Wand andBrush
Telescopic wand
Carriage handle Wand handle
Control buttons Hose
Display Control buttons
Dust bin Connector
Wheels Brush
Characteristics of PRODUCT
14
![Page 15: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/15.jpg)
Characteristics of TASK
Hierarchy of task
Category Task Subtask
Infrequenttask
Preparation
Power supply
Brush installation
Brush exchange
Interactive task Operation
MovementUse of carriage handle
Use of wheels
Button controlPower on/off
Suction power control
15
![Page 16: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/16.jpg)
Usability measures
Category ComponentComfortable
postureThe extent to which comfortable postures are maintained while operating the product
Efficient motionThe extent to which motions are efficiently used to operate the product
Natural motion The extent to which natural motions are used to operate the productEffective use of
forceThe extent to which forces used to operate the product are acceptable
Fit to the handThe extent to which the handle or grip fits to the size and shape of the hand
Ease of use The extent to which a user easily operates the product
Characteristics of USER
Force- Muscular strength- Force distribution
Posture/Motion- Joint angle- Joint motion range
Grip fit
Efficiency
Comfort
Fatigue
Fitness ofgrasp
Joints
Muscles
Anthropometry
User
16
![Page 17: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/17.jpg)
Environment
Characteristics of ENVIRONMENT
floor bed
upon furniture under furniture gaps
17
![Page 18: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/18.jpg)
As conducting usability test by tasks, related characteristics are also evaluated
Task ⅹ Component
Relationship among characteristics
Task
ComponentBody Wand
Bodycase
Controlbuttons Display
Carriagehandle
Telescopicwand
Brush
PreparationPower supply OBrush installation O O
Brush exchange O O
Operation
MovementUse of carriagehandle
O O
Button control
Power on/off O O
Suction power control
O O
18
![Page 19: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/19.jpg)
Task ⅹ Usability measures
Relationship among characteristics
TaskUsability measures
Comfortableposture
Efficientmotion
Naturalmotion
Fit tothe hand
Ease of use
PreparationPower supply O OBrush installation O O OBrush exchange O O O
Operation
MovementUse of carriage handle
O
Button control
Power on/off O O
Suction power control
O O
19
![Page 20: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/20.jpg)
Approach
20
![Page 21: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/21.jpg)
Compo-nent
Task Usability questionsProduct Model
Model A Model B Model C
Brush
Brushinstallation
The extent to which brush is connected to tube at a single trial WITHOUT ERROR
Low High
Low High
Low High
The extent to which brush is EASILY connected to tube
Brushexchange
The extent to which release button on brush is pressed by applying PROPER FORCE
The extent to which brush can be EASILY disconnected from tube
Step 2. Questionnaire development
Based on analysis of characteristics of TASK, PRODUCT, and USER categories and its relationship
21
![Page 22: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/22.jpg)
Based on analysis of characteristics of TASK, PRODUCT, and USER categories and its relationship
Compo-nent
Task Usability questionsProduct Model
Model A Model B Model C
Brush
Connection of brush to tube
The extent to which brush is connected to tube at a single trial WITHOUT ERROR
Low High
Low High
Low High
The extent to which brush is EASILY connected to tube
Disconnection of brush from tube
The extent to which release button on brush is pressed by applying PROPER FORCE
The extent to which brush can be EASILY disconnected from tube
Step 2. Questionnaire development
Task ⅹ Component
Task
ComponentBody Wand
Bodycase
Controlbuttons Display
Carriagehandle
Telescopicwand
Brush
PreparationPower supply OBrush installation O O
Brush exchange O O
Operation
MovementUse of carriage handle
O O
Button control
Power on/off O O
Suction power control
O O
22
![Page 23: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/23.jpg)
Based on analysis of characteristics of TASK, PRODUCT, and USER categories and its relationship
Compo-nent
Task Usability questionsProduct Model
Model A Model B Model C
Brush
Connection of brush to tube
The extent to which brush is connected to tube at a single trial WITHOUT ERROR
Low High
Low High
Low High
The extent to which brush is EASILY connected to tube
Disconnection of brush from tube
The extent to which release button on brush is pressed by applying PROPER FORCE
The extent to which brush can be EASILY disconnected from tube
Task ⅹ Usability measures
TaskUsability measures
Comfortableposture
Efficientmotion
Naturalmotion
Fit tothe hand
Ease of use
Preparation
Power supply O O
Brush installation O O O
Brush exchange O O O
Operation
MovementUse of carriage handle
O
Button control
Power on/off O O
Suction power control
O O
Step 2. Questionnaire development
23
![Page 24: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/24.jpg)
Compo-nent
Task Usability questionsProduct Model
Model A Model B Model C
Brush
Brush installation
The extent to which brush is connected to tube at a single trial WITHOUT ERROR
Low High
Low High
Low High
The extent to which brush is EASILY connected to tube
Brush exchange
The extent to which release button on brush is pressed by applying PROPER FORCE
The extent to which brush can be EASILY disconnected from tube
Step 2. Questionnaire development
Based on analysis of characteristics of TASK, PRODUCT, and USER categories and its relationship
Quantitative results
24
![Page 25: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/25.jpg)
Step 2. Conducting usability testing
2.1m
1.1m
1.1m
0.5m 0.3m
1.4m
0.6m
floor
desk
shelters
Evaluation room set up: based on analysis of ENVIRONMENT
Participants evaluated every TASK related components and usability measures (PRODUCT)
(USER)
25
![Page 26: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/26.jpg)
Approach
26
![Page 27: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/27.jpg)
Compo-nent
Task Usability questionsProduct Model
Model A Model B Model C
Brush
Brush installation
The extent to which brush is connected to tube at a single trial WITHOUT ERROR
Low High
Low High
Low High
The extent to which brush is EASILY connected to tube
Brush exchange
The extent to which release button on brush is pressed by applying PROPER FORCE
The extent to which brush can be EASILY disconnected from tube
Step 3. Quantification of evaluation result
Evaluation conducted by component, task, and usability measure Qualitative results calculated by usability measure, task, and
component
Component TaskUsability measure
Model A Model B
Score Task score Score Task score
Brush
BrushInstallation
Comfortable posture
3.2
3.1
3.9
3.7Efficient motion 3.2 3.7
Easy of use 2.9 3.5
Brushexchange
Comfortable posture
3.5
3.5
4.2
3.9Efficient motion 3.4 3.7
Easy of use 3.6 3.8
27
![Page 28: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/28.jpg)
Step 3. Analysis of usability problems
Good Fair Bad
Category Design criteria Description Image
Preferred
Weight Model A (290 g) is lighter than Models B, E (513 g), D (610 g) and C (545 g)
-
Visual information Arrows to indicate connecting location and direction
Auditory feedback Sound feedback when pipes are connected properly
Requiring improvement Use of force Models B, C, D, and E require larger force to connect pipe
with brush-
Click
Standard Error
Brush installation
Quantitative result of components
were established
Qualitative design guidelineswere suggested by benchmarking analysis
28
![Page 29: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/29.jpg)
Advantages• Comprehensive evaluation by using Product-User Interface
Model• Analysis of characteristics of TASK, PRODUCT, USER, and
ENVIRONMENT and its relationships⇒ Systematically applied to make questionnaire and analyze of testing result
Limitations• Just considered qualitative comparisons of evaluated
products⇒ Need to find optimal values of design criteria
• Not considered importance between components or tasks⇒ Applied this concept as weight of scores on the next research
Discussions
29
![Page 30: Wonsup Lee, Kihyo Jung, Heecheon Youedt.postech.ac.kr/homepage_data/publication_proceedings_international/... · Wonsup Lee, Kihyo Jung, Heecheon You . Department of Industrial and](https://reader033.vdocuments.us/reader033/viewer/2022041418/5e1d29605cc3bc37d376a267/html5/thumbnails/30.jpg)
Need faster analysis and easier interpret results⇒ We have developed several types of evaluation systems
Web-based type
Stand-alone type
Future study