data-collection techniques
DESCRIPTION
Data-collection techniques. Contents. Types of data Observations Event logs Questionnaires Interview. Two types of data. Usability Measures. Use/problem Descriptions. Observations. Observe in real-time or from video recordings Can be used both for measures and descriptions. - PowerPoint PPT PresentationTRANSCRIPT
Data-collection techniques
Contents
Types of data Observations Event logs Questionnaires Interview
Two types of data
Usability MeasuresUse/problemDescriptions
Observations
Observe in real-time or from video recordings
Can be used both for measures and descriptions
Usability MeasuresUse/problemDescriptions
http://www.mangold.de/english/intoverview.htm
Measures from observations
Particularly suitable for measuring effectivity and efficiency
How many tasks were performed?
How quickly was the task performed?
Were the goals achieved?
How many errors were made?
Effectivity
Efficiency
Use/problem descriptions from observations
Description of breakdowns Describe situations when there is a
”breakdown” in the users interaction with the product
Description of use Describe how the user actually used the
product
Event logs
Logging of the user’s interaction with the product
Typically collected automatically Can be used both for measures and
descriptions
Usability MeasuresUse/problemDescriptions
http://zing.ncsl.nist.gov/WebTools/WebVIP/overview.html
http://zing.ncsl.nist.gov/WebTools/FLUD/example-pretty.html
http://zing.ncsl.nist.gov/WebTools/VisVIP/overview.html
http://zing.ncsl.nist.gov/hfweb/proceedings/etgen-cantor/
http://zing.ncsl.nist.gov/hfweb/proceedings/etgen-cantor/
http://zing.ncsl.nist.gov/hfweb/proceedings/etgen-cantor/
http://www.mangold.de/english/patternexample.htm
http://zing.ncsl.nist.gov/WebTools/VisVIP/overview.html
Measures from event logs
Suitable for measuring effectivity and efficiency
Effectivity
Efficiency
Use/problem descriptions from event logs
Visualise interaction patterns (Quantify use)
Questionnaire
Self-made or validated Most useful for collecting measures
Usability Measures
Self-made or validated?
Designing your own questionnaire allows you to address issues of particular interest; however, it is difficult to create good items
Validated questionnaires (e.g. SUMI) have typically gone through extensive testing. Thus, test items are generally good, but they may not address the issues you are particularly interested in.
http://www.acm.org/~perlman/question.cgi?form=NAU
http://www.acm.org/~perlman/question.cgi?form=ASQ
http://www.acm.org/~perlman/question.cgi?form=PUEU
Measures from questionnaires
Can be used for measuring all attributes, but is particularly suitable for measuring satisfaction; we are interested in the user’s subjective opinion
Satisfaction
Interview
Structured, semi-structured, open Most useful for use/problem
descriptions
Use/problemDescriptions
Use/problem descriptions from interviews
Information about previous experiences and knowledge that might explain the user’s performance
Obtain explanations: the user’s preferences Breakdowns errors
Student tasks
• What data-collection techniques do you think are suitable for testing your product?
• What do you believe are the strengths and weaknesses of the technique?