eVision Review Project - EngagementMonitoringMonitoring

Simon McLean, Head of Web & IT Support

Information & Data Services.

What is Usability?

Why Bother?

Types of usability testingTypes of usability testing

Usability Testing in eVision

Report findings

What is Usability?

Definition from Wikipedia:

• “Usability is the ease of use and learnability of a human-made object.…” (Physical or Virtual)made object.…” (Physical or Virtual)

• “…Usability includes methods of measuring usability, such asneeds analysis and the study of the principles behind anobject's perceived efficiency or elegance….”

• “…usability studies the elegance and clarity with which theinteraction with a computer program or a web site (webusability) is designed.”

• In E-Commerce - it could save millions based onspeed/transaction times alone.

• In Education – It could improve the ‘Student Experience’. Wego beyond ‘it works functionally’ to perhaps ‘that helped meachieve my objective in the easiest possible way’.

• Staff – Efficiency/Speed of operation for routine administrativetasks + Positive User experience/Intuitive.

• Provide consistency.• Provide consistency.

• Highlight areas which need further work on that wouldotherwise not be picked up on by conventional testing.

• Pick up any real ‘klangers’ before a live launch.

Note the need to separately address Novices (public web access)from Experts (Staff who use for administrative duties regularly)

A selection of methods available:

• Cognitive Modelling (pre-design stage)

• Eye Tracking

• Cognitive Walkthrough

Heuristics (Nielsen)-Visibility of system status-Match between system andthe real world-User control and freedom-Consistency and standards-Error prevention-Recognition rather than recall• Cognitive Walkthrough

• Heuristics (single person can test)

• Think Aloud

• Scenario Based tests

• Interviews

• Grounded Theory

• Observation

-Recognition rather than recall-Flexibility and efficiency ofuse-Aesthetic and minimalistdesign-Help users recognize,diagnose, and recover fromerrors-Help and documentation

E.g. Nielsen ‘F-shaped pattern’ -http://www.useit.com/alertbox/reading_pattern.html

Users first read in a horizontal movement, usually across the upper part of the content area. This initial element forms the F's topbar.Next, users move down the page a bit and then read across in a second horizontal movement that typically covers a shorter areathan the previous movement. This additional element forms the F's lower bar.Finally, users scan the content's left side in a vertical movement. Sometimes this is a fairly slow and systematic scan that appearsas a solid stripe on an eyetracking heatmap. Other times users move faster, creating a spottier heatmap. This last element formsthe F's stem. (Source: Website link above)

See also Wired Magazine for other uses of Eyetracking e.g.http://www.wired.co.uk/magazine/archive/2011/06/start/this-page-has-been-eye-tracked

Usability Testing on eVision

• Testing performed independently of the Portico Services group(objectivity)

• Scenario based testing plus ‘Think Aloud’ Method (Lewis & Rieman)

• Minimalist approach – resource was 1 laptop with Recording Mic,one meeting room. Environment is artificial – it is not the workplace.Camtasia software recorded screen & audio.

• “It takes only five users to uncover 80 percent of high-level• “It takes only five users to uncover 80 percent of high-levelusability problems.” (Jakob Nielsen). Range of administrators testedfor Eng. Mon.

• Permission to record granted (permissions forms used)

• Interview questions combined with the testing process to make itmore fluid.

• Analysis included audio transcription & axial coding (groundedtheory)

• A permissions form is signed to allow recording of data

• These normally follow a ‘script’ that is used during the testprocess. The script should follow a realistic scenario of usage.

• The tester should state what they are doing during the test(think aloud)(think aloud)

• For eVision related with student tests we moved away fromprescribed scripts since the test setup was close enough to areal environment.

• Camtasia Studio software used. Transcribe + Observe.

• Groundedtheory can tellyou a lot aboutthe field ofexpertise,peoples

What about those sites that is good?

They are very logical and very user friendly. I don’t find myself getting frustrated with

them.

How does Evision compare?

It has a function and most of the time I can use it as I need to Sometime I get frustrated

Part of Interview TranscriptAudio Record -> Transcribe

peoplesemotional state,process andother criteriayou wish to lookfor.

• KeyEmotional

Environment

Productivity

with it .

It loggin you out for no good reason I find frustrating I spoke to someone in Portico serv

recently who said it was to do with the number of tabs. And it never occurred to me

before but yes sometimes it won’t acutally tell you why there is a problem.

If you’re trying to get a report and can’t get any data it doesn’t tell you – just says

confined –

(bus objects issue)

I don’t see why there is a timeout.

• Full Report of Findings will be made available on PUGMeetings system

• Measurements: Average time taken to undertake the task

For Engagement Monitoring:For Engagement Monitoring:

• Slow system responses

• Lack of system feedback

• Insufficient Help

• Wording/Labels unclear on some screens (see picture)

• Assumptions about the UI sometimes incorrect

• Increase font size• Produce a standard method of use for buttons – size, location, and wording.

Increasing the size of buttons makes it easier for the user to navigate/move themouse to.

• Labels need to be more consistent and clear (standardise)• Distinguish clearly hyperlinks from buttons in context• Too much text content on screen – can this be minimised into a help link? Some

graphical cues suggested (pictures/icons at key points)graphical cues suggested (pictures/icons at key points)• More feedback to the Administrator for them at key points of submitting form

data. Provide status messages when core actions have been submitted.• All Administrators indicated that it would be useful for a longer timeout, to be

able to keep the system logged in for longer in the background for task switchingto during the day.

• Other issues: Exams and Progression Calculation – although unrelated toEngagement Monitoring, seemed to be causing difficulties and had been raisedwith the Portico Services group. Another was that of reports on a particulardegree course – in which there was insufficient applicant data on the system toutilise report functions.

Contact: s.mclean@ucl.ac.uk

Books on Usability

• Slides Beyond this point are informational/were not shown atthe PUG meeting

Heuristics (Nielsen)Visibility of system statusMatch between system and the real worldUser control and freedomConsistency and standardsError prevention

Nielsen:• http://www.useit.com/papers/heuristic/heuristic_list.html

• http://designingwebinterfaces.com/6-tips-for-a-great-flex-ux-part-5

Error preventionRecognition rather than recallFlexibility and efficiency of useAesthetic and minimalist designHelp users recognize, diagnose, and recover fromerrorsHelp and documentation

• Interviews provide an opportunity to gain advice from expertsin the field. They can also be part of the process for scenariobased testing e.g. Evision Administrators you might want torecord their screen actions.

• Novices are often used for Scenario based testing to revealnew issues– they should be in the target web audience groupnew issues– they should be in the target web audience groupe.g. students.

• You can combine Scenario based tests with interviewtechniques and record the results. Screen capture software(e.g. camtasia studio) can be used to track paths users takelater as well as audio.

• Interviews generate a lot of data that you can examine later ifrecorded. Beware of response to recording/what people willreveal under interview conditions.

• Usability results from a range of methods cover differenttypes of testing from different perspectives.

• Triangulation of results helps confirm findings.

• In practical terms the usability methods employed might berestricted by time and budgetary constraints.restricted by time and budgetary constraints.

• These are largely qualitative measurements. To measure thesuccess of the changes to websites based on usability findingsyou might e.g. measure number of user accesses or perhaps areduction in helpdesk calls or other ways to gain theinformation

• You need to be specific about what you want to addressthrough the usability study.

• Cognitive Walkthrough

• Eye Tracking (e.g.Nielsen ‘F’)

• Cognitive Modelling

Q & A o/ break before continuing?Q & A o/ break before continuing?

Walking through the tasks

• After the task analysis has been made the participantsperform the walkthrough by asking themselves a set ofquestions for each subtask. Typically four questions are asked:

1. Will the user try to achieve the effect that the subtask has?Does the user understand that this subtask is needed to reach the user's goal?Does the user understand that this subtask is needed to reach the user's goal?

2. Will the user notice that the correct action is available? E.g. isthe button visible?

3. Will the user understand that the wanted subtask can beachieved by the action? E.g. the right button is visible but the user does notunderstand the text and will therefore not click on it.

4. Does the user get feedback? Will the user know that they have donethe right thing after performing the action?

By answering the questions for each subtask usability problemswill be noticed.

• Cognitive Modelling examines areas such as Task analysis – the hierarchicalbreakdown of subtasks to achieve goals. It allows us to predict some of thehuman processes which occur.

Measuring task completion times:-Cogtool (Open Source) is a downloadable tool for working out times taken toachieve tasks such as searching a website. It simulates the effect of an expert useron your designs.on your designs.• http://cogtool.hcii.cs.cmu.edu/Changing the Interface design in the prototype and re-modelling can be done toreduce these times.-Cognitive Modelling applied within Cogtool follows KLM-GOMS Task Analysismodel that predicts the times taken to move the mouse, type on the keyboard,think and select items within a screen.-Cogtool is in the design stage for prototyping

-It is also possible to predict or reduce Human Error through website re-design andError Theory – such as Just in Time indicators that point to the final step in asequence of web tasks that have been defined in the interface.

• Usability tests to perform should bebased on Experts (Administrators) andStudents/Other casual users (Novices).

• Suggest that budget and time constraintswould mean a more qualitative approachtaken.

• Define what we wish to improve. Thereare a lot of aspects to examine such asdesign related (use of colour, navigation,design related (use of colour, navigation,visibility of options), navigation, ‘userexperience’….

• Suggest working out some scenariobased tests for both administrators andend users. Use of Screen Recordingsoftware and audio record.

• Note/understand the constraints on howwe can define websites and structurewithin the SITS interface.

• Heuristics are useful as an individualstarting point as this can be done beforethe test participants are available.

Designing Web Usability – NielsenThe Design of Everyday Things - NormanInteraction Design – Beyond HCI – Helen SharpThe Elements of the user experience – Jesse James GarrettDon’t make me Think – Steve Krug