interact 2013 klm fa-v1

1

KLM Form Analyzer: Automated Evaluation of Web Form Filling Tasks using Human Performance Models

Interact 2013, Cape Town, 2-6 September 2013

Christos Katsanos | [email protected] Karousos | [email protected] Tselios | [email protected] Xenos | [email protected] Avouris | [email protected]

Motivation & Purpose Usability of interactive forms is a critical aspect

of Web UX

Form filling is a data entry task Task efficiency is important

User efficiency affected by many factors e.g. fields’ size/position/type, interaction

device/strategy, age, typing expertise

Lack of easy-to-use, efficient and flexible simulation tools for practitioners

Interact 2013, Cape Town, 2-6 September 2013 2

KLM-FA Algorithm

Our approach: KLM-Form Analyzer (KLM-FA)


Web Form

Modeled User Profile (age, typing expertise)

Interaction Scenarios (devices, initial state, keystrokes, fields to fill)

Modeling parameters (rules, values, adjustments)

or

Web Forms

Parsing Module(forms & elements identifier)

Analysis Module (Keystroke Level Model, Fitts’ Law)

Task-time prediction

Trace model steps

Simulated actions

Save/Export Project

Output

KLM-FA tool: Main Interface

Form URL

Preview Browser

Synchronized Views Results Panel

Point time calculations

Analysis preferences (modeled user, interaction

scenarios)

Analysis parameters

Mass-scale evaluation

Trace step-by-step modeling

5Interact 2013, Cape Town, 2-6 September 2013

Flexible: Test different interaction scenarios

Form Fields to be filled

Modeled user profile (e.g. old & poor typist)

Interaction strategy (e.g. tab-based navigation)

Initial state (e.g. cursor position in first field after form loads)

6Interact 2013, Cape Town, 2-6 September 2013

Flexible: Employ rules & change modeling parameters

7

Efficient: Mass scale evaluation (saved as XML file)



Efficient: TextField names and default keystrokes mappings

Research-based Defaults

9

Studies


1) Validation study: KLM-FA VS User testing

2) Case Study1: Benchmarking with KLM-FA

3) Case Study2: Form redesign with KLM-FA

10

Validation study: Methodology


KLM-FA predictions VS User testing data

6 form-filling tasks (3 forms x 2 interaction strategies) 3 signup forms of social networking sites: Facebook | Myspace | Twitter 2 interaction strategies: mouse-only* | keyboard only

* except for input in text fields

15 participants (12 male, mean age 27, mean of 42wpm#)#as measured by a typical typing speed test

10 trials per task (to allow users to reach skilled performance)

In-house developed software for experiments (interaction logging, conditions counterbalancing, instructions)

11

Validation study: Results (1/2)


30.727.3

33.229.6

22.5 23.1

35.3

25.6

39.1

28.3 26.424.2

0

5

10

15

20

25

30

35

40

45

FB_mouse FB_keyboard MS_mouse MS_keyboard TW_mouse TW_keyboard

Task

tim

e (s

ec)

KLM-FA vs User testing

User testing

KLM-FA

*Error ba rs i n user testing data represent 95% confidence interva ls

12

Validation study: Results (2/2)


Signup

form

Interaction

strategy

KLM-FA

Error rate

Facebook Mouse-based -14.9%

Facebook Keyboard-based 6.1%

Myspace Mouse-based -17.6%

MySpace Keyboard-based 4.5%

Twitter Mouse-based -17.5%

Twitter Keyboard-based -4.7%

Error rate=Users ′ time− KLM−FA time

User s′ time

KLM-FA tended to overestimate (16.7% on average) task time in mouse-based interaction underestimate (5.1% on average) task time in keyboard-based interaction

Within the 20% margin of error (Card et al. 1980; Ivory, 2001)

Two Pilot Case Studies Benchmarking study

Time to register to a social networking site?

16 popular sites (e.g. Facebook, Google+)

30 mins to do it (<2 mins per signup form)

Mouse-based

Keyboard-based

Mean 33.64 23.25

SD 13.54 11.46

Min 14.12 (LinkedIn)

5.92 (LinkedIn)

Max 62.25 (Flickr)

46.48 (Google+)

Form redesign Webmaster unfamiliar with HCI models & our tool New design: decrease in signup time of 55.8% (mouse) and

60.6% (keyboard) Comments: “intuitive and easy to use”, “valuable asset to

effective form design”

14

Discussion - Advantages


Mass scale evaluation/benchmarking

Cost-effective (time & resources)

Can be used early in the design/evaluation cycle

Simple to learn and apply

Can be used for educational purposes

Possibility for wider adoption

Future Research

Conduct additional validation studies

Teaching HCI with KLM-FA & students’ learning

outcome

Add enriched models of KLM and Fitts (e.g.

stochastic model of errors)


Summary

Proposed an approach that predicts the actions and time required for form filling tasks

Case studies depicted the usefulness of the proposed tool

Free + Fast + Easy = Possibility for wider adoption

Complementary to user-based methods


Tool free to download and useTry it out! – Questions?

http://klmformanalyzer.weebly.com


http://quality.eap.gr

http://hci.ece.upatras.gr

interact 2013 klm fa-v1

Education