Christopher ParkerHow VGI Influences Usability

Past ResearchHow We Got To Where We Are Now

Usability of VGITHE IMPACT OF VGI ON USER JUDGEMENT

Study Aims

• The extent to which the content of the information within the mashup affects the users’ judgements on the mashup.

• The extent to which the user judgements on the information influence the overall usability and system acceptance of the mashup.

• The extent to which the facets of the users’ judgements which may be harnessed to optimise the design of future mashups combining both VGI and PGI information.

What We Are Trying To Work Out

Study RationalWhat We Are Really Getting At

Presenting VGI Alongside PGI in a mashup The Utility of VGI,

what it may offer to users that PGI does not

The Style of VGI, a different way of presenting information

Telling Participants Their Mashup Contains VGI The degree to which

current perceptions of information from amateur volunteers influences perception of neogeography

09/04/2023

Study Rational

Quality Authority

Good Trustworthy

Accurate Credible

Current Reliable

Useful Official

Important Authority

Dependable Variables

Usability

Usefulness

Clear

Efficient

Satisfaction

User Judgements of Online InformationRieh (2002)

System Acceptance ModelMaguire (1998)

MethodologyTHE IMPACT OF VGI ON USER JUDGEMENT

Methodology

• Need to critically evaluate information• Can be bounded in as a simple user group• Make use of PGI as well as generating and utilising

VGI• Critically assess information for risk management• BUT be normal people doing normal things.

Selection of a Participant Community

• Selection Criteria• Age 18-65• Full time wheelchair user• Non cognitive or Sensory Disabled• Non-house bound• Other (regular ethics)

Methodology

• Part 1• Observation (6 wheelchair users)

• VGI: Generate Data Set• Literature Review

• PGI: Pool data

• Part 2• Mono Methods• Experimental• Online (101 participants)• Four Groups – with a unique combination of

contents of the map and information about the map.

Two Parts of the Study

MethodologyFixed Variables and Groups

  Information Presented

In Map

PGI PGI + VGI

Participant Told

What Map

Contained

PGI Group 1 Group 3

PGI + VGI Group 2 Group 4

Part 2Experiment Website: Free Traveller

Study Rational

Quality Authority

Good Trustworthy

Accurate Credible

Current Reliable

Useful Official

Important Authority

Dependable Variables

Usability

Usefulness

Clear

Efficient

Satisfaction

User Judgements of Online InformationRieh (2002)

System Acceptance ModelMaguire (1998)

Part 1Collecting Information

Part 1Data Collection Routes

Part 1VGI Data Collection

Part 1PGI Data Collection

Part 2The Experiment

Part 2www.FreeTraveller.co.uk

Part 2

Quality Authority

Good Trustworthy

Accurate Credible

Current Reliable

Useful Official

Important Authority

Assumption Testing

Usability

Usefulness

Clear

Efficient

Satisfaction

User Judgements of Online InformationRieh (2002)

System Acceptance ModelMaguire (1998)

Part 2

Quality & Authority

Good Trustworthy

Accurate Credible

Current Reliable

Useful Official

Important Authority

Assumption Testing: Factor Analysis

Usability

Usefulness

Clear

Efficient

Satisfaction

User Judgements of Online InformationRieh (2002)

System Acceptance ModelMaguire (1998)

Part 2

Quality & Authority

Good Trustworthy

Accurate Credible

Current Reliable

Useful Official

Important Authority

Assumption Testing: Collinearity

Usability

Usefulness

Clear

Efficient

Satisfaction

User Judgements of Online InformationRieh (2002)

System Acceptance ModelMaguire (1998)

ResultsTwo-Way MANOVA

ResultsQuality & Authority

09/04/2023

• Information Presented in Mashup• 101 Participants Significant

• F (9, 89) = 2.99• p = .004• Wilks’ Lambda = .77• ηp2 = .232

• Information Told to Participants• 303 Participants (Estimation)

• F (9, 392) = 3.48• p = 3.5E-7 > .00268• Wilks’ Lambda = .93• ηp2 = .074

• No Statistically Significant Interaction

Results

09/04/2023

Currency Importance Usefulness Credibility Authority Goodness Reliable Accurate Official0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0.11

Information Presented

Information Told

Quality & Authority

Small Effect Size

Medium Effect Size

ResultsQuality & Authority

09/04/2023

ResultsQuality and Authority

• Presenting VGI alongside PGI• Most notable influence is on currency, as

perceived by the inclusion of VGI within the data set and should be considered

• Importance, Credibility, Usefulness and Authority are all positively influenced, but how much consideration needed is debatable

• Telling People their mashup contains VGI alongside PGI

• Most notable influence is on Authority• Usefulness and Credibility worth considering• The importance of there variables and effect

of being told is limited

ResultsSystem Acceptance

09/04/2023

• Information Presented in Mashup• 303 Participants (Estimation)

• F (3, 297) = 4.67• p = .003 > .00851• Wilks’ Lambda = .96• ηp2 = .045

• Information Told to Participants• 404 Participants (Estimation)

• F (3, 398) = 4.38• p = .005 > .00851• Wilks’ Lambda = .97• ηp2 = .032

• No Statistically Significant Interaction

Usefulness Efficiency Satisfaction0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0.11

Information Presented

Information Told

Results

09/04/2023

Quality & Authority

Small Effect Size

Medium Effect Size

ResultsSystem Acceptance

• Presenting VGI alongside PGI• Most notable influence is on usefulness

• May result from more info = more utility

• Telling People their mashup contains VGI alongside PGI

• Most notable influence is on slightly increased levels of satisfaction• May be due to idea of social interaction?

ConclusionsWhat does this mean?

Conclusions

• Presenting VGI Alongside PGI• Reinforces the mantra of right data for right

task• Does VGI cover more ground than PGI?

• Judgements of Quality & Authority increased along with Usability

• No negative Influence!

• Telling Users They Are Using PGI• Personally held biases have a measurable

effect on Quality & Authority and overall Usability

• Effects are relatively minimal and attention may be best given to other factors

• No negative influence!

From Research

Conclusions

• VGI did not change the game dramatically• Though currency is a very interesting outcome

• In every case VGI enhanced the user perceptions and usability

• VGI is not necessarily a game changer in normal situations, but instead, if applied correctly is polish.

• Wont make a bad system good• But has potential to enhance what is already

good, making it more satisfying.• Little point polishing low end mashups, but high

end a small amount of polish is VERY important

For Design