Evidencing Digital Inclusion in UK

Dr Alice Mathers, Online Centres Foundation

5th Social Digital Research Symposium

The key issue…

‘there is little incentive for digital inclusion practitioners to critically evaluate their

practices in a meaningful way’

(Seale and Dutton, 2012)

Evidence from practiceTop down interventions and national programmes

Bottom up community initiatives and innovation

Differing approaches to measuring impact

on and ofDigital Inclusion

Existing evidence of impactDigital Unite

• Providers of free learning content 

• Online community forum.

• Home visit tuition through network of local Digital Unite Tutors

• Training for organisations through community learning programmes,

staff training programmes and digitally-orientated customer care

services. 

• Structured skills development through Digital Champion ITQ (DC

ITQ)

• Advocacy

Existing evidence of impactDigital Intervention:

• Get Digital, Digital Unite (2010-2012)

Programme focus:

• Digital inclusion through comprehensive,

structured learning programme for

residents in sheltered housing, working

with staff, landlords and wider community.

Evaluation measurement:

• Mixed method: quantitative (online and

paper surveys, baseline, progression,

outcome), plus qualitative data from 12

case study sites.

Existing evidence of impact

Impact: • Skills and confidence: 57% of staff have increased confidence in use of

ICT, 83% of residents have a more positive attitude towards computers

and the internet, 88% likely to use a computer/the internet for email,

phone calls, search for information, access public services and use

social networking sites.

• Reduced isolation and loneliness: 50% find it easier to keep in touch,

42% easier to meet new people, 42% contact with family and friends

• Enhanced health and wellbeing: Almost 20% of residents now look

after themselves better and know more about their health.

Evaluators: NIACE (2012)

Existing evidence of impactUK online centres

• National coverage through a network of 5,000 centres

• Longitudinal daily DI data collection + daily surveys

• Products, support, advocacy, research

• 4 specialist networks: – Disabled people

– Carers

– Older people

– Into work

(Image source: Helen Milner, Nov 2012)1,089,402 people as of 24th April 2013

Existing evidence of impact

Digital Intervention:

Social Impact Demonstrator Project (2007-8)

Programme focus:

• Partnership working through UK online

centres to reach socially disadvantaged

people and engage them in ICT activities.

Evaluation measurement:

• 3 stages of data collection: baseline,

progression and outcomes.

• 4 focus groups and 8 individual semi-structured

interviews

• 20 individual project evaluations

Existing evidence of impact

Impact:

• Cost efficiency: 12,234 people were engaged at a cost of

£163 per person.

• Skills & confidence: 60% now happy using computers and

internet, 70% now confident.

• Employability: 40% progressed in terms of training and

employment etc.

• Social capital: 39% met up with new friends and 32%

volunteered.

Evaluators: Ipsos MORI (2011)

Existing evidence of impactAge UK

• Digital Inclusion Network of over 200 members

• The Network is a UK-wide membership programme managed and

delivered by Age UK in partnership with Age Scotland, Age Cymru

and Age NI.

• Best practice: developing and share best practice approaches.

• Advocacy: working with key stakeholders to gather intelligence,

provide a channel for discussion on digital inclusion issues at a local,

regional and national level.

• Communications: regarding funding‚ information & advice,

publications, networking opportunities, events and resources.

Existing evidence of impact

Digital Intervention:

• Reach for IT, Age UK (2010-2011)

Programme focus:

• Tackling the digital exclusion of older

people in residential care homes

through partnership working with five

community-based organisations

Evaluation measurement:

• Qualitative practice reporting

• Social Return on Investment (intended)

Existing evidence of impactImpact (organisational and individual):

• Engagement: 20 care homes, 34 volunteers delivering IT

training sessions to 234 residents.

• Communicating DI benefits: offering a new services

increases profile, reputation and attractiveness to potential

residents and their families.

• Meeting consumer demand: for personal development of IT

skills in IT-friendly environments.

• Quality of life: creation of ‘happier homes’ and better

working environments for staff.

Academic Evaluation

• Sus-IT: Loughborough University– Participative research into older people in the

digital world

• University of London, Imperial College, evaluation of NHS Choices– Measurement of financial benefits to

Government of digital interventions

Practical concerns• The limited extent to which impact of digital practice

is currently being evidenced.

• That evidencing impact is not considered a priority (due to lack of political and financial support).

• That smaller organisations delivering digital inclusion on the ground do not have the capacity or expertise or money to carry out robust evaluation of impact.

• Funders pay lip service to project evaluation – no support up front for projects

• That approaches and outcomes of evaluation do not always ‘empower’ the most digitally excluded individuals.

Current and New Approaches

1. Social Impact evaluation of Community Capacity Builders

• Online Centres Foundation

2. Evaluation of ‘Get Connected’ • Digital Unite

3. Regular online reporting of ‘Get IT together’ • Citizens Online

Valuing Evaluation

What Works Centres

• Launched: March 2013.• Aim: to improve the evidence used in decision making

across a number of key policy areas.• Approach: creation of specialist independent research

centres• Partnership working and accountability: with the ‘Alliance

for Useful Evidence’, a network of over 1,000 members who champion evidence, the opening up of government data for interrogation and use, alongside the sophistication in research methods and their applications. 

Suggested areas for development

1. Robust evidence regarding the social and economic value of digital inclusion.

2. Methodologies that may be utilised by non-academic organisations to capture and communicate impact.

3. Political support behind the importance of evaluation as a means to develop effective practice.

4. Legitimising communication of ‘failure’ in approaches as a learning tool: agile development and iteration

5. LONG TERM GOAL? Creation of a single digital inclusion framework ..…

Thank you

Contact: alice.mathers@ukonlinecentres.com