measuring the impact of digital inclusion at scale (shlb may 2014)
DESCRIPTION
My presentation at SHLB May 2014, describing the evaluation metrics and methodologies we use at TInder Foundation and UK online centres. Ranging from: large scale data collection from our MOOC Learn My Way; online and telephone surveys (in the field 52 weeks a year); and deep dive evaluation and volumetrics. Using these three methodologies we've proven that we've saved the UK Government at least £232.4m (more than $1bn) in the past three years - proving digital inclusion does help the bottom line for countries struggling with recession.TRANSCRIPT
![Page 1: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/1.jpg)
Scale & Impact
Helen Milner, @helenmilner8 May 2014
Tinder Foundation makes these good things happen:
![Page 2: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/2.jpg)
Tinder Foundation
We are a staff-owned mutual and social enterprise
Vision: A better world for everyone through the use of digital technology
Purpose: We make good things happen through digital technology
![Page 3: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/3.jpg)
![Page 4: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/4.jpg)
Local+
Digital+
Scale FreeOptimised for mobile learning
£232.4m
![Page 5: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/5.jpg)
UK online centres: community centres, public libraries, cafes, schools, housing, places of worship, fish & chip shop, …. a barn
![Page 7: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/7.jpg)
Three ways of measuring impact• Learning Data: automated from Learn My Way, learners,
learning activity, viewed at centre and UK-wide level
• Surveys: Learner demographics (online survey) and Impact data (telephone) for progression to learning and employment, use of Government websites, information around confidence and wellbeing. In field 52 weeks a year.
• Further impact evaluation: applying volumetrics to economic impact for Government; regular research projects eg social inclusion and digital inclusion, innovative health and digital outcomes
![Page 8: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/8.jpg)
1. Data
![Page 9: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/9.jpg)
Learner, learning, course data• Learn My Way
• Learners: personal planner, bookmarks, progress, badges
• Centres: totals and per learner• National/Aggregated data
• 135,320 people last 12 months, 12,930 in March• Logins: 141,789• Favourite courses: 94,961 Online Basics, 66,376
email course• Website visits: 1m
![Page 10: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/10.jpg)
![Page 11: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/11.jpg)
2. Surveys: online and telephone
A monthly online learner survey among registered users of Learn My Way
>7000 respondents per year
Among those who opt in from the above, a telephone progression survey one month later
>1200 interviews per year
Measures the learner profile • Including the % of socially excluded learners
Customer satisfaction • Including to what extent users felt the service
helped them learn about computers/the internet or increased their confidence
Evidence of impact and outcomes
• Progression into formal / informal learning • Progression into employment• ‘Quality of Life’: impact on skills and
behaviour • Access to public services online and average
no. of contacts shifted
![Page 12: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/12.jpg)
D2=1Education level: 1 to 4 GCSEs or O Levels at any grade, Foundation GNVQ, NVQ Level 1 or equivalent Education
D2=5 Education level: No qualifications
D6=3 Do you: Live in a house rented from the council or housing association
HousingD6=5 Do you: Live in sheltered accommodation
D6=6 Do you: Have no current home
D5=1 Whether receive benefits: Jobseekers Allowance
Benefits
D5=2 Whether receive benefits: Employment and Support Allowance
D5=3 Whether receive benefits: Income Support
D5=4 Whether receive benefits: Working Tax Credit
D5=5 Whether receive benefits: Housing Benefit
D5=6 Whether receive benefits: Carers’ Allowance
D5=7 Whether receive benefits: Council Tax Benefit
D5=8 Whether receive benefits: Disability Living Allowance
D5=9 Whether receive benefits: Pension Credits
D5=10 Whether receive benefits: Child Tax Credits
D5=11 Whether receive benefits: Other
D4=1 Income: Up to £9,999 Income
D3=4 Current main activity: Unemployed UnemployedS
ocial E
xclusio
n
Definition of Social Exclusion (>7000 surveys)
![Page 13: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/13.jpg)
80% Socially Excluded
![Page 14: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/14.jpg)
B1_C Searched for jobs
Job search & prospects
B1_E Written a CV
B1_G Applied for jobs
B1_I Been for job interviews
B1_K Improved job prospects
B4_A More interesting work
Improvement at work
B4_B More job satisfaction
B4_C Better job security
B4_D Pay & promotion prospects
Employment Progression
B1_J Started full or part time workEntered employment
B1_J Started voluntary work Voluntary work
C1_1 Learning towards formal qualification
Further LearningC1_2 Learning without formal qualification
B1_A Sought/received careers advice
Learning Progression
Em
plo
ym
en
t or
Le
arn
ing
P
rog
res
sio
n
Impact on employment and learning(>1,200 interviews a year)
![Page 15: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/15.jpg)
## ## ## ## ## . . . . .
## . . . . . . . . .
## . . . . . . . . .
## . . . . . . . . .
## . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
## ## ## ## ## ## ## ## ## ##
## ## ## ## ## ## ## ## ## .
## ## . . . . . . . .
## ## . . . . . . . .
## ## . . . . . . . .
## ## . . . . . . . .
## ## . . . . . . . .
## ## . . . . . . . .
## ## . . . . . . . .
## . . . . . . . . .
## ## ## ## ## ## ## ## ## ##
## ## ## ## ## ## ## ## ## ##
## ## ## ## ## ## ## ## ## ##
## ## ## ## ## ## ## ## ## ##
## ## ## ## ## . . . . .
## ## ## ## . . . . . .
## ## ## ## . . . . . .
## ## ## ## . . . . . .
## ## ## ## . . . . . .
## ## ## ## . . . . . .
## ## ## . . . . . . .
## . . . . . . . . .
## . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
91%
66%
35%
95%
Any Positive Outcome:
Quality of Life
Employment Progression
Learning Progression
Base: All learners (1,270)
![Page 16: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/16.jpg)
![Page 17: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/17.jpg)
3. Other
Volumetrics, Financial impact, Deep evaluation
![Page 18: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/18.jpg)
Moving people to online public services• No-one’s ‘spark’ to get digital skills is to interact
with Government online (except to get a job)• After gaining digital skills via UK online centres
(March 2014 data):• 81% visited central/local Government websites• 56% moved at least one (average 3.8) face-to-face or
telephone contact to an online contact with Government• Calculation of £232.4m for 1.2m people (2010 – present)
![Page 19: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/19.jpg)
B1_C Searched for jobs
Job search & prospects
B1_E Written a CV
B1_G Applied for jobs
B1_I Been for job interviews
B1_K Improved job prospects
B4_A More interesting work
Improvement at work
B4_B More job satisfaction
B4_C Better job security
B4_D Pay & promotion prospects
Employment Progression
B1_J Started full or part time workEntered employment
B1_J Started voluntary work Voluntary work
C1_1 Learning towards formal qualification
Further LearningC1_2 Learning without formal qualification
B1_A Sought/received careers advice
Learning Progression
Em
plo
ym
en
t or
Le
arn
ing
P
rog
res
sio
n
Impact on employment and learning (>1,200 interviews a year)
Survey data leads to measurement of economic impact: (1) Entered employment, (2) Started voluntary work, (3) Further learning, (4) Started a formal qualification
![Page 20: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/20.jpg)
![Page 21: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/21.jpg)
Conclusions
![Page 22: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/22.jpg)
A Leading Digital Nation by 2020
With current efforts there will still be 6.2m people in 2020 without basic online skills.
The total investment required to equip 100% of the UK adult population with the Basic Online Skills they need to regularly
use the internet for themselves by 2020 is £875 million.
We suggest investment might be split equally between Government; the private sector, and the voluntary and community sector.
The investment required to ensure a nation with 100% Basic Online Skills
will be £292 million for each sector.
www.tinderfoundation.org/Nation2020
![Page 23: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/23.jpg)
6.2 million people left behind in 2020 and we know who they will be
![Page 24: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/24.jpg)
5% of 16-64s left to reach
In 2020 if carry on at pace and impact of present time (in UK)
![Page 25: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/25.jpg)
![Page 26: Measuring the impact of digital inclusion at scale (SHLB May 2014)](https://reader033.vdocuments.us/reader033/viewer/2022042813/5473e4e9b4af9fa90a8b54a6/html5/thumbnails/26.jpg)
PEOPLE make good things happen
People becoming a volunteer when he’s
93 years old as his mates aren’t getting
the benefits of the web
People helping job seekers to
look for work online when they’ve not had a job for 10 years
People helping patients to use digital in the ward and take it home with them