innovations in metrics of learning lingos meeting london meeting 18 th september 2009 anne kristine...
DESCRIPTION
The Difference between Data, Information and Knowledge Data discrete, unorganised facts. Information data that is organised into groups or categories which can alter the way a person perceives something. Knowledge familiarity, awareness or understanding gained through experience or study. ‘Because knowledge is intuitive it is difficult to structure, can be hard to capture on machine and is a challenge to transfer.’ Definition: IO Council (2001).TRANSCRIPT
Innovations in Metrics of Learning
LINGOS meetingLondon Meeting 18th September
2009Anne Kristine Arbon(former Save the Children UK)
Arbon Consulting
Definition of Learning• Learning is the process
by which individuals, teams, organisations and groups of organisations create, transfer and use knowledge in order to achieve positive change and realise their goals.*
*Definition: Learning by Field Level Workers, ALNAP Review of Humanitarian Action in 2003, Publication date: 2004 (http://www.alnap.org/resources/studies/knowledge.aspx)
The Difference between Data,Information and Knowledge• Data discrete, unorganised facts.• Information data that is organised into
groups or categories which can alter the way a
person perceives something.• Knowledge familiarity, awareness or
understanding gained through experience or study.
‘Because knowledge is intuitive it is difficult to structure, can be hard to capture on machine and is a challenge to transfer.’
Definition: IO Council (2001).
How does learning happen?
Information Knowledge
Why do we measure?
• ROI (Return on Investment)• Mission critical
• What kind of data do we use to measure impact of our learning?- How do you know the data is relevant?- Who says so?
What should we measure?
• Process OR• Results OR• Both• Or?
Kirkpatrick 4 levels of evaluation
Example of ROI
% ROI = (benefits / costs) x 100
payback period = costs / monthly benefits
Duration of training33 hrs
Estimated student numbers 750
Period over which benefits are calculated 12 months
Costs
Design and development £40,930
Promotion £4,744
Administration £12,713
Faculty £86,250
Materials £15,000
Facilities £40,500
Students £553,156
Evaluation £872
Total cost £754,165
Benefits
Labour savings £241,071
Productivity increases £675,000
Other cost savings £161,250
Other income generation £0
Total benefits £1,077,321
Return on investment 143%
Payback period 8 months
Deciding on measurements
• Your trip here:• Process: length of travel, time spent on travel,
mode of travel, comfort of travel, cost of travel• Result: arrival on time, harassed, happy, tired,
annoyed
(Length of travel – time spent on travel)* cost of travel Proximity of other commuters=
What information do we have available?
Metrics @ Save the Children
Metrics for global Core Learning:No. of staff uploadedNo. of people who are in progress/completed coursesSurveymonkey results of e-modules (incl. qualitative data)No. of face to face core learning workshops heldNo. of participants on core learning workshops
Where does the money go?
• "...international staff accessed approximately 10 times more explicit knowledge assets from their organisations than the national staff. International staff also attended co-coordinating structure meetings at approximately 10 times the rate of national staff..." ALNAP
LDPSenio
r mgt dev
Management
development
Core learning
200
300
200
5000
Total investment since 2004 from HQ
No of staff Training programmes
+ 3 % of country programme salary budget
£60.000
£250.000
£200.000
£300.000
How can we use metrics to change where the money goes?
What metrics will your organisation use to measure learning solutions?
