![Page 1: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/1.jpg)
1
Demonstrating ValueThrough Learning Analytics
![Page 2: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/2.jpg)
2
Overview of KnowledgeAdvisors
Visit our website at www.newwavelearning.co.uk
The Standard in
Learning Analytics
Benchmarking
We help organizations measure and improve the results of their learning programs.
![Page 3: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/3.jpg)
3
Metrics that Matter® Enterprise Scalability
![Page 4: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/4.jpg)
4
Analytics Past, Present and Future
•Strategic Initiatives•Outsourcing•Technology•Consulting•Suppliers
•Talent Measurement•Knowledge Measurement•Recruiting•Compensation•Processes•Leadership
•On-line•Coaching•Mentoring•Blended•On the Job
•Instructor-led
![Page 5: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/5.jpg)
5
revenu
e
employees
ORrevenu
eemployee
s
What Matters to the CEO
![Page 6: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/6.jpg)
6
What is Happening with Measurement?
““Proving Value / ROI of Training.”Proving Value / ROI of Training.”#1 Challenge of L&D Organizations#1 Challenge of L&D Organizations
[[T&D Magazine, Pulse T&D Magazine, Pulse Check January 2006]Check January 2006]
““Communicating and Measuring Results”Communicating and Measuring Results”#1 Challenge in current position as CLO#1 Challenge in current position as CLO
[T&D Magazine, The CLO Path, [T&D Magazine, The CLO Path, February 2006]February 2006]
![Page 7: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/7.jpg)
7
Learning Measurement Resource Allocation
Source: KnowledgeAdvisors Learning Analytics Best Practices Research Study.
81% of resources for measurement today are tied up in administrative activities (collection, aggregation, filtering, reporting) leaving less than 20% for analysis and decision-making. Technology reverses that ratio – 80% on analysis and 20% on
administration.
![Page 8: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/8.jpg)
8
Measurement ‘Reasonable Indicators’
Source: KnowledgeAdvisors Learning Analytics Best Practices Research Study
Independent studies have shown that there is an enormously high cost to data accuracy versus obtaining a reasonable indicator. Organizations should balance when to take the extra time and money to yield highly statistical results. Given how executives make decisions, often times reasonable data provided in a more timely manner outweighs data with more precision delivered in a less timely manner.
In your organization, what best describes the level of accuracy that analytics must have in order to be perceived as useful for information decision-making purposes of senior management?
![Page 9: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/9.jpg)
9
Enter Learning Analytics
Learning Analytics technology helps organizationsunderstand how to better train & develop employees, partners and customers.
![Page 10: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/10.jpg)
10
Analytics Model
A process and toolset enabling learning organizations to measure and improve business results and bottom-line impact.
![Page 11: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/11.jpg)
11
Analytics Process
Data Collection Data Storage Data Reporting
• Standard Post Event ‘end of class evaluation
• Automated Follow Up evaluation
• Constantly gathering• Satisfaction indicators• Learning
Effectiveness Indicators
• Job Impact indicators• ROI predictors
• Mainly collected online• Paper input into system
• Central Database• No need for paper• Secure storage• Self-service access
•Transactional Reports• Aggregate Reports• ROI Analysis• Benchmark Reports• Executive Reports• Ad-hoc Reports• Linkage to Learning Levels
Data Processing
• Aggregation of data• Filtering of data• Convert into metrics• 1000s of queries and
algorithms performed on the data automatically
Metrics that Matter® Learning Analytics Technology
![Page 12: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/12.jpg)
12
KnowledgeAdvisors Analytics Standards
ReactionLevel 1
ReactionLevel 1
LearningLevel 2
LearningLevel 2
Job ImpactLevel 3
Job ImpactLevel 3
ResultsLevel 4 & EstimationIsolation
Adjustment
ResultsLevel 4 & EstimationIsolation
Adjustment
ROI/ROELevel 5
ROI/ROELevel 5
![Page 13: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/13.jpg)
13
KnowledgeAdvisors Analytics Templates
A tool enabling the learning team to conduct more comprehensive business impact and ROI analysis. The tool allows you to store actual business results tied to multiple programs and show trainings impact and ROI.
*Sample numbers for demonstration only.
![Page 14: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/14.jpg)
14
KnowledgeAdvisors Analytics Dashboards
A tool enabling the learning team to conduct more comprehensive actual results tracking and analysis. The tool allows you to store actual results (revenue, cost per student day, test scores) and trend them against goals and prior performance.
*Sample numbers for demonstration only.
![Page 15: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/15.jpg)
15
LMS
Metrics that MatterTM
Upon completion of a learning intervention, LMS sends course, class, learner, manager etc. information to MTM via XML.
At the same 60 day point, MTM also sends a follow up survey to the learner’s manager. The results go back to the MTM database.
MTM recognizes that it has been 60 days since learner completed a course and emails a follow up survey automatically to that learner. The results go back to the MTM database.
The passing of learner course completion information from LMS triggers MTM to send a post-event survey to learner via email. The results go back to the MTM database. ‘Post Event’ instrument
‘Follow Up’ instrument
‘Manager’ instrument
Data Integrations
![Page 16: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/16.jpg)
16
Analytics Reports
*Sample numbers for demonstration only.
![Page 17: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/17.jpg)
17
Benchmarking / Goal Setting
Benchmark across the key performance indicators on your evaluation forms that link back to the 5 levels of learning measurement. Compare the actual performance to goals and to internal or external benchmarks so you properly monitor using a balanced scorecard approach.
![Page 18: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/18.jpg)
18
Measure Impact Easily
The job impact by course indicators clearly show which courses are having the biggest affect on the job. This report stratifies the responses by the percent that fall above or below ranges of performance. Percentages in red are indicators of little or no job impact versus percentages in green represent significant job impact.
![Page 19: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/19.jpg)
19
Who are my top instructors?
Monitor trainers’ performance across key indicators such as learning effectiveness. Then compare each trainer to an internal or external average for more proactive monitoring by measurement.
![Page 20: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/20.jpg)
20
Examples of Macro Learning Constructs
• Operational– Activity constructs (how much we train)
• Performance– Optimization constructs (how well we train)
• Financial– Budget/Fiscal constructs (how much it costs)
• Cultural– Supportive Environment (how conducive is
it)
Note: these constructs are part of KnowledgeAdvisors Metrics that Matter® standards
![Page 21: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/21.jpg)
21
Sample Input Tools to Store and Track Data
Benchmarking
Source: Metrics that Matter®
A template to add, edit, and manage your dashboard metrics. This is an example of the operational dashboard metrics added to a web-based template. Metrics can be added manually or via a feed from other systems (HRIS, LMS, financial)
![Page 22: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/22.jpg)
22
Sample Dashboards to Analyze Metrics
A report showing actual results over time with a color coded analysis to compare variances against prior periods and goals/benchmarks.
Source: Metrics that Matter®
Operational Performance
![Page 23: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/23.jpg)
23
Enterprise Solutions Capabilities
Enterprise Learning Organization
Employee Training
Americas Region
Partner Training
Customer Training
Europe Region
Metrics that Matter can organize and aggregate data to match an enterprise learning organizational structure
Asia RegionLeadership
ProgramSales
ProgramTechnology
Program
Aggregate
Data
Segmented by Program
Segmented by Audience
Segmented by Region
![Page 24: 1 Demonstrating Value Through Learning Analytics](https://reader035.vdocuments.us/reader035/viewer/2022062714/56649d415503460f94a1c9c4/html5/thumbnails/24.jpg)
24
Contact Information
New Wave Learning
Tel: +44 (0)1977 628980
Fax: +44 (0)1977 668519
www.newwavelearning.co.uk