getting started using learning data to improve design
TRANSCRIPT
#CLOwebinar
Sponsored By:
The presentation will begin at the top of the hour.
A dial in number will not be provided.
Listen to today’s webinar using your computer’s speakers or headphones.
Welcome to the webinar!
#CLOwebinar
Tools You Can Use Audio Control
– A dial in number will not be provided.
– Adjust the volume on your computer or headphones
#CLOwebinar
Questions and Handouts
You can submit questions by clicking on this icon here.
You can download a PDF of the slide deck by clicking here.
#CLOwebinar
Sponsored By:
1. May I receive a copy of the slides? YES! Click on the handouts list located on the right
portion of your screen.
2. May I review the webinar recording at a later date?
YES! You may log in again using today’s link to review the presentation on-demand.
3. Is this webinar HRCI certified?
YES! The HRCI certification code will be sent in the same follow up email as the recording
Frequently Asked Ques0ons
Using Data to Improve Learning Design Lori Niles-Hofmann @loriniles | www.lori.ca
Lori Niles-Hofmann
Senior learning strategist
20 years L&D experience (banking, consulting, marketing)
Author: Data-Driven Learning Design
Blog: www.lori.ca
@loriniles www.lori.ca
Diagnose vs. Autopsy
@loriniles www.lori.ca
The Learning Autopsy?
▪ Completions ▪ Level 1-4 Evaluation
▪ ROI
▪ Too late
▪ Costly redesign
▪ Reputational risk
▪ Questionable Accuracy …the room was too cold, the food was not tasty
@loriniles www.lori.ca
In the Classroom
@loriniles www.lori.ca
Digital Distant Relationship
@loriniles www.lori.ca
Paper to Online News
▪ “If it bleeds, it leads” ▪ Frontpage news cycle: 12-24 hours
▪ One metric: newspapers sold
@loriniles www.lori.ca
Digital Changed News Media
▪ Real-time metrics on consumption ▪ Minute-by-minute pivoting
▪ Data revealed insights that challenged assumptions
@loriniles www.lori.ca
Digital Changed News Media
▪ Real-time metrics on consumption ▪ Minute-by-minute pivoting
▪ Data revealed insights that challenged assumptions
@loriniles www.lori.ca
Insights
@loriniles www.lori.ca
Insights
@loriniles www.lori.ca
Prediction vs. Reality: November 28th
@loriniles www.lori.ca
Prediction vs. Reality: November 28th
Data reveals truths that could challenge what we
presume about our learners…
…it also explains the
Kardashians
@loriniles www.lori.ca
Is Marketing a Bad Word?
@loriniles www.lori.ca
Marketing vs. Learning
Both:
▪ Inform their audience with the intent of changing a behaviour
▪ Struggle with short attention spans and competition
▪ Face to face relationship is gone
@loriniles www.lori.ca
Two Approaches
Marketing
• Data analysis for trends
• Content designed against insights
• Monitor and pivot
• Cyclical
Learning
• Performance consulting
• Learning objectives
• Content design and development
• Evaluation
• Linear
Downloads Drop-offs Time of Day
Likes
Shares
Comments
Trends
Mobile vs. Desktop
“Every drop-off, click, or share, is a learner shouting their likes and dislikes. These actions are the eye-rolls, smiles, and arms-crossed from the classroom, simply in digital format” Digital Body Language
@loriniles www.lori.ca
What If…
…we considered the digital body language of our audiences to make design decisions?
…we used data like digital marketers do?
I Predict We will no longer measure learning by completions, but by engagement and
sentiment scores
@loriniles www.lori.ca
Engagement vs. Completion
Company Response?
The manager had completed AHS food safety training with a mark of 96 per cent statement noted Webb had completed AHS food safety training with a mark of 96 per cent
@loriniles www.lori.ca
Data-Driven Learning Design
Respond
Monitor
Discover Insights
@loriniles www.lori.ca
Discover Insights
LMS IT Department Analytics
Most LMS track far more than most organisations explore. Consider: • Time of day/day of
week most active • Mobile vs. Laptop • Search terms • Incompletes
If your company has an intranet, inquire about some of these data points: • Top searches • Failed searches • Pages most visited • Time of day/day of
week most active
Simple internet searches will reveal loads of data on trends. Consider: • YouTube or Vimeo
Analytics • BuzzFeed • Marketing blogs
@loriniles www.lori.ca
Insights Strong preference for video Most active Fridays at noon
Most active Monday morning Strong preference for video
High mobile usage Text format preferred
@loriniles www.lori.ca
Respond
Data Point Action
Top searches, Failed searches
• Promote content aligned with search preferences
• Determine if you need content to fill gap from failed searches
• Use in performance consulting conversations
Pages most visited
• Determine why the pages are most visited and what it tells you about what your learners value
• Promote content on these pages
@loriniles www.lori.ca
Respond
Data Point Action Time of day/day of week most active
• Publish content at these key times for maximum exposure
Drop off rates • Align length of content to be lower than attention threshold
Mobile vs. Laptop • Use this data to make better decisions about media choices
@loriniles www.lori.ca
Data-Driven Learning Design
Respond
Monitor
Discover Insights
@loriniles www.lori.ca
Uncomfortable Truth?
The best performance consulting can be derailed by time, money, and
stakeholder opinion
@loriniles www.lori.ca
5 Pieces of Data to Enhance Performance Consulting
▪ Intranet Search Terms
▪ Time of Day/Week Most Active
▪ Seat Time
▪ Downloads, Views, Likes, and Shares
▪ Mobile vs. Desktop
@loriniles www.lori.ca
Story 1: Talking Heads
▪ Stakeholder wanted to be featured in seven minute video
▪ Similar videos: ~80% drop-off rates by 30 seconds
▪ Testimonial style videos by colleagues had significantly higher viewing lengths
@loriniles www.lori.ca
Story 2: Content Gap
▪ SME and stakeholders want elearning solution on topic X
▪ Feed of intranet searches reveal topic X ranking #152
▪ Decision is made to pivot to build content against top 3 search terms and minor article on topic X
@loriniles www.lori.ca
Cautionary Tale
▪ The average life expectancy in Canada is 81.24 years ▪ Most Canadians like maple syrup ▪ Therefore, maple syrup prevents people from living past 82
@loriniles www.lori.ca
Only the Beginning
“It is not enough to make learning shorter, into micro-sized bites, to appease short
attention spans. The real magic is decoding digital body
language to captivate and engage learners, extending the impact and depth of learning”
Look at your data before designing Go beyond the LMS Be prepared to pivot Use data points to better manage SME and stakeholder conversations Digital Body Language
@loriniles www.lori.ca
Final Thought
#CLOwebinar
Register for the next webinar!
Are You Ready to Embrace Informal Learning?
Wednesday, December 6, 2017
Webinars start at 2 p.m. Eastern / 11 a.m. Pacific
Register for all upcoming Chief Learning Officer Webinars at
clomedia.com/webinars