An everyday guide to learning analytics

How to use data to decode the digital body language of your audience and design elearning that worksBy Kirstie Greany and Lori Niles-Hofmann

IntroductionImagine if you knew the winning formula for creating great digital learning design that engages thousands of users.

News agencies, media companies and marketing teams have all been using data to inform decisions on content for years. Now, it’s L&D’s turn.

Data and analytics are critical for designing digital learning that works. To engage your audience, drive up usage and get them back for more, you need data on your side to help shine a light on what’s working and what isn’t.

But fret not. Tapping into data to help you design winning strategies doesn’t require a lab coat or complex spreadsheets with Einstein-style equations. In this guide, we’ll share tips and tricks for tapping into typically available data and drawing insights that can make a real success story for your project – stats you can play back to managers and peers and use to create an even more engaging project next time.

Use this guide to sprinkle some science onto your creative approaches and design from a place of knowledge, not assumption.

Lori Niles-Hofmann Kirstie Greany

www.elucidat.com

@loriniles @elucidat

ContentsWhy data is needed in digital learning design

Where to get data from

How to analyze data to draw meaningful insights

How to make data part of your design toolkit: A 3-stage strategy

P3

P6

P7

P12

Data - a story full of surprises

The key is not to think about data as numbers, but as a story. Just like storytelling has twists and turns, data can challenge your assumptions and surprise you. And surprise us it does.

Why data is needed in digital learning design

www.elucidat.com

Across the globe, lunchtimes are the peak time for elearning usage. Sandwiches with a side of digital learning makes for a great meal combo. But there’s a bigger window on Mondays and Tuesdays, with people accessing their digital learning between 10am and 2pm on these days. So, if you were thinking of launching a brand new piece of content, it’s probably a good bet to publish it around those times.

For those accessing digital learning on their mobiles, we uncovered that 9pm is the most popular time. Who’d have guessed that this creeps up to 10pm in the UK? Digital learning as bedtime reading is a scary thought, but the point we’re making is that analyzing data opens up a lot of fascinating insights you couldn’t have guessed.

Hottest usage times

Some global stats from Elucidat

When we uncover the data trails left by Elucidat’s 3 million users across the globe, we find some interesting stats. For example, the global average session time for accessing elearning content is 15 minutes. However, this increases to 30 minutes for the USA and Germany. Who’d have guessed? That’s some serious learning dedication going on – or, hopefully, some really great designs that are keeping users hooked for longer. Diving in to find out what approaches are keeping users hooked tells a tale for future designs.

Who’d have guessed that the birth of a Prince would be trumped by a story about a man walking into McDonalds with a horse?

www.elucidat.com

So what?

Digital body language

A feature of traditional classroom learning – where the student and the trainer are in the same environment at the same time – is that the trainer can take cues from the audience to see whether they are engaged or not. The students will give the trainer signals, both consciously and subconsciously. If the trainer notices that the learners are disengaged, they can adapt their approach to try and increase engagement.

The amazing thing is that these clues are still very much present in the digital environment. Look for drop-offs in learners, a reduction in clicks or fewer likes and shares. These are all indications of low engagement, simply in the digital format.

Reading data is like reading the digital body language of your audience.

“Every drop-off, click or share is a learner shouting their likes and dislikes. These actions are the eye-rolls, smiles and crossed arms from the classroom, simply in digital format.”

Lori Niles-Hofmann

Is getting learners’ attention the be all and end all?

No, but in the digital age, a learner is bombarded with a wealth of information that is constantly fighting for their attention. This includes instant messages, tasks, email pop-ups, alerts and notifications. Combined with the fact that the average person will turn to Google as their #1 option to find a solution to a problem, your content has to stand out from the crowd. Engaging audiences is a major hurdle. If your content doesn’t connect and engage with individuals, it’s unlikely to be talked about and is highly unlikely to drive change.

On the flip side, reading digital body language, listening to feedback and taking a more performance consulting-based approach increase the chances of your content making an impact.

We’re talking data-driven learning design

Imagine if before you began designing a brand new digital learning experience, you knew the formats your audience preferred and the optimum video length for expert videos – Or how about a proven method that get learners to hop from one topic to another and choose to extend their overall session time?

Data puts strategies in your pocket you might not otherwise have. It enables you to put away the crystal ball and invest in ideas that have a greater chance of paying off. That doesn’t mean you stop being creative – quite the opposite. Instead, you can channel your creative designs to solve the problems and opportunities that data throws up, stick with trends that are working, and then use data to tweak and refine to get the very best results while your project is live.

www.elucidat.com

www.elucidat.com

Where to get data from

So, you’re sold on the idea of using data to inform important decisions about your elearning content… but how are you going to collect this information?

The easiest option is to use an elearning authoring tool that has a live dashboard built into its features. However, if that’s not available to you, other possibilities for gathering data include:

• Combining your Google Analytics with your LMS

• Talking to your IT team

• Sourcing the top 10 search items on your intranet

• Surveying your learners

We’re not talking about rocket science or “big data” here. Using data can mean just tapping into simple streams, observing the information and using it in clever ways.

If you’ve chosen to survey your learners, couple what your users say they want and need with the real digital stories from the data. These stories show what learners actually prefer, and what they’re actually doing when it comes to their learning.

One of Elucidat’s built-in data dashboards.

www.elucidat.com

How to analyze data to draw meaningful insights

When collecting data, there’s an almost endless list of different stats you can look at. But what exactly do they all mean, and what insights can you draw from each of them? If you’re going to deduce anything useful, you need to be able to analyze your analytics.

Using Elucidat’s dashboards, here are some suggestions for how to make a range of typical metrics meaningful for you and your elearning content design.

#1 Top level stats

Visits

Are the number of visits to your content going up? When does your elearning have peaks and troughs? How can you feed this into your deployment plan?

Look at session times vs. number of users, and monitor if people are coming back for more. This could indicate on-the-job use, and it’s great if this is the case – or perhaps they are biting off one short topic at a time?

Completion rate

In its own right, this doesn’t mean anything other than that someone has gotten through to the final page. So, take it with a pinch of salt and instead ask yourself:

• Are your learners going through a fair number of your pages?• Are there certain pages that are most popular?• Are they liking your content and sharing it with others?

1

1

2

2

Audience Usage and Location

Are you reaching your target audience(s)? Do you need to focus your marketing strategy on a specific location to boost numbers?

3

3 3

www.elucidat.com

How to analyze data to draw meaningful insights

Device

If mobile usage is lower than you would expect it to be among your learners, ask yourself why this could be. Is the design just not mobile-friendly? Or is it because despite the BYOD policy, users have to use their own data allowance?

Authoring time

If you’re using a collaborative authoring tool, with which multiple editors can contribute to the content, tracking authoring time helps you measure ROI and assess if there are ways to up efficiency going forward.

Session time

Are you getting higher session times than average? Everyone’s talking about making content shorter. But it’s not just about dividing an hour-long course into 5-minute segments… it’s still about designing really good content! For good content, learners will spend a good amount of time using it. You need to understand the types of content that warrant this and compel learners to increase their seat time.

4

6

5

“It is not enough to make learning shorter to appease short attention spans. The real magic is decoding digital body language to captivate and engage learners, extending the impact and depth of learning.”

Lori Niles-Hofmann

4

56

www.elucidat.com

How to analyze data to draw meaningful insights

#2 Page analysis

Losing people

Look for pages where the number of users starts to tumble. Why is this happening? Is there a problem with usability or navigation? Perhaps the page links are broken? Or maybe the content on that page is just not relevant to your learners.

If possible, couple your page audit with feedback from users to make sure you’ve got to the root of the problem. Pulling in feedback from your users will help you combine hard data against what your audience says. There might not be a major issue, just a small fix needed.

Popular pages

Which pages are most popular? Analyze why. Can you mimic the approach elsewhere? Do you need to bump those pages higher or spin off some standalone resources that focus on just them?

Branches

If you’ve created personalized branches of content, where different users go down different pathways depending on how they answer questions or what they select from menus, a page progress dashboard will reveal which pathways are most visited.

What does this tell you about the needs, choices and mindsets of your audience? Did they choose surprising options in a scenario? Is there a difference across the different audiences? Dig deeper to find out.

1

1

2

2

A page progress dashboard, like this one from Elucidat, will tell you how much of a page your audience has seen. This is useful to see where your learners are dropping off and identify what’s most popular.

www.elucidat.com

How to analyze data to draw meaningful insights

#3 Question responses

Most LMS track how users have answered assessments. Elucidat tracks responses to any questions – scored or not – and also the audiences’ responses to polls.

Ask yourself:

Are there any killer questions users should be getting right, but aren’t? Are there any questions everyone is getting right, but you thought they wouldn’t? Go back to your content and consider bolstering complex points with further explanations and removing or editing content that’s just too easy.

1

1

2

2

Use polls

You can also pose poll questions to survey your audience around their current behaviors (for a given workplace scenario) and opinions. Look at the results to gauge if you need to address mindset or cultural issues ahead of skills-training, for example.

www.elucidat.com

How to analyze data to draw meaningful insights

#4 Surveys

You can also consider pulling in data via surveys. Gather feedback on:

• Your content• What users need to help them perform better (i.e., their current headaches)• What learners would like from their learning

Embracing data and utilizing the insights you draw from it means taking on an iterative design mindset, where you constantly refine and improve what you do based on what you find out to increase engagement. The good news is that leading authoring tools enable you to easily and quickly make updates to content that’s already live, without disrupting learners’ experience.

To make data part and parcel of your digital learning design toolkit, use the “Diagnose, Respond, Monitor” cycle:

www.elucidat.com

How to make data part of your design toolkit: A three-stage cycle

3. Monitor

Continually monitor the performance of your content and take an iterative approach

to improve it. This method is far more effective and will yield far

better results than if you wait to measure your content or avoid making any

adjustments at all.

“Cultivating your digital learning is more than doing, waiting then evaluating. To be the best, stay ahead, meet customer expectations and needs and hit those important performance targets, you need to read your audience, uncover problems, spot opportunities and be reactive.”

Kirstie Greany

1. Diagnose insights

Look at the data to see the behavior of your learners and the performance

of existing content before you even start designing or choosing a topic

for your project.

2. Respond

Assess the insights you’ve gained and design your content

in a way that responds directly to them. Frequently gather data on

performance, instead of waiting a long time for your learners to work

their way through your content.

Learn more...

Check out Lori’s latest thinking

www.elucidat.com www.loriniles.com

Read Elucidat’s Cultivate guide

www.elucidat.com | @elucidat | info@elucidat.com

Start Elucidat Free Trial

Learn more...