Image Credit: …Time… by Darren Tunnicliff, CC by 2.0 license

It’s About Time4th International Workshop on Temporal Analyses of Learning Data

LAK’15 Learning Analytics & Knowledge ConferenceMarch 16, 2015 | Poughkeepsie, NY

Organizing TeamSimon Knight, Bodong Chen, Britte Cheng & Alyssa Wise

Schedule for the Day

• 8:30 - 9:00 Introduction to the Workshop

• 9:00 - 10:00 Group Work Session 1

– Conceptual Session: Intro to the Approach, Tools & Data

• 10:00 - 10:15 Coffee Break

• 10:15 - 11:45 Group Work Session 2

– Application Session: Hands-on putting data into the tools /critical discussion of its application to specific data sets

• 11:45 - 12:30 Group Share, Discussion & Wrap-Up

Why Temporal Analysis & Learning Analytics?

• Temporality is fundamental in understanding learning processes

• But traditional research methods have not taken advantage of temporal information embedded in data resulting in limited explanatory power

• There have also been challenges in collecting sufficient data over time to allow such analyses

• New analytic methods, expanding sources of data and learning analytics' focus on studying processes create the perfect storm to move the sophistication of temporal analyses of learning forward

Coding and Counting vs True Process Data

Start End

Progressive phases

Repeating sequences

Temporal information lost

Coding and Counting vs True Process Data

Start End

Sequence captured but rate is lost

Distribution over time is not always equal

Temporal information lost

Snapshots vs Accumulation

…

Current state indexed as all events leading up to and including the most recent one

Current state indexed as the most recent event

Temporality as a Continuous Flow of Activity

• Examines characteristics of particular kinds of events within the activity stream

– Position in time (questions asked early on set the tone of discussion)

– Duration over time (how long spent answering each question)

– Rate over time (how quickly each problem is solved)

– Acceleration / deceleration (speed of problem-solving increases)

• Each can be thought of in absolute or relative terms

• Can consider growth and decay patterns

Temporality as an Arrangement of Events

• Examines ordered relationships of multiple kinds of events within the activity stream

– Co-occurrence (high coherence of student talk when gaze is aligned)

– Re-occurring sequences (arguments often followed by rebuttals)

– Non re-occurring sequences (aka phases) (debating of different points occurs before negotiation of a synthesis)

• Again can think absolutely or relatively (adjacency)

• Can consider ‘common’ vs ‘consequential’ patterns

Some Issues to Keep in Mind

• Granularity

– Segmentation of Time Windows

– Aggregation of Unit of Analysis

• Coordinating Multiple Data Streams

– Varying units and timescales

• Beyond Process for its Own Sake

– Connecting with Inputs and Outcomes

– Where does theory fit in?

GranularityAggregation & Segmentation

Whole Time

Time Window Window

Unit of Analysis

Data Data Data Data Data Data Data Data Data Data Data Data

Unit of Analysis

Unit of Analysis

Time Window Window

Un. Un. Un. Un. Un.

Temporal Analysis

Coordinating Multiple Data Streams

• Concurrent collection of multiple types of data– Activity logs and post contents

• Multi-dimensional coding– At the same or different levels of aggregation (unit of analysis)

• Question of what are the relevant timescales on which to consider these

Beyond Process for its Own Sake

Connecting with Inputs and Outcomes

Where does theory fit in?

A Bit of History

• It’s About Time v1.0– Alpine Rendezvous 2009

• It’s About Time v2.0– ICLS 2010

• It’s About Time v3.0– Alpine Rendezvous 2013

• Process vs Practice in Learning Analytics– ICLS 2014

Explored specific temporal analyses of group learning

Focused on analyzing multiple data streams

Mapped dimensions of temporal analyses

Questioned how to make meaning of processes

Goals for Today

It’s About Time v4.0 @ LAK 2015

Explore specific opportunities and challenges of temporal analysis for learning analytics

• Examine what particular methods can tell us about different kinds of data sets

• Consider what concepts of time different methods let us investigate

• Probe questions of interpretation and use of process measures

Framing QuestionsFor each analytic approach:

• What kinds of data is it suitable for?

• What grain size of ‘time’ does it address?

• What dimensions of temporality does it deal with?

• What kinds of insights about learning does it provide?

• What new learning/learning process constructs could emerge from the analytic approach?

• (How) could an educator interact with information produced by the analytic approach/tool as part of their teaching, assessment or other practice?

• (How) could students interact with information produced by the analytic approach/tool as part of their learning process?

• What limitations, gaps or other issues are there with the approach? How could the approach be developed further?

Our Groups for the Day

• Using Sequence Analysis and Optimal Matching to Analyze Classroom-Based Video Data [Betsy McEneaney]

• Statistical Discourse Analysis applied to F2F Turn-Taking Data [Ming Chiu]

• Temporal patterns in Assessing Collaborative Learning on Wikis in Secondary and Primary Schools [Xiao Hu]

• Epistemic Network Analysis to understand Trajectories of Development [Golnaz Arastoopour & Wesley Collier]

• Hidden Markov Modeling [Britte Cheng]

Staying Connected

• #LAK15Time

• Creating a shared resource repository

• Special Issue in JLA?