real-time visual analytics to evaluate online collaboration (166277994)

7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166277994)

http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166277994 1/61

Real-Time Visual Analytics to Evaluate

Online Collaboration

John McCormick | Sarah Krongard

Agenda

1. Learning analytics defined

2. Discourse and collaboration

3. Social Networks Adapting Pedagogical Practice(SNAPP): A tool for visualizing interaction

4. Future of discourse learning analytics tools

5. Q and A

Horizon Report: Emerging Technologies

Mobile

Learning(1-2 yrs)

TabletPCs(1-2 yrs)

Learning

Analytics(2-3 yrs)

Definitions

The measurement, collection, analysis and reporting of dataabout learners… for purposes of understanding and optimising

learning and the environments in which it occurs

- 1st international Conference on Learning Analytics and Knowledge (LAK 2011)

----------------…selection, capture and processing of data that will be helpfulfor students and instructors at the course or individual level.

…able to adjust content, levels of support and other personalized

services by capturing, reporting, processing and acting on data

on an ongoing basis in a way that minimizes the time delay

between the capture and use of data.

-Elias, T. (2011). Learning Analytics: Definitions, Processes and Potential

Definitions (cont.)

Learning Analytics:

• is the analysis of many kinds of learner-produced and

learner-related data;

•seeks to monitor learner activity and progress and to

predict learner outcomes; and

• enables interventions and decision making about

learning by instructors and students.

- From Educause/ELI Focus Session on Learning Analytics (2012)

Example 1: Signals: Purdue

Source: http://www.itap.purdue.edu/studio/signals/

Signals: Results

“When instructors follow recommended Course Signals bestpractices, grade performance improvement has been consistently

demonstrated at both the course and departmental level.”

• 10% increase in numbers of As and Bs awarded in semester-

long courses, and a 6.41% decrease in Ds, Fs and withdrawals.

• Greatest improvement seen in students who were initially

receiving Cs and Ds in early assignments, and pull up half a

letter grade or more to a B or C.

• “…the overwhelming response from the students is that Course

Signals is a helpful and important tool that aids in their overall

academic success”

Arnold and Pistilli (2012)

The seven dimensions of ‘Learning Power’

1. Critical curiosity

2. Changing and learning

3. Meaning making

4. Creativity

5. Learning relationships (interdependence)

6. Strategic awareness

7. Resilience

Example 2: Effective Lifelong Learning

Inventory (ELLI)

Source: http://www.slideshare.net/Ruthdeakincrick/learning-dispositions-and-transferable-competences-pedagogy-modelling-and-learning-analytics

Source: http://www.heacademy.ac.uk/assets/documents/what-works-student-retention/Northumbria_ELLI_Dec_11.pdf

Spider Diagram

ELLI ResearchFindings from research and development projects indicate that

learners who are aware of their learning power profile using ELLI and

who have structured learning opportunities to improve it, become:

• More reflective and aware of themselves as learners

More confident and motivated to learn• More willing and able to take responsibility and ‘own’ their

learning

• More keen to carry on learning

• More likely to attain higher levels and grades• More likely to fulfil their potential

• More sociable and able to learn with and from each other

• More able to manage change and transitions

Source: My Learning Power: Using ELLI in Higher Education

Enquiry Blogger

Source: http://wordpress.org/extend/plugins/eb-enquiryblogbuilder/

Instructor Dashboards

ELLIMent

Source: http://people.kmi.open.ac.uk/ullmann/projects/ELLIMent/index.html

Academic vs. Learning Analytics

Long, P. & Siemens, G. (2011). Penetrating the Fog: Analytics in learning and education.

Initial Thoughts about Learning Analytics?

Discourse & Online Collaboration

• What is the value of collaboration in an online

environment?

What are the challenges & opportunities doinstructors face in facilitating online

collaboration?

To what extent have the roles of instructorand student changed?

The Importance of Collaboration

• To refine and confirm personal understanding

with a community of learners

• To develop, foster, and practice higher-order

thinking skills

• To share and co-construct knowledge to

support emerging collectives

Online Collaboration

Challenges

• Increased Workload

• Difficulty of Encouraging

and Sustaining High-LevelDiscourse

• Asynchronous Interactions

• Lack of F2F Connections

• Challenge of Self-Motivation

• Technical Difficulties

Opportunities

• Time Allocated for

Thoughtful and Reflective

Dialogue• Permanency of the Written

• Tools Available to Facilitate

Interaction and Engagement• Opportunities for Student

Agency and Efficacy

Making the Shift in Roles from…

… Traditional Instructor to Guide and Facilitator

… Presumed Authority to Collective Credibility

… Student as Passive Recipient to ActiveProducer and Co-Creator

“Learning is in social relationships –situations of cooperation with others…

Both the learner and the community in which

they participate are transformed by thecommunity as a complementary unit thatwork in harmony to promote learning…”

(Dooley, Lindner, Dooley, & Murphy, 2005, p. 49)

SNA Concepts: Centrality

Source: http://www.wolfram.com/mathematica/new-in-8/graph-and-network-analysis/degree-centrality-in-social-networks.html

Node Size: Number of Interactions

Source: http://socialmedialab.ca/wp-content/uploads/2011/02/Fig.1.jpg

Information Brokers

Frequency of Interaction

SNA and Online Discourse

• Very little SNA research on online discussion for learning

General Findings:

•More evenly distributed activity beneficial

• Low hierarchical structures better (groups interacting,teacher not playing central role)

• More interaction not necessarily better

Toikkanen , T. & Lipponen, L. (2011). The applicability of social network analysis tothe study of networked learning. The Journal of Interactive Learning Environments,19 (4), 365-379.

Social Networks Adapting Pedagogical Practice (SNAPP)

Other SNAPP metrics

What SNAPP can tell you

Dawson, S., Bakharia, A., Lockyer, L., & Heathcote, E. (2011). ‘Seeing’ Networks: Visualising and evaluating student learning networks.Australian Learning and Teaching Council, Canberra, Australia.

Dawson et al findings: Instructor use

“extremely effective in promoting reflection onteaching activities and rapidly assessing overalleffectiveness..”

“However, teachers require additional support

and training to better understand therelationship between their implemented online

learning activities and the resultant onlinestudent behavior…”

Dawson, S., Bakharia, A., Lockyer, L., & Heathcote, E. (2011). ‘Seeing’ Networks:

Visualising and evaluating student learning networks. Australian Learning and

Teaching Council, Canberra, Australia.

Interpretation and Learning Design

“Future research and professional development

should seek to merge learning design with

learning analytics and data interpretation. A

sound understanding of learning designprinciples will greatly assist academics in better

interpreting the learning analytics data into

informed pedagogical action”.

Action Research Steps

1. Create visual taxonomy

2. Choose instructors

3. Collect data

4. Send “Visual Key” and “Custom Report”

5. Interview instructors

6. Follow- up with instructors

Visual Key

Visual Features (filtering)

Instructor-Led

Learning Community

Combined Patterns

Learning Community with strong instructor presence

Emerging Learning Community

Emerging Learning Community with strong instructor

presence

Weak Learning Community

Learning Community Development

Two Facilitation Styles

Facilitator Role: Central vs Peripheral

Instructors critical to success of discussion• Blignaut & Trollip, 2003; Ferdig & Roehler, 2004; Greenlaw & DeLoach,

2003; Love, 2002; Roblyer & Wiencke, 2003; Kearsley, 2000; Arnold &

Ducate, 2006; Darabi etal., 2011; Ertmer et al., 2011; Liu & Yang, 2012;

Instructors should take a back seat, let students

construct own knowledge• Burstall, 2000; Li, 2003; Mazzolini & Maddison, 2003; Poole, 2000; Rourke

& Anderson, 2002; Paloff & Pratt, 2001; Wang & Chen, 2010, Baram &Correia, 2009

Learner Population: Undergraduate Business

Typical Pattern: Learning Community with strong instructor

presence

Intervention Suggestions: Reduce number of instructor postings,

add summary or “weave” following discussion, use student-led

facilitation, organize discussions with use of titled posts.

Intervention Plans: Reduce instructor posting in some

discussions and better organize initial posts.

Instructor: High Centrality, Low In-Degree

Future learning analytics

Source: http://www.studentsoftheworld.info/sites/people/32917.php

Meerkat-ED

Group interaction over time

Networks of Terms

d d l h d l k d

Individual Discussion Thread: Click on edges

Natural Language Processing

https://www.coursera.org/course/nlp

Dashboards: Desire2Learn

“The ties between people make

the whole greater than the sum

of its parts”

Nicholas Kristof, Sociologist

Questions, Comments, Answers?

real-time visual analytics to evaluate online collaboration (166277994)

Documents

cisco prime collaboration assurance and analytics 11.6...

collaboration analytics:unlock the value of...

the interagency food safety analytics collaboration (ifsac...

using analytics and collaboration to improve healthcare...

using analytics for successful plan and provider...

evaluate content with google analytics (oct 2009)

predictive analytics for supply chain collaboration, risk...

accelerating innovation in government through advanced...

lisbon mro summit - collaboration in big data analytics

prime collaboration analytics dashboards and reports prime...

strategy and tools to evaluate learning analytics ... ·...

benchmarking smart meter data analytics · [19]....

social collaboration analytics for enterprise ...social...

learning analytics to evaluate and confirm pedagogic...

hr moves boldly into advanced analytics with collaboration

getting the measure of analytics: using bibliometrics and...

using exam analytics to evaluate student use of lecture...

cisco prime collaboration assurance and analytics 11.6...

using analytics to improve agent performance & customer...

evaluate your content with google analytics