successful transition from secondary to higher ... -...

Successful transition from

secondary to higher

education using learning

analyticsTINNE DE LAET, TOM BROOS (KU LEUVEN)

JAN-PAUL VAN STAALDUINEN (TU DELFT)

PHILIPP LEITNER, MARTIN EBNER (TU GRAZ)

1

STELA - Erasmus+ Project

562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD

Welcome!

Who are we?

Tinne De Laet ( KU Leuven)

Head of Tutorial Services of Engineering Science

promotor STELA

background = mechanical engineer

Tom Broos (KU Leuven)

Doctoral researcher STELA

Philipp Leitner (TU Graz)

Doctoral researcher STELA

2



Welcome

Would you please introduce yourself to the people on your table

Who are you?

How familiar are you with learning analytics?

Is your institute applying learning analytics?

3



Agenda for workshop

Welcome

What is learning analytics? (10 minutes)

STELA project:

Presentation of project goal & challenges (2 minutes)

Group discussion: how could learning analytics help? (10 minutes)

STELA project:

Presentation of project ideas (10 minutes)

Group discussion on project ideas (10 minutes)

Learning analytics technology (10 minutes)

Groups present their remarks, suggestions, … (10 minutes)

Wrap up, goodbye (3 minutes)

4



Agenda for workshop

…. so what we expect from you

participate in discussion in groups

each group gathers 3 ideas, thoughts, remarks, suggestions, ..

present them to each other at end of the workshop

5



What is learning analytics?

6



What is Learning Analytics? no universally agreed definition

“the measurement, collection, analysis and reporting of data about

learners and their contexts, for purposes of understanding and

optimizing learning and the environments in which it occurs” [1]

[1] Learning and Academic Analytics, Siemens, G., 5 August

2011, http://www.learninganalytics.net/?p=131

[2] What is Analytics? Definition and Essential Characteristics, Vol. 1, No. 5. CETIS Analytics Series,

Cooper, A., http://publications.cetis.ac.uk/2012/521

“the process of developing actionable insights through problem

definition and the application of statistical models and analysis against

existing and/or simulated future data” [2]

7



http://www.learninganalytics.net/?p=131

http://publications.cetis.ac.uk/2012/521

What is Learning Analytics?

no universally agreed definition

[3] Learning Analytics and Educational Data Mining, Erik Duval’s Weblog, 30 January 2012,

https://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/

“learning analytics is about

collecting traces that learners

leave behind and using those

traces to improve learning”

[Erik Duval, 3]

† 12 March 2016

8

https://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/

What is Learning Analytics? How is learning analytics different from institutional data? [4]

High-level figures:

provide an overview for internal and external reports;

used for organisational planning purposes.

Academic analytics:

figures on retention and success, used by the institution to assess performance.

Educational data mining:

searching for patterns in the data.

Learning analytics:

use of data, which may include ‘big data’,

to provide actionable intelligence for learners and teachers.

[4] Learning analytics FAQs, Rebecca Ferguson, Slideshare, http://www.slideshare.net/R3beccaF/learning-

analytics-fa-qs

9



http://www.slideshare.net/R3beccaF/learning-analytics-fa-qs


Different level of analytics

adapted from http://www.slideshare.net/gsiemens/learning-analytics-educause

level beneficiaries

course-level learners, teachers, faculties

aggregate learners, teachers, tutors, counsellors, faculties

institutional administrators, funders, marketing

regional administrators, funders, policy makers

national and

international

national and international governments, policy

makers

10




data visualization versus predictive analytics

is showing data enough?

how to show data to create sense-making and impact?

is predicting study success/drop out the only thing that matters?

can both be combined?

11




learning analytics process model

[Verbert et al. 2013] Verbert K, Duval E, Klerkx J; Govaerts S, Santos JL (2013) Learning analytics dashboard

applications. American Behavioural Scientist, 10 pages. Published online February 201, doi:

10.1177/0002764213479363

12




how to evaluate learning analytics?

is perceived usefulness enough?

is increased self-awareness enough? How will you measure this?

is increased sense-making enough? How will you measure this?

how could impact be measured?

13




six critical dimensions of learning analytics

[Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning

Analytics. Educational Technology & Society, 15 (3), 42–57. http://ifets.info/journals/15_3/4.pdf ]

14



http://ifets.info/journals/15_3/4.pdf


six critical dimensions of learning analytics



Tutors

Policy makers

15



data subjects

here: students

(could also be teachers)

data clients

students

tutors

academic administrators

policy makers



16





what are the objectives?

is it about reflection or prediction?

is showing data enough?

how to show data to create self-awareness, sense-making

and impact?



17





which data is available?

is it open? protected?

is it ethical to use the data?



18





Technology, algorithms, theories are at the basis of

learning analytics

open source or as close as possible to (proprietary)

university systems?

pedagogic theories for supporting students



19







• conventions: ethics, personal privacy, and similar socially motivated limitations

• norms: restricted by laws or specific mandated policies or standard

ethics and privacy IS a big issue

20







• competences: application of learning analytics requires new higher-order competences to enable fruitful exploitation in learning and teaching

• acceptance: acceptance factors can further influence the application or decision making that follows an analytics process

21




Agenda for workshop

Welcome


STELA project:


Group discussion: how could learning analytics help? (10 minutes):






22



STELA project

Project goal & challenges

23



STELA project

focus: transition from secondary to higher education

(summer before entering university + 1st year experience)

goal: applying learning analytics to support transition

student-centered

additional focus to tutors, student counsellors

1. beyond identifying at-risk students

inclusive approach: feedback for every student

2. beyond single course

entire program

3. provide actionable feedback

ability to remediate based on feedback

24



STELA project

Group discussion: how could

learning analytics help?

(10 minutes)

25



Group discussion: how could learning

analytics help? (10 minutes)

What are challenges in the transition from secondary to higher

education?

Which actionable feedback could support students in the

transition?

How could learning analytics help in this matter?

26



Agenda for workshop

Welcome


STELA project:








27



STELA project: project ideas

four areas of focus

1. academic performance

2. academic engagement

3. academic skills

4. academic well-being

How?

position students with respect to peers

show impact on future study pathway→ show how students with similar profile did in the past

28



STELA project: project ideas(1/4) academic performance

29



data (often) easily available

present at all universities

after the facts

strong relation performanceT-1 ~ performance

Source: ABLE project (2015-1-UK01-KA203-013767)

STELA project: project ideas(2/4) academic engagement

30



depends on context

• e.g. card swipes at Nottingham Trent University,

• learning platform activity at KU Leuven.

relation engagement ~ performance not always

straightforward.

data not always available

STELA project: project ideas(3/4) academic skills

31



Number of peers:

Motivation Score

Very high

Perf

orm

an

ce

An

xie

ty

High

Average

Low

Very Low

Very weak Weak Average High Very High

Recommendations

First aid for stress workshop at Student Health Center

Talk to a student counselor.

often self-reported

tests and questionnaires available form pedagogy

and psychology

• Time management, motivation, persistence, …

opportunity to move from information to

recommendation

STELA project: project ideas(4/4) academic well-being

32



How do you feel about this course now?

Last week, others felt like this:

opportunity to complement with continuous, bi-

directional feedback

tests and questionnaires available form pedagogy

and psychology

Agenda for workshop

Welcome


STELA project:








33



Learning analytics

technology

34



Learning Analytics technology

Challenges

multiple and different kind of data-sources

lots of data

need to be combined and analyzed

scalable

real-time?

visualized

...

35



Learning Analytics technology 36



Concept

Learning Analytics technology(1/3) Data Collection: Logstash¹

37



flexbile and open source (Apache 2 Open Source License)

data collection, enrichtment and transportation pipeline

efficiently process log, event and unstructured data sources

¹) https://www.elastic.co/products/logstash

https://www.elastic.co/products/logstash

Learning Analytics technology(2/3) Search and Analytics Engine: Elasticsearch²

38



distributed, open source search and analytics engine

scalability, reliability, and easy management

developer-friendly query language covering structured,

unstructured, and time-series data

²) https://www.elastic.co/de/products/elasticsearch

https://www.elastic.co/de/products/elasticsearch

Learning Analytics technology(3/3) Visualization: Kibana³ or Grafana4

39



3) https://www.elastic.co/products/kibana4) http://grafana.org/

open source data visualization platform

interact with data through powerful graphics

easy extensibility and a variety of panels

https://www.elastic.co/products/kibana

Agenda for workshop

Welcome


STELA project:








40



Groups present their

remarks, suggestions, …

41



Wrap up, goodbye

42



successful transition from secondary to higher ... -...

Documents