Mo#va#onal  Social  Visualiza#ons  for  Personalized  Elearning

Sharon Hsiao & Peter Brusilovsky Sept. 2012

School of Information Sciences, University of Pittsburgh

1

Agenda  

•  Introduc#on  •  Background  •  Progressor+:  An  Innova#ve  Tabular  Open  Social  Student  Modeling  Interface  

•  Evalua#on  &  Results  •  Summary  

2  

INTRODUCTION  

3

Personalized vs. Social?

Personalized Learning

Social Learning

increase motivation better performance

development of high level thinking skills

higher satisfaction

higher self-esteem, attitude better retention

increase learning rate increase learning quality

current knowledge level relevant interesting content

reduce navigational overhead increase satisfaction

increase motivation

What We Did Before

QuizGuide (Brusilovsky, Sosnovsky, et al., 2004; Sosnovsky &

Brusilovsky, 2005)

Knowledge Sea II (Brusilovsky, Chavan, & Farzan, 2004)

Personalized Learning Social Learning

Why do We Want a Merge?

Increasing amount of educational resources –  High costs of maintain the associations between

content and domain –  Social technologies provide collective wisdom

that might replace knowledge engineering Motivation is important –  Even a great guidance will not provide good

impact without motivation

Model precision

Model complexity

Target area

Personalized approach Social-based approach

Integrated-approach (Hybrid)

Problems illustration

7

Challenge

How do we introduce personalized guidance to social technologies and harness the benefits from both approaches? •  Keep the benefits of personalized guidance •  Increase user motivation

This work…

1.  Personalized Guidance – Navigation support: topic-based & progress-based

adaptation 2.  Social Visualization – easy-to-grasp and holistic view of student model &

content model 3.  Integration of 1&2 above in Open Student

Modeling visualization

Navigation Support in Open Social Student Modeling Visualization

Research Questions

1: What are the design principles (key features) to implement personalized guidance in open social student modeling visualizations?

2: Will navigation support combined with open social student modeling visualization work in realistic content collections?

3: Will this approach guide students to the right content at the right time?

4: Will this approach increase students motivation & engagement?

10

Research model

11

BACKGROUND  

12

Supporting Theories •  Self-regulated theory (Zimmerman, 1990)

–  High jumpers (students who gained higher conceptual understandings): good at using effective strategies, creating sub-goals, monitoring emerging understanding, and planning their time and effort.

–  Low jumpers: did no spend much time monitoring their learning, tend to engage help seeking behavior.

•  Social comparison theory (Festinger, 1954; Dijkstra. et al., 2008) –  lateral comparison: self-evaluation –  downward comparison: self-enhancement –  upward comparison: self-improvement

13

Related work

14

Adaptive navigation support in E-Learning –  AHA! (De Bra & Calvi, 1998) –  ELM-ART (Weber &

Brusilovsky, 2001) –  KBS-Hyperbook (Henze &

Nejdl, 2001) –  INSPIRE (Grigoriadou,

Papanikolaou, Kornilakis, & Magoulas, 2001)

–  InterBook (Brusilovsky, 1998) –  NavEx (Brusilovsky, et al.,

2009) –  ISIS-Tutor (Brusilovsky,

1994) –  QuizGuide (Brusilovsky,

Sosnovsky, et al., 2004)

Social navigation and visualization for E-Learning –  EDUCO (Kurhila, Miettinen,

Nokelainen, & Tirri, 2006) –  KnowledgeSea II (Brusilovsky,

et al., 2009) –  AnnotatEd (Farzan &

Brusilovsky, 2008) –  Comtella (Vassileva & Sun,

2007) –  OLMlets (Bull & Britland,

2007) –  CourseVis (Mazza & Dimitrova,

2007)

THE  WAY  TO  PROGRESSOR+  

15

QuizMap  (EC-­‐TEL  2011)  

16

Parallel  Introspec#ve  Views  

17

Progressor    

18

PROGRESSOR+  

19

Progressor+ design rationale

•  Navigating and comparing segments of pie graphs in a huge dataset takes longer time for comprehension (Gillan & Callahan, 2000)

•  Interacting and visualizing large data in Table Lens (Rao & Card, 1994)

•  Small multiples principle (Tufte, 1990)

20

Progressor+

21

1.  Sequence  

•  provides the direction for the students to progress through the course  

22

2.  Identity  

•  simple rows & columns table representation, fragments can be easily cohesively shown  

23

3.  Interactivity  •  Direct accessing content, sorting, comparing,

collapse-and-expand  

24

4.  Comparison  

•  macro- and micro- comparisons

25

5. Transparency

•  Holistic view of all the models

26

EVALUATION  &  RESULTS  

27

Students spent more time in Progressor+

Quiz =: 5 hours Example : 5 hours 20 mins

28

60.04  

150.19  

224.7  

296.9  

69.52  

121.23  110.66  

321.1  

0  

50  

100  

150  

200  

250  

300  

350  

400  

QuizJET   JavaGuide   Progressor   Progressor+  

Total time spent (minutes)

Quiz

Example

•  the more diverse of the questions the students tried, the higher success rate they obtained (r=0.707, p<.01)

•  the more diverse of the example the students studied, the higher success rate they obtained (r=0.538, p<.01)

More diversity helped increase problem solving success

33.37

46.18 52.7

61.84

0

20

40

60

80

QuizJET JavaGuide Progressor Progressor+

distinct questions

10.86

17.3

25.125 27.37

0

10

20

30

QuizJET JavaGuide Progressor Progressor+

distinct examples

29

7.81

11.77 11.47

12.92

8.48 9.15

12.28 12.2

0

2

4

6

8

10

12

14

QuizJET JavaGuide Progressor Progressor+

Topic Coverage

Quiz

Example

Students achieved higher Success Rate

30

42.63%

58.31%

68.39% 71.20%

0.00%

20.00%

40.00%

60.00%

80.00%

QuizJET JavaGuide Progressor Progressor+

Success Rate

p<.01  

Impact on Learning – cont.  

31

• The more time the students spent on the content (quizzes and examples), the higher the level of knowledge gain they obtained (r=0.563, p<.01; r=0.448, p<.01)

•  The more the students studied (more lines), the higher level of knowledge they gained (r=0.492, p<.01)

-200.00

0.00

200.00

400.00

600.00

800.00

1000.00

1200.00

1400.00

1600.00

1800.00

time spent sorted by knowledge gain

example

quiz

Linear (example) Linear (quiz)

Time

Knowledge Gain 0 0 0 0 0 0 0 0 3 4 5 5 5 6 7 7 7 8 8 8 8 10 11 11 11 11 12 13 13 13 13 13 14 14 14 14 14 14 15 15 15 15 16 16 18 20

The Mechanism of Social Guidance

stronger students left the traces for weaker ones to follow

32 Time

Topics

uu  uu  uu  uu  uu  

uu  uu  

uu  

uu  uu  uu  uu  uu  

uu  uu  

uu  

uu  uu  uu  uu  

uu  uu  uu  

uu  

Strong students lead ahead in Progressor+

33

Mixed collection

Quizzes Examples

34

Non-adaptive adaptive

Social, adaptive, single content Progressor+

Students  worked  with  the  systems  during  exam  prepara#on,  especially  in  final  exam  period  

35

Non-adaptive adaptive

Progressor Progressor+

17.15

13.39

83.59

9.17

19.63

84.3

0

20

40

60

80

100

Easy Moderate Complex

Strong students were hours ahead of weak ones

Progressor

Progressor+

Strong students worked earlier than weaker ones

Subjective Evaluation

•  Usefulness •  Ease of Use •  Ease of Learning •  Satisfaction •  Privacy & Data Sharing

36

Students’ opinions

•  Praised Progressor+

•  “…it’s a great tool, should be used in other classes…”

•  “… I find the examples and quizzes helped. I did recommend other students use it…”

37

CONCLUSION  

38

Results Summary •  Engaged longer with Progressor+

•  Attempted more self-assessment quizzes •  Explored more annotated examples & lines •  Obtained higher knowledge gain •  Achieved higher Success Rate •  Stronger students left the traces for weaker ones to follow •  Effectively led students to work at the right level of

questions among mixed collections of educational content •  Both strong and weak student had consistent performance

across all different questions’ complexities

39

THANK  YOU  J    

40