iui2017 smartlearn keynote: intelligent interfaces for open social student modeling
TRANSCRIPT
Intelligent Interfaces for Open Social Student Modeling
Peter BrusilovskySharon Hsiao,Tomek Loboda, Julio
Guerra, Jordan Barria-PinedaPAWS Lab,
University of Pittsburgh
Overview
• Goals– Why we are doing it?
• Open Student Models– From ANS to OSM
• Open Social Student Models– QuizMap, Progressor, Progressor+
• Mastery Grids– Topic-level OSLM in Mastery Grids – Concept-level OSLM in Mastery Grids
From Goals to Technologies
• Technologies–Adaptive Navigation Support–Open Student Models–Open Social Student Modeling
• Why to use it– Increase user performance– Increase motivation and retention
Targets Engaged
• Adaptive Navigation Support• Topic-based Adaptation• Open Student Modeling• Social Navigation and Comparison• Open Social Student Modeling• Social Educational Progress Visualization• Multiple Content Types• Open Source• Concept-Based Adaptation
Adaptive Link Annotation: InterBook
1. Concept role2. Current concept
state
3. Current section state4. Linked sections state
4
3
2
1
√
Questions of the current quiz, served by QuizPACK
List of annotated links to all quizzes available for a student in the current course
Refresh and help icons
QuizGuide = Topic-Based ANS
Topic-Based Adaptation
Concept A
ConceptB
ConceptC
Each topic is associated with a number of educational activities to learn about this topic
Each activity classified under 1 topic
QuizGuide: Adaptive Annotations• Target-arrow
abstraction:– Number of arrows –
level of knowledge for the specific topic (from 0 to 3). Individual, event-based adaptation.
– Color Intensity – learning goal (current, prerequisite for current, not-relevant, not-ready). Group, time-based adaptation.
Topic–quiz organization:
QuizGuide: Success Rate
QuizGuide: Motivation
Within the same class QuizGuide session were much longer than QuizPACK sessions: 24 vs. 14 question attempts at average.
Average Knowledge Gain for the class rose from 5.1 to 6.5
• Topic-Based interface organization is familiar, matches the course organization, and provides a compromise between too-much and too-little
• Two-way adaptive navigation support guides to the right topic
• Open student model provides clear overview of the progress
Topic-Based ANS: Success Recipes
Targets Engaged
Adaptive Navigation SupportTopic-based AdaptationOpen Student Modeling• Social Navigation and Comparison• Open Social Student Modeling• Social Educational Progress Visualization• Multiple Content Types• Open Source• Concept-Based Adaptation
Social Navigation
• Concept-based and topic-based navigation support work well to increase success and motivation
• Knowledge-based approaches require some knowledge engineering – concept/topic models, prerequisites, time schedule
• In our past work we learned that social navigation – “wisdom” extracted from the work of a community of learners – might replace knowledge-based guidance
• Social wisdom vs. knowledge engineering
Knowledge Sea – Social Navigation
Farzan, R. and Brusilovsky, P. (2005) Social navigation support through annotation-based group modeling. 10th International User Modeling Conference Lecture Notes in Artificial Intelligence, vol. 3538. Berlin: Springer Verlag, pp. 463-472.
Open Social Student Modeling• Motivation
– Combine benefits of Open Student Models with social navigation and social comparisons
• Key steps– Assume simple topic-based design – Show topic- and content- level knowledge progress of a student
in contrast to the progress of the class– The design should guide students to most appropriate topics
and content• Main challenge
– How to design the interface to show student and class progress over topics?
– We went through several attempts…
16
QuizMap
17
Parallel Introspective Views
18
Progressor
• Topic organization should follow the natural progress or topics in the course
• Clear comparison between “me” and “group”
• Ability to compare with individual peers, not only the group
• Privacy management
OSLM: Success Recipes
The Value of OSLM
Progres
sor
QuizJE
T+IV
QuizJE
T+Portal
Java
Guide
0
50
100
150
200
250205.73
113.05
80.81
125.5
Attempts
AttemptsProg
resso
r
QuizJE
T+IV
QuizJE
T+Portal
Java
Guide
0.00%
20.00%
40.00%
60.00%
80.00%68.39% 71.35%
42.63%
58.31%
Success Rate
Success Rate
The Mechanism of Social Guidancestronger students left the traces for weaker ones to follow
21Time
Topics Strong Weak
The Secret
Targets Engaged
Adaptive Navigation SupportTopic-based AdaptationOpen Student ModelingSocial Navigation and ComparisonOpen Social Student ModelingSocial Educational Progress Visualization• Multiple Content Types• Open Source• Concept-Based Adaptation
24
Progressor+ OSLM for two types of content• macro- and micro- comparisons (group or peers)
Students Spent More Time in Progressor+
Quiz =: 5 hours Example : 5 hours 20 mins 25
QuizJET JavaGuide Progressor Progressor+0
50
100
150
200
250
300
350
60.04
150.19
224.7
296.9
69.52
121.23110.66
321.1
Total time spent (minutes)
QuizExample
26
Students Achieved Higher Success Rate
QuizJET JavaGuide Progressor Progressor+0.00%
20.00%
40.00%
60.00%
80.00%
42.63%
58.31%
68.39% 71.20%
Success Rate
p<.01
27
Mastery Grids
28
Mastery Grids: Content Access
29
Mastery Grids: Group and Peer OSLM
MG flexibility
• Parameters to set the visualization:– show hide toolbar or any of its elements– set the (sub) groups: top N, other sub groups– preset values (for example load individual
view by default)– enable/disable recommendation
• Parameters can be specified by group or by user
31
Mastery Grids Engage More
_x00
08_B
aseli
ne
_x00
05_ O
SSM0
204060
Problems Solved
_x00
08_B
aseli
ne
_x00
05_ O
SSM0
10203040
Examples Viewed
And social comparison (OSSM) features strengthen the effect
32
OSSM Engages Persistently
PART 1 PART 210
15
20
25
30Activity by Session
OSM OSSM
Step-wise regression: being in the OSSM group means an increase of about 30 activities, as compared to being in the OSM group.
33
OSSM Group Becomes More Effective• Instructional Effectiveness (Paas & Van
Merriënboer, 1993)Relates performance in problems and time spent
PART 1 PART 2-0.4
-0.2
0
0.2
Effectiveness Score
OSSM
OSM
Targets Engaged
Adaptive Navigation SupportTopic-based AdaptationOpen Student ModelingSocial Navigation and ComparisonOpen Social Student ModelingSocial Educational Progress VisualizationMultiple Content TypesOpen Source• Concept-Based Adaptation
Concept-Based Student Modeling
Example 2 Example M
Example 1
Problem 1
Problem 2 Problem K
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N
Examples
Problems
Concepts
These cells (first row) shows your progress in the topics of the course
This bar chart shows your progress in the concepts of the course
Each topic has several concepts associated to it. Mouseover a topic to highlight its concepts
This bar chart (upside-down) shows the average progress of the rest of the class on the concepts
Middle row shows the difference between your progress and the progress of the group
Third row shows the progress of the group in blue
Concept level OSLM
An overlayed pane opens indicating which topic you are inspecting (in this case the topic "Comparisons")
The concepts within the selected topic are highlighted
Mousing over this activity
Concepts in the selected activity are highlighted
This gauge estimates the how much you can learn in the selected activity. You will probably learn more in activities that have more new concepts
See more in IUI 2017 Demo! "Concept-Level Knowledge Visualization for Supporting Self-Regulated Learning"
Targets Engaged
Adaptive Navigation SupportTopic-based AdaptationOpen Student ModelingSocial Navigation and ComparisonOpen Social Student ModelingSocial Educational Progress VisualizationMultiple Content TypesOpen SourceConcept-Based Adaptation
Acknowledgements• Joint work with
– Sergey Sosnovsky– Sharon Hsiao– Julio Guerra– Jordan Barria-Pineda
• NSF Grants– EHR 0310576– IIS 0426021– CAREER 0447083
• ADL “PAL” grant to build Mastery Grids
Read About It!• Brusilovsky, P., Sosnovsky, S., and Yudelson, M. (2009) Addictive links: The
motivational value of adaptive link annotation. New Review of Hypermedia and Multimedia 15 (1), 97-118.
• Brusilovsky, P., Hsiao, I.-H., and Folajimi, Y. (2011) QuizMap: Open Social Student Modeling and Adaptive Navigation Support with TreeMaps. Proceedings of 6th European Conference on Technology Enhanced Learning (ECTEL 2011), pp. 71-82
• Hsiao, I.-H., Bakalov, F., Brusilovsky, P., and König-Ries, B. (2013) Progressor: social navigation support through open social student modeling. New Review of Hypermedia and Multimedia
• Brusilovsky, P., Somyurek, S., Guerra, J., Hosseini, R., Zadorozhny, V., and Durlach, P. (2016) Open Social Student Modeling for Personalized Learning. IEEE Transactions on Emerging Topics in Computing 4 (3), 450-461.
• Jordan, B.-P., Guerra, J., Huang, Y., and Brusilovsky, P. (2017) Concept-Level Knowledge Visualization for Supporting Self-Regulated Learning. In: Proceedings of Companion of the 22nd International Conference on Intelligent User Interfaces (IUI '17), Limassol, Cyprus, ACM, pp. 141-144 also available at https://doi.org/10.1145/3030024.3038262.