lak12 jlsantos-v2
DESCRIPTION
TRANSCRIPT
![Page 1: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/1.jpg)
Goal-oriented visualizations of activity tracking: a case study with engineering
students
José Luis Santos, Sten Govaerts, Katrien Verbert, Erik Duval
1
http://www.slideshare.net/jlsantoso
![Page 2: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/2.jpg)
2
Motivation
1. Means: Information visualization
2. Goals1. Awareness2. Self-reflection3. Sense-making
but even more important…
to engage students on these processes.
![Page 3: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/3.jpg)
3
LAMS DOKEOS
KHAN ACADEMYMOODLE
Related work
![Page 4: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/4.jpg)
4
1. Upton et al. Narcissus: Group and Individual Models to Support Small Group Work. UMAP’09 http://goo.gl/ZzNbF2. Derick et al. GLASS: A first look through a Learning Analytics System. LAK’12 http://goo.gl/eY4w23. Mazza et al. Exploring usage analysis in learning systems. Workshop at AIED’05 http://goo.gl/ycrFQ
![Page 5: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/5.jpg)
5
1. Santos et al. Visualizing PLE usage. EFEPLE’11 workshop. http://goo.gl/PWbBF2. Govaerts et al. The student activity meter for awareness and reflection. ACM SIGCHI 2012. http://goo.gl/SGfta3. Duval et al. Learning dashboard and learnscapes. Workshop at ACM SIGCHI 2012. http://goo.gl/j7NeJ 4. Duval. Attention please! Learning analytics for visualization and recommendation. LAK’11. http://goo.gl/Kfwii
![Page 6: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/6.jpg)
![Page 7: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/7.jpg)
7
Design based research methodology
Design Implementation Deployment
Observation & Evaluation
AnalysisRequirements
![Page 8: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/8.jpg)
One… two… three… and… four!
1. Paper prototype– Professors and teacher assistants.
2. Digital prototype– Professors and teacher assistants.
3. First release– Students
4. Second release– Students
8
![Page 9: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/9.jpg)
1st and 2nd iterations
9
-Generic (CAM)-Usability-Think-aloud protocol.-15-30 minutes- 1st -> 7 participants- 2nd -> 5 participants
![Page 10: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/10.jpg)
Results
10
• Size of tables• Redundant information• Personalization• Context centralization vs diversification• Motion chart
![Page 11: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/11.jpg)
11
Scenario
Awareness and reflection
Goal-oriented
How do students spend their time during the lab sessions?
![Page 12: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/12.jpg)
Why and what do we track?
• Individual and group spent time • Type of activity
(development, writing, …)• Application• Program code• Websites
12
+
![Page 13: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/13.jpg)
How do we visualize this data?
13
http://role-sandbox.eu/spaces/dashboardkulor
http://goo.gl/qzP9H
http://sourceforge.net/apps/mediawiki/role-project
![Page 14: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/14.jpg)
14
![Page 15: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/15.jpg)
15
![Page 16: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/16.jpg)
16
![Page 17: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/17.jpg)
17
![Page 18: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/18.jpg)
18
![Page 19: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/19.jpg)
19
![Page 20: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/20.jpg)
20
![Page 21: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/21.jpg)
21
![Page 22: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/22.jpg)
First evaluation with real data
• 36 students• Questionnaire after seeing a demo
– From USE questionnaire• Perceived usefulness • Privacy concerns
22
![Page 23: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/23.jpg)
23
Conclusions
![Page 24: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/24.jpg)
24
CAM Dashboard will help me to understa
nd how I use my tools
CAM Dashboard is useful
CAM Dashboard will help me to
be more effective
CAM Dashboard is easy to
use
Using CAM Dashboard is
effortless
I am sa
tisfied with
CAM Dashboard
CAM Dashboard is fun to
use
I like to
see what o
ther members d
o during th
e course
I feel confident u
sing th
e tracking syste
m in the P&O lab
during th
e course
I would also
feel confident usin
g the tra
cking system
outside of th
e P&O lab during th
e course
• Useful• Helpful• Lab tracking• Usability?
![Page 25: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/25.jpg)
Second evaluation with real data
• 10 students• 4 lab sessions• 15 minutes• SUS• Perceived usefulness • Tracking/privacy concerns.• Questionnaire
25
![Page 26: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/26.jpg)
26
Conclusions
• SUS score: 72 • Useful?• Detection of
certain patterns
![Page 27: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/27.jpg)
27
Conclusions
![Page 28: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/28.jpg)
28
Final conclusions
• Not included in this paper• 14 students• SUS: 66 • Useful for:
– Teachers (6) – Students (5)– Both (3)
• Satisfaction:– 4.4 (over 7)
![Page 29: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/29.jpg)
Although they consider it useful, understandable, usable…
29
![Page 30: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/30.jpg)
…students did not use the tool frequently
30
WHY?
![Page 31: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/31.jpg)
One student said: “We had other priorities”
31
![Page 32: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/32.jpg)
How can we engage students in this process?
32
![Page 33: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/33.jpg)
33
![Page 34: Lak12 jlsantos-v2](https://reader036.vdocuments.us/reader036/viewer/2022081518/5461f126b1af9f936c8b4c5a/html5/thumbnails/34.jpg)
Thank you! Any question?
@jlsantoso@erikduval@katrien_v
http://jlsantoso.blogspot.com
http://erikduval.wordpress.com34