samos project: a data analysis model based on control charts to monitor online learning processes...
DESCRIPTION
Presentation "SAMOS Project: A data analysis model based on control charts to monitor online learning processes IEMAE". Author: Angel A. Juan for the journey "Análisis del comportamiento de los estudiantes de la UOC"TRANSCRIPT
SAMOS Project: A data analysis model based on control charts to monitor online learning processes
IEMAE
Angel A. Juan, Thanasis Daradoumis, Santi Caballé, Fatos XhafaDep. of Computer Science, Multimedia and Telecommunication
Open University of Catalonia (Spain)
{ajuanp, adaradoumis, scaballe, fxhafa}@uoc.edu
July 2nd, 2008Barcelona, Spain
This work is partially supported by the Innovation Vice-rectorate of the Open University of Catalonia under grant IN-PID0702
1. Introduction
� E-learning models can provide high quality educational
offerings at the same time they allow for convenient and
flexible learning environments without space, distance or
time restrictions (Seufert et al., 2002)
� Nowadays, most universities and colleges worldwide are
using some learning management system (LMS) –such
as Moodle, Sakai or WebCT/Blackboard– as part of the
technical resources they make available to their students
and instructors.
� The instructor’s role is moving from one related to a
knowledge transmission agent to another related to a
specialist agent who designs the course, guides, assists
and supervises the student's learning process (Simonson
et al., 2003; Engelbrecht and Harding, 2005)
� Instructors need information systems and tools that help
them monitoring the e-learning process in a similar way
engineers and managers need efficient information
systems and tools that help them to control, in real time,
business or service processes.
2. Need for Monitoring E-learning Processes
� Any type of distance education program
presents higher dropout rates than more
conventional programs (Sweet, 1986) � It is
necessary that instructors provide just-in-time
guidance and assistance to students
� Communication among students need to be
facilitated and promoted by instructors
� Monitoring students’ and groups’ activity and
performance can help to understand students’
interactions and anticipate potential problems:
abandonment, group malfunction, etc.
(Dillenbourg, 1999; Daradoumis et al., 2006)
� Monitoring reports can be used by instructors
to easily track down the learners’ online
behavior and group’s activity and performance
at specific milestones, gather feedback from
the learners and scaffold groups. Some of
these reports can also be employed as
feedback for students
3. Quick Review of Existing Research
� Instructors participating in online learning environments have
very little support by integrated means and tools to monitor and
evaluate students’ activity (Jerman et al., 2001; Zumbach et al.,
2002)
� Rosenkrans (2000) states the necessity of using assessment
tools that monitor students’ progress as a way to empower
instructors’ role in online environments and also as a way to
provide a feedback to students
� Rada (1998) is the first author who proposed the use of
statistical quality control methods and tools to monitoring the
student transactions in online learning environments
� Other authors (Simoff and Maher, 2000; Juan et al., 2008;
Gaudioso et al., 2008) discuss the lack of online data analysis
and analytical processing features of Web-based educational
environments and propose the use of data analysis, data mining
and visualization as an integral part of these environments.
� Recently, an emergent research area called Educational Data
Mining is focused on the application of data mining techniques to
discover information and knowledge from log files data registered
in course management systems (Romero and Ventura, 2007;
Romero et al., 2008)
4. Main Contribution of our approach
� The main goal is to develop, implement and test a
practical information system that allows instructors
in most e-learning environment to efficiently monitor
students’ and groups’ activity and performance in e-
learning courses
� Also, our work can serve as a conceptual
framework that can be used for tracking groups’
and individuals’ activity in any collaborative e-
learning courses that:
a) Span over one or more semesters,
b) Involve a large number of groups and
students that need to carry out a continuous
and intensive collaborative activity, and
c) Need to analyze and evaluate specific
situations at different granularity levels, e.g.:
at the instructor level, the course manager
level and the student level
� Three-layer structure
� Objective: the set of monitoring tools should be independent of the LMS
employed, so that they could be used in combination with any of the most
popular e-learning platforms (Moodle, Sakai, WebCT, etc.)
� The monitoring tools will make use of the server log files and/or academic
database records provided by the LMS to generate the graph-based reports
� Then, an e-mail containing personalized reports will be automatically sent by the
system to each addressee, either instructor or student (“push” strategy)
5. Layer Diagram of SAMOS
1. Students develop different activities (events) in the LMS spaces. Some of them have only
an activity dimension while others have also an academic performance dimension.
2. Events generated by students are registered in log files at the LMS server or in records at
the database server.
3. After processing log files and database records, personalized students and instructors
reports are generated and sent to the e-mail server.
6. Functionality of the Model (1/2)
4. The e-mail server sends the performance reports to each student (personalized feedback)
5. Instructors receive personalized reports regarding both students’ academic activity and
performance. These reports will allow them to easily identify those students who are “at
risk”, i.e.: students with low activity levels and students which are underperforming
6. This way, instructors can offer them “just-in-time” guidance and support
6. Functionality of the Model (2/2)
� We chose to generate periodical (weekly, etc.) monitoring reports with the aim
to show a small set of graphs that were easily and quickly understood by
instructors
� For flexibility and efficiency reasons, we decided that these graphs should
contain only critical information about groups’ and students’ activity &
performance levels
� They should provide instructors with a rough classification for each kind of
entities –groups and students– according to their corresponding activity &
performance levels
� These graphs should also provide information about the historical evolution of
each student activity and performance with respect to the rest of the class
� Having these considerations in mind, we designed the following four charts:
a) Students’ classification according to their activity level,
b) Activity control chart for each student,
c) Students’ classification according to their performance level, and
d) Performance control chart for each student
7. Designing the Charts
� Students’ classification according to their activity level (weekly, Inst.):
• A scatter plot of X = “Number of events generated by student i during this (current)
week” and Y = “Number of events generated by student i during an average week”
• It also includes the vertical lines defined by the first, second and third quartiles of X
8. The SAMOS Monitoring Reports (1/4)
� Activity control chart for each student (weekly, Inst.):
• It is a monitoring chart which shows the weekly evolution of each student academic
activity levels (represented by circular dots connected by line segments)
• The graph also contains quartiles bands
8. The SAMOS Monitoring Reports (2/4)
� Students’ classification based on their performance (at each test, Inst.):
• It is a scatter plot of X = “Score obtained by student i in this (last) test” and Y =
“Average score obtained by student i in the past tests (including the last one)”
8. The SAMOS Monitoring Reports (3/4)
� Performance control chart for each student (at each test, Inst. & Student):
• It is a monitoring chart which shows the evolution of each student academic
performance
• It also includes: (a) the updated average student’s score, and (b) quartiles bands
8. The SAMOS Monitoring Reports (4/4)
� We have discussed the convenience of adapting process
control charts, extensively used in quality engineering, to
the e-learning arena
� We have focused in two major related problems in
distance learning courses: (a) assure that students will
reach a satisfactory performance level in the learning
process, and (b) avoid high dropout rates caused by the
lack of adequate support and guidance
� Monitoring students’ activity and performance is needed
in order to identify non-participating and underperforming
students in “real time”
� The model presented here can be easily adapted and
used in different LMS and in most online courses from
any knowledge area
� It is expected that SAMOS will add value to the
instructors’ role as designers and supervisors of the
learning process and will allow them to offer flexible and
just-in-time guidance and assistance to students
9. Conclusions
� Juan, A.; Daradoumis, T.; Faulin, J.; Xhafa, F. (under review): “A Data Analysis Model
based on Control Charts to Monitor Online Learning Processes”. Int. Journal of
Business Intelligence and Data Mining. ISSN: 1743-8187
� Juan, A.; Daradoumis, A.; Xhafa, F.; Caballe, S.; Faulin, J. (2009): Monitoring and
Assessment in Online Collaborative Environments: Emergent Computational
Technologies for E-Learning Support. IGI Global, Hershey, Pennsylvania, USA.
� Juan, A.; Daradoumis, T.; Faulin, J.; Xhafa, F. (in press): “SAMOS: A Model for
Monitoring Students’ and Groups’ Activity in Collaborative e-Learning“. International Journal of Learning Technology. ISSN: 1477-8386
� Daradoumis, A.; Faulin, J.; Juan, A.; Martinez, F.; Rodriguez, I.; Xhafa, F. (in press): “CRM Applied to Higher Education: Developing an e-Monitoring System to Improve
Relationships in e-Learning Environments”. International Journal of Services
Technology and Management. ISSN: 1460-6720
� Caballe, S.; Juan, A.; Xhafa, F. (2008): “Supporting Effective Monitoring and
Knowledge Building in Online Collaborative Learning Environments”. In Proceedings of
the First World Summit on the Knowledge Society (Springer Lecture Notes in
Computer Science). Athens, Greece, September 24-28.
� Daradoumis, A.; Faulin, J.; Juan, A.; Martinez, F.; Rodriguez, I.; Xhafa, F. (2008):
“Expanding the Customer Relationship Management Scope to the Non-Profit Organizations: an Analysis Focused on the E-University Domain”. In Proceedings of
the IADIS International Conference, e-Commerce 2008. Amsterdam, Netherlands,
July, 25-27.
� Juan, Α.; Daradoumis, Τ.; Faulin, J.; Xhafa, F. (2008): “Developing an Information
System for Monitoring Student’s Activity in Online Collaborative Learning”. In
Proceedings of the 2nd International Conference on Complex, Intelligent and Software
Intensive Systems, pp. 270-275. IEEE Computer Society. ISBN: 0-7695-3109-1. Barcelona, Spain, March 4-7.
10. Related Work
SAMOS Project: A data analysis model based on control charts to monitor online learning processes
IEMAE
Angel A. Juan, Thanasis Daradoumis, Santi Caballé, Fatos XhafaDep. of Computer Science, Multimedia and Telecommunication
Open University of Catalonia (Spain)
{ajuanp, adaradoumis, scaballe, fxhafa}@uoc.edu
July 2nd, 2008Barcelona, Spain
This work is partially supported by the Innovation Vice-rectorate of the Open University of Catalonia under grant IN-PID0702
Thank You!Thank You!