Download - Online Environment Evaluation
-
8/14/2019 Online Environment Evaluation
1/25
Evaluating students' performance in discussion
boards in Thinking and Creativity blended course
at Sultan Qaboos University
Hiba AL-Julandani Tamadher AL-Za'abi
usueduom usueduom
Instructional and Learning Technologies Department, College of Education,
Sultan Qaboos University
May ,
-
8/14/2019 Online Environment Evaluation
2/25
Abstract
This paper examines students' performance in asynchronous discussion in Thinkingand Creativity blended course. Specifically, students' participation in terms of number ofmessages posted in different discussion boards. Moreover, the quality of messagesposted is also examined. Students' participation is observed throughout the course to
address the number of messages posted by each individual student. The quality ofmessages is examined according to a checklist that contains items about message start,opinion responses, clarity and topic context.
Introduction
As we are moving fast in the digital age, here at Sultan Qaboos University manyapplications have been implemented by academics. The entrance of online learningthrough moodle and WebCT has influenced students learning and performance. Usingmoodle allows asynchronous communication between students and instructors. In the
discussion boards, an instructor and students in the course share their thoughts and ideaswith each other and essentially engage in "conversation. This is an important aspect offully online courses. But, what about blended courses. Usually, the more interactionbetween and among the students, instructor, and course content, the more successful thedistance course. In the case of blended courses, there should be a balance between classactivities and online activities. Repeating same activities makes learning process boring forboth instructor and students.
Participants and context of the study
Participants are 240 students. The participants are enrolled in three sections of anelective two-credit course entitled Thinking and Creativity, spring 2009 at Sultan QaboosUniversity. This course addresses the development of human intelligences and discussesthe recent researches in this filed. It aims to improve students' creativity through helpingthem to identify their learning styles, types and through helping them to draw conceptmaps that leads to improve their cognition and meta-cognition abilities. Moreover, thiscourse addresses some success techniques. This course is a blended course. The e-partof the course is delivered using moodle. Students have to have an enrolment key from thecourse tutor to access the course.
-
8/14/2019 Online Environment Evaluation
3/25
Course Home Page
Forums Page
-
8/14/2019 Online Environment Evaluation
4/25
Topic (4) page
Purposes of the study
This short paper aims to:
1. Specify the degree of student participation in discussion forums in terms of number ofmessages posted by each student.
2. Analyze the quality of messages in two topics of Thinking and Creativity blendedcourse posted by students in terms of Self-introduction, Repeating information in anothermessage, responding to accept or reject others views and opinions without explanation,
responding to accept or reject others views and opinions with more explanation, contextrelated, clarity, grammar and provocative.
Questions of the study:
1. How many messages are posted in the whole discussion boards and by eachindividual student?
2. For which extent is the quality of messages posted by students in terms of Self-introduction, Repeating information in another message, responding to accept orreject others views and opinions without explanation, responding to accept or reject
others views and opinions with more explanation, context related, clarity, grammarand provocative?
-
8/14/2019 Online Environment Evaluation
5/25
Evaluation Methodology
These researchers will utilize a quantitative method in this study to find out the totalnumber of messages posted by student. Moreover, a qualitative methodology will be
applied to evaluate the quality of messages posted by students. A checklist will be used toanalyze messages' contents. Checklist consists of seven items that evaluate the messagefrom different aspects. First, the checklist tends to examine if the student introduceshim/herself at the beginning of the post. The second item examines repeating informationin another message. The third and fourth items examine the student's response to acceptor reject other's views with or without explanation. The rest examine the message clarity,context related and correct grammar. Moreover, the checklist examines massageproactive. Proactive means that the message prompts other student to respond.
Results and Discussion
Question 1 finding:
Question 1 was: How many messages are posted in the whole discussion boardsand by each individual student? For answering this question the discussion boards havebeen observed throughout the course. The average number of messages sent by studentsduring studying the whole course includes:
1. The total number of messages posted to the discussion boards
In this course, seven discussion topics were addressed by the course tutor. Resultsfrom observing the discussion board showed that the total responses of students'messages are 878 through out the course. Assuming that each student from those whoenrolled in the course (240 students) should participate at least by posting one message toeach topic, this number (878) represents only 52.3% of the expected total number ofmessages (1680) that should be sent to the discussion boards (Table 1).
The total number of messages posted to the discussion boards (Table 1)Number ofstudents
Total number of messagesassumed to be sent
Actual total number ofmessages sent by
students
Percent
240 7*240= 1680 878 52.3%
-
8/14/2019 Online Environment Evaluation
6/25
2. The total number of messages posted to each topic, number of posts and
number of replies to discussion board:The number of messages posted to each topic differs from topic to other topics. This
difference may be as a result of how much students like the topic. In addition, the degree
of students' motivation to reply and post differ from topic to another according to theirinterests. Table 2 shows the total number of messages posted to each topic. It is clear thatSuccess Secrets topic has the highest number of posts (132) and the highest number ofreplies (317). However, topic No.6 that asks students to critique a video clip for Dr. AL-Faqi has the lowest number of replies.
The total number of messages posted to each topic, number of posts and number of
replies to discussion board (Table 2)
TopicNumber
Topic title Number of posts Number of replies
1. - 1 45
.2 95 140
.3 132 317
4. 1 38
5. 13 19
6. . 13 12
7. : 1 51
Sub Total 256 622
Total 878
3. The minimum and maximum number of messages per topic
As discussion boards have been observed, we found that the minimum number ofmessages per topic posted is 25 messages, whereas the maximum number of messagesper topic posted per topic is 449 messages as shown in table 3.
-
8/14/2019 Online Environment Evaluation
7/25
The minimum and maximum number of messages per topic (Table 3)
Number ofmessages per topic
N Minimum Maximum Mean
7 25 449 125.42
4. The number of messages posted per student
Students' responses varied from ignoring all topics or some topics in the discussion
boards to positive involvement. The minimum number of messages per student sent to
the discussion boards was zero (Table 4). Table 5 shows that the majority of students
posted between one and thirty messages. Around 112 students choose not to respond
to any topic in discussion boards.
The minimum and maximum number of messages per student (Table 4)
Number ofmessages per
student
N Minimum Maximum Mean
240 0 74 3.56
The number of messages per student(Table 5)
Number ofmessages per
student
Frequency Percent ofstudents
0 112 47%
1 20 8%
2 27 11%
3 15 6%
4 9 4%
5 6 3%
6 9 4%
7 2 1%
8 8 3%
9 6 3%
-
8/14/2019 Online Environment Evaluation
8/25
Number ofmessages per
student
Frequency Percent ofstudents
10 2 1%
11 4 2%
12 2 1%
13 3 1%
14 2 1%
15 2 1%
16 1 0%
17 2 1%
20 2 1%
22 1 0%
25 1 0%
29 2 1%
30 1 0%
74 1 0%
Question 2 finding:
Question 2 was: For which extent is the quality of messages posted by students intwo topics in terms of Self-introduction, Repeating information in another message,responding to accept or reject others views and opinions without explanation, respondingto accept or reject others views and opinions with more explanation, context related,clarity, grammar and provocative?
Topic1:
Topic one asks students to post references of human resources development. Thistopic has 45 replies. Table 6 shows frequencies of messages after evaluating posts
quality.
-
8/14/2019 Online Environment Evaluation
9/25
Table 6
INT_SEL
INT_SEL
noyesFrequency
40
30
20
10
0
Chart (1) Number of messages that contain self introduction in topic No (1)
Around nine messages contain self introduction before the real post. Some examples forthe replies that have self-introduction are:
Frequency
(Grammer)
Frequency
(Prov)
Frequency
(Clear)
Frequency
(Related)
Frequency
(Int-Acex)
Frequency
(Int-Acc)
Frequency
(Int-Rep)
Frequency
(Int-sel)
yes 36 36 45 22 1 44 1 9no 9 9 23 44 1 44 36
Total
45 45 45 45 45 45 45
-
8/14/2019 Online Environment Evaluation
10/25
INT_REP
INT_REP
noyesF
requency
50
40
30
20
10
0
Chart (2) Number of messages that contain repeated statements from anothermessages in topic No (1)
The chart shows that one message only contains repeated statement. That isbecause students tend to post their responses to the original topic directly without reading
or commenting on others posts.
-
8/14/2019 Online Environment Evaluation
11/25
INT_ACC
INT_ACC
noyesF
requency
40
30
20
10
0
Chart (3) number of messages that shows students replies to accept or rejectothers views and opinions without explanation in topic No (1)
The chart shows that most students responded to accept or reject others'views andopinions without explanation. Example of message that accept others' view without
explanation:
-
8/14/2019 Online Environment Evaluation
12/25
INT_ACEX
INT_ACEX
noyesF
requency
50
40
30
20
10
0
Chart (4) number of messages that shows students replies to accept or rejectothers views and opinions with explanation in topic No (1)
The chart shows that most students responded to accept or reject others'views andopinions without explanation. Example of message that accept others' view with
explanation:
-
8/14/2019 Online Environment Evaluation
13/25
RELATED
RELATED
noyesF
requency
30
20
10
0
Chart (5) Number of messages that are related to topic context in topic No (1)
Around 22 messages are related to the context of the topic posted by the tutor. However,around 23 messages are not related to the topic context.
CLEAR
CLEAR
yes
Frequency
50
40
30
20
10
0
Chart (6) Number of clear messages in topic No (1)
The chart shows that all messages posted by students are clear.
-
8/14/2019 Online Environment Evaluation
14/25
PROV
PROV
noyes
Frequency
40
30
20
10
0
Chart (7) Number of messages that prompt students to respond in topic No (1)
Around 36 messages are provocative, whereas the rest are not.
-
8/14/2019 Online Environment Evaluation
15/25
GRAMMER
GRAMMER
noyesF
requency
40
30
20
10
0
Chart (8) number of grammatically correct messages in topic No (1)
The chart shows that around 36 messages are grammatically correct. That is may bebecause most messages are copied from different sources without any changes.
Topic2:
This topic asks students to post some stories about successful people. This topic has 38replies. Table 7 shows frequencies of messages after evaluating posts quality.
Table (7)
Frequency
(Grammer)
Frequency
(Prov)
Frequency
(Clear)
Frequency
(Related)
Frequency
(Int-Acex)
Frequency
(Int-Acc)
Frequency
(Int-Rep)
Frequency
(Int-sel)
yes 35 30 32 28 4 2 9no 3 8 6 10 38 34 36 29
Total
38 38 38 38
38 38 38
-
8/14/2019 Online Environment Evaluation
16/25
Chart (9) Number of messages that contain self introduction in topic No (4)
Most of the replies does not have self introduction. Some examples for the repliesthat have self-introduction are:
INT_SEL
INT_SEL
noyesFrequen
cy
40
30
20
10
0
*
*
- - *
*
*
-
8/14/2019 Online Environment Evaluation
17/25
Chart (10) Number of messages that contain repeated statements from
another messages in topic No (4)
There are about 2 repeated replies in this topic and this may be by mistakes from thestudents because they were done by the same students and at the same time with thepost which is repeated from.
INT_REP
INT_REP
noyesFrequen
cy
40
30
20
10
0
-
8/14/2019 Online Environment Evaluation
18/25
Chart (11) number of messages that shows students replies to accept or reject
others views and opinions without explanation in topic No (4)
INT_ACC
INT_ACC
noyesFrequen
cy
40
30
20
10
0
-
8/14/2019 Online Environment Evaluation
19/25
Chart (12) number of messages that shows students replies to accept or
reject others views and opinions with explanation in topic No (4)
There are about 4 students who comments for other replies by agree or disagreewithout explanation and no replies by agree or disagree with explanation. This maybecause of the nature of the topic which is written real stories of people success in theirlife. Because the stories are real there is no agree or disagree. Almost of the replies aredifferent stories rather than comments for others. Some example for those replies is below:
INT_ACEX
INT_ACEX
noFrequen
cy
40
30
20
10
0
...
..
...
-
8/14/2019 Online Environment Evaluation
20/25
Chart (13) Number of messages that are related to topic context in topic No(4)
Ten replies from (38) replies are unrelated to the main topic. One example ofthe unrelated replies is below:
RELATED
RELATED
noyesFrequen
cy
30
20
10
0
.
-
8/14/2019 Online Environment Evaluation
21/25
This reply is not related to the topic and it is not related to others replies. Most ofthe unrelated replies are taking about tips in how to success in life which is interfered withthe main topic. So, it maybe the misconception of the topic is the reason.
Chart (14) Number of clear messages in topic No (4)
Most of the replies in this topic are clear accept about 5 of them. They are not clearbecause they are not related to the topic and not related to other replies which means thatthey are not meaningful.
CLEAR
CLEAR
noyesFrequency
40
30
20
10
0
-
8/14/2019 Online Environment Evaluation
22/25
Chart (15) Number of messages that prompt students to respond in topic No (4)
Most of the replies are provocative which means that they encourage othersto reply and read them. This may be because of the nature of the topic. The mostprovocative replies are those which talk about stories about successful Omani people. Thismay be because they are more related to the participants' context and culture.
PROV
PROV
noyesFrequency
40
30
20
10
0
-
8/14/2019 Online Environment Evaluation
23/25
Chart (8) number of grammatically correct messages in topic No (1)
Most of the replies in this topic are written in good grammar and using correct Arabiclanguage.
Pros and Cons of using discussion boards in blendes courses
Instructors in face-to-face universities should think carefully before integrating
online learning for their students. One advantage of using such technology is that it allowsasynchronous communication between classmates and course tutor. However, studentswho prefer face-to-to-face communication can not respond effectively in discussionboards. Moreover, instructors should specify the evaluation criteria for students at thebeginning of the course to lead the discussion messages posted by students. In addition,course instructor should participate on the discussion boards and comments on students'posts which will influence and motivate students to respond.
GRAMMER
GRAMMER
noyesFrequen
cy
40
30
20
10
0
-
8/14/2019 Online Environment Evaluation
24/25
References:
1. http://www.mirandanet.ac.uk/ejournal/uploads/48/Critique.pdf
2. www.nnu.edu/fileadmin/Elearning/bb_tips/BbAssess_EvaluateDiscussion.pdf3. www.stjohns.edu/media/3/c84d5c3eb0f54c19baba0ba5e59af116.pdf4. http://www.surveyconsole.com/console/TakeSurvey?id=570745
-
8/14/2019 Online Environment Evaluation
25/25
Appendices
Appendix I : checklist
N Item Yes No1. The student introduces him/herself at the beginning of the post
2. The student repeats information in another message.
3. The student Responds to accept or reject others views and opinionswithout explanation.
4. The student Responds to accept or reject others views and opinions withmore explanation.
5. The student replies to the post in the same context.
6. the student reply is clear
7. The student reply stimulate others responses.
8. The student reply is grammatically correct.