evaluation report for fdp design activity on active ...€¦ · evaluation report for fdp design...
Post on 18-Oct-2020
3 Views
Preview:
TRANSCRIPT
Team 117 – Self Evaluation Report
1
Evaluation Report
For
FDP Design Activity on
Active Learning - Peer Instruction
and Think-Pair-Share
Team Id : 117
Team Details:
Team Leader : Dr.D.Jeya Mala
Team Member 1: Dr.Harun Patel
Team Member 2: Mr.Sanju Mahawar
Team Member 3: Dr.Ranjan Kumar Mandal
Team 117 – Self Evaluation Report
2
TABLE OF CONTENTS
I. ANALYSIS OF ENGAGEMENT 3
A) What was the gross engagement in the course? [Macro level] 3
B) What was the engagement pattern of individual LeDs, LbDs, LxIs, and LxTs?
[Micro Level] 5
C) What is the transition pattern of engagement of learners across LeD, LbD, LxI,
and LxT? 7
II. ANALYSIS OF LEARNING 8
A) What is the gross learning in the course?[Macro] 8
B) What is the learning pattern across LbDs and Quizzes? [Micro] 10
C) What is the transition pattern of learning across LbDs and Quizzes? [Meso
Level] 11
III. ANALYSIS OF PERCEPTION 12
What was the perception of learners about the course? [Macro] 12
REFLECTIONS ON ANALYSIS 18
What do gross data about learning, perception and engagement tell about the
course? 18
What does the analysis tell about the course experience? 18
What needs improvement in the current course offering? 18
IV. WORKLOAD DISTRIBUTION 20
Team 117 – Self Evaluation Report
3
I. ANALYSIS OF ENGAGEMENT
A) What was the gross engagement in the course? [Macro level]
The MOODLE course on Active Learning – Peer Instruction and Think-Pair-Share was originally
designed as part of a blended course which was to be taken by participants of IITBx – FDP301x -
Mentoring Educators in Educational Technology. There were 7 peer reviewers who have evaluated this
course and 4 teachers along with 14 colleagues and so a total of 25 participants have registered for this
course.
The course involved the following activities that were to be completed by participants during the duration
of the course:
LeDs on Active Learning – 1 No.
LeDs on PI – 3 Nos
LeDs on TPS - 2 Nos
LbDs on Active Learning , PI and TPS- 6 Nos
Discussion on Active Learning , PI and TPS - 3
Reflection Quiz (RQ)
Advancing Your Learning (Lxt) - 8 Videos and 3 Resources
Assimilation Quiz (AQ)
Knowledge Quiz (KQ)
Resource Creation Assignment (RCA) - Designing your own PI and TPS
Peer review of RCAs
Evaluation and Feedback
During this FDP design activity, as a team we have discussed a lot on how to develop it as an effective
tool for the learners to practice Active Learning in their classroom by implementing Peer Instruction and
Think-Pair-Share (TPS) activities.
The team has applied TPACK model and Kirkpatrick's Training Evaluation Model throughout the
design of the course.
The activity completion report generated from the course Reports section shows the engagement of
participants in the activities as shown in Table 1.]
Table 1: Activity Completion Summary
Team 117 – Self Evaluation Report
4
Activity Completion Status
LeDs 1 25
2 25
3 25
4 25
5 25
6 25
LbDs 1 16
2 18
3 17
4 16
5 16
6 16
Discussion on LxT 16
Discussion on LxI 15
RQ 19
LxTs 11
AQ 17
KQ 19
RCA - Design your own PI and TPS -
Peer Review PI and TPS -
Team 117 – Self Evaluation Report
5
B) What was the engagement pattern of individual LeDs, LbDs, LxIs, and
LxTs? [Micro Level]
Looking into the performance and engagement of individuals in the course activities, following is the
engagement pattern of selected participants (those who have peer reviewed our course design) in the
course. Seven participants were selected for the analysis so that we have a sample groups showing high
engagement, medium engagement and low engagement pattern. Table 2 shows the level of engagement
of the seven sampled participants in the course.
Table 2: Participation of individual learners in the course
Participant
1
Participant
2
Participant
3
Participant
4
Participant
5
Participant
6
Participant
7
LeDs 1 Yes Yes Yes Yes Yes Yes Yes
2 Yes Yes Yes Yes Yes Yes Yes
3 Yes Yes Yes Yes Yes Yes Yes
4 Yes Yes Yes Yes Yes Yes Yes
5 Yes Yes Yes Yes Yes Yes Yes
6 Yes Yes Yes Yes Yes Yes Yes
LbDs 1 Yes Yes Yes Yes Yes Yes Yes
2 Yes Yes Yes Yes Yes Yes Yes
3 Yes Yes Yes Yes Yes Yes Yes
4 Yes Yes Yes Yes Yes Yes Yes
5 Yes Yes Yes Yes Yes Yes Yes
6 Yes Yes Yes Yes Yes Yes Yes
Discussion
on PI
Yes No Yes No Yes No Yes
Team 117 – Self Evaluation Report
6
Discussion
on TPS
No No No Yes No Yes No
RQ Yes No Yes Yes Yes Yes Yes
LxTs No No Yes No Yes No Yes
AQ Yes No Yes Yes Yes Yes Yes
KQ Yes No Yes Yes Yes Yes Yes
RCA -
Design
your own
PI and TPS
-- -- -- -- -- -- --
Peer
Review
RCA - PI
& TPS
-- -- -- -- -- -- --
From table 2, it can be inferred that, the sample participants taken from the list have attended the
initial set of activities to attend the LbDs based on LeDs. But then, the extended resources such
as LxTs in terms of videos and research papers have been given less importance and many of the
listeners have not viewed them.
Similarly, the sample data shows that, the participants are posting their replies to a single
discussion forum topic only and not the next one. This seems that, the discussion forums must be
restricted to one for any of the topic and several threads should be created from that topic.
Team 117 – Self Evaluation Report
7
C) What is the transition pattern of engagement of learners across LeD,
LbD, LxI, and LxT?
Team 117 – Self Evaluation Report
8
II. ANALYSIS OF LEARNING
A) What is the overall learning in the course “Peer Instruction &
Think Pair Share” at macro level?
Evaluation of the learning of the participants can be known by evaluating the marks
scored by the participants in attempting Knowledge quiz (KQ), Assimilation quiz (AQ),
Reflection quiz (RQ) and LbDs activities. Team were incorporated eight quizzes including
KQ,AQ,RQ and LbDs. And weighted score for each activity as follows:
Learning by Doing (LbD) Activities [50 marks]
Reflection Quiz [10 marks]
Assimilation Quiz [10 marks]
Knowledge Quiz [10 marks]
Peer Instruction Design Activity [20 marks]
Total 100 marks
Team 117 – Self Evaluation Report
9
Table 1: No. of participant’s attempted, average marks, maximum marks and average percentage
of participants who attempted each activity and their score.
LbDs KQ AQ RQ Activity Any All
No of
Participants
Attempted
15 16 16 15 0 11 8
Average
Marks
24.26 3.68 4.25 3.92
0 72 55.23
Maximum
Marks
50 10 10 10 20 100 100
Average
Percentage
48.52
36.8 42.5 39.2 - 72 55.23
Team 117 – Self Evaluation Report
10
B) What are the patterns of marking to learners across LbDs and
Quizzes?
Table 2: No. of participant’s attempted, average marks, maximum marks and average percentage
of participants who attempted each activity and total is calculated in 50 as it was the weightage
given to the LbDs in the whole course scoring.
LbD 1 LbD 2 LbD 3 LbD 4 LbD 5 Any of the
LbDs
All of the
LbDs
No of
Participants
Attempted
18 16 16 14 15 4 14
Average
Marks
7.88 6.28 8.28 8.28 7.73 13.12 39.3
Maximum
Marks
10 10 10 10 10 50
(Weighted)
50
(Weighted)
Average
Percentage
78.8 62.8 82.8 82.8 77.3 26.24 78.6
Team 117 – Self Evaluation Report
11
c) What is the transition structure of learning across LbDs and Quizzes?
Team 117 – Self Evaluation Report
12
III. ANALYSIS OF PERCEPTION
What was the perception of learners about the course? [Macro]
Analysis of perception of the designed FDP has been done through the feedback of the
participant in the three surveys:
1) Perception of Learning
2) Perception of Engagement
3) Perception of Usability
The detail of survey and responses are given below
1) Perception of Learning
Total 15 candidates have given their feedback on survey of “Perception of Learning”.
Table 3.1. Responses for the perception of learning
Questions Asked Strongly
Disagree
Mildly
Disagree
Neutral Strongly
Agree
Mildly
Agree
I understood the learning
objectives 0 1 2 9 3
I was able to relate each
learning objectives to the
learning I achieved
0 4 2 7 2
I was appropriately
challenged by the course
material
0 0 1 7 7
My learning was enhanced
due to the course 0 1 2 7 5
Team 117 – Self Evaluation Report
13
Figure 3.1. Bar diagram for the perception of learning
From the obtained responses, it can be seen that among the 15 participant, maximum participants
are strongly agree about perception of learning. 04 participants were saying that they were mildly
disagreeing in “relating the learning objectives to the learning they have achieved”. This result
suggests that there is need of further slender improvement in the course content.
2) Perception of Engagement
Total 12 candidates have given their feedback on survey of “Perception of Engagement”
Table 3.2. Responses for the Perception of Engagement
Questions Asked Strongly
Disagree
Mildly
Disagree
Neutral Strongly
Agree
Mildly
Agree
I was well engaged during the
course 0 0 2 7 3
It was easy for me to get actively
involved each time I accessed the
course
0 0 1 5 6
I was given ample opportunity to
practice the skills I was to learn 0 1 1 5 5
I found quizzes/activities 1 1 2 3 5
Team 117 – Self Evaluation Report
14
interesting
I found the discussion forum
threads motivating enough to
participate regularly
1 1 2 4 4
I viewed/read all the additional
resources/reference materials. 0 1 2 7 2
Figure 3.2. Bar diagram for the perception of engagement
Figure 3.3. Bar diagram for the perception of engagement
From the feedback of participants (Bar Diagram) about the perception of engagement, we
concluded that all the participants were well engaged actively involved during the course and
they also read and viewed the additional resource/reference material given in the course. One
Team 117 – Self Evaluation Report
15
participant is strongly disagreeing in question asked related to the quizzes and discussion forum,
it indicate that there is need of improvement to make the quizzes and discussion thread more
interesting.
2) Perception of Usability
Total 11 candidates have participated in the feedback of perception of usability
Table 3.3. Responses for the Perception of Usability
Questions Asked Strongly
Disagree
Mildly
Disagree
Neutral Strongly
Agree
Mildly
Agree
I found the course materials easy to
navigate 1 2 1 5 2
I was comfortable with the pace of the
program 1 2 1 4 3
I was comfortable with the duration of
the session 2 4 1 3 1
I found consistency in the course
material design 2 0 2 5 2
I found the course material clear and
understandable 1 1 0 4 5
I was given ample opportunity to
share my suggestion/feedback 1 2 1 3 4
I was given ample opportunity to get
answers to my queries 1 1 2 5 2
I found the discussion forum threads
relevant 1 1 0 2 7
I think that I would like to frequently
use what I learnt. 0 1 2 4 4
Team 117 – Self Evaluation Report
16
Fig. 3.4 – Bar diagram for the perception of Usability
Fig. 3.5 – Bar diagram for the Perception of Usability
Fig. 3.6 – Bar diagram for the Perception of Usability
Team 117 – Self Evaluation Report
17
We obtained multiple responses for the perception of usability (bar diagram and questions asked
in the feedback for the perception of usability). Majority of participants are strongly agreed in all
the question asked to them about the usability of the course. Two participants are strongly
disagreeing about the duration of the programme and consistency in the course material. There is
need to reduce the time of FDP and also slight changes are needed in the consistency of the
course.
Team 117 – Self Evaluation Report
18
REFLECTIONS ON ANALYSIS
1. What do gross data about learning, perception and engagement tell
about the course?
The gross data about engagement shows that the number of participants completing the LbDs/quizzes and LxTs / LxIs and discussion forum activities have decreased as the course progressed.
Also, from the data on scores of participants, we can see that the scores of the participants are gradually fluctuating (3.68% in KQ, 3.92% in RQ and 4.25% in AQ), although the number of participants attempting the final Knowledge Quiz (KQ) has drastically fallen (64% of the total participants enrolled). This indicates that the perceived learning value of
the course needs improvement. From the participants' perception of the course, it can be observed that the pace of the course and consistency of course material design needs improvement.
2. What does the analysis tell about the course experience?
Some of the positive sides in this analysis are
Learning part is good as the LeDs are viewed by all
LbDs are mostly attended by the participants
At least one of the discussion forums have been attended
Some of the LxT resources have been viewed
The analysis shows that the course needs the following improvements
Users engagement
levels of usability
3. What needs improvement in the current course offering?
The following sections need considerable improvement:
Engagement:
Engagement in Discussion forums
Completion of LxT resource readings
Completion of Activities and Quizzes Usability
Consistency in course material design
Duration of the session Learning
Reference materials/additional resources
More samples for practice
Team 117 – Self Evaluation Report
19
From the above sections, the following inferences have been made:
The FDP design has a lot of good features from which the participants were able to apply
this active learning strategy by means of Peer Instruction and Think-Pair-Share in their regular
class room set-up.
Based on the observations, it has been identified that, the time to complete the course
should be reduced in terms of reduction of number of LeDs, LxTs, LxI discussion forums and
LbD attempts to complete the course.
Also, to encourage the participants to be more engaged more examples to be provided
based on their domain such as Computer Science, Pharmacy, Bio-Technology, Maths etc.
In order to improve the scoring of the participants in Quizzes, the LbDs design must be
improved.
As a summary, this FDP design activity provides a good learning experience to the
learners in spite of the above said improvements.
Team 117 – Self Evaluation Report
20
IV. WORKLOAD DISTRIBUTION
The team consisted of 4 participants. The workload is distributed among them as follows:
1. Member 1: (Team leader)
1. Preparing the workload distribution 2. LxTs, LxIs; 3. Discussion Forums;
4. Resource Creation Assignment1 (RCA 1) for Peer Instruction; 5. Self Enrollment in Moodle 6. Progress Bar creation 7. Uploaded LxT resources to Moodle.
8. FDP design analysis report: Analysis and reflection of Engagement 9. FDP design report: reviewing and compiling reflections by team members
2. Member 2
1. Course Design Layout
2. Moodle Administrator (Setup) 3. Course Entry Survey, Exit Survey; 4. Feedback Forms; 5. Resource Creation Assignment 2 (RCA 2) for TPS
6. Moodle Gradebook setup and configurations 7. Survey report on Perception of Engagement, Learning and Usability
3. Member 3
1. LbDs,
2. Assimilation Quiz 3. Reflection Quiz 4. Knowledge Quiz 5. FDP design analysis report: analysis and reflection of learning
4. Member 4
1. LeDs 2.1, 2.2, 3.1, 3.2; 2. Uploading of LeDs 3. Moodle Course Creation Video upload; 4. FDP design analysis report: analysis and reflection of perception
top related