normative and self-referenced feedback
TRANSCRIPT
The Effects of Normative and Self-Referenced Feedback on
Students’ Motivation, Punctuality, and Essay Lengths in Online
Learning Environments
Tae Seob ShinEPET, CEPSE
Overview
• Theoretical Background• Research Purpose• Research Hypotheses & Questions• Research Design• Sample• Procedures• Measures• Analyses• Results• Discussion• Limitations• Significance
Theoretical Background
• Research on Online Learning:– Potential benefits of Web-based courses– Characteristics of online learning– Student engagement in online learning environments– Effective teaching strategies
(Davies & Graff, 2005; Gilbert et al., 2007, Lebaron & Miller, 2007; Tallent-Runnels et al., 2006)
• Research on Online Learners’ Motivation– Focuses on intrinsic motivation (Sins et al., 2007; Hoskins & van
Hooff, 2005)
– Little research on why and how students are engaged why and how students are engaged in an academic activityin an academic activity in an online learning environment (Wang & Reeves, 2007)
Theoretical Background
• Achievement Goal Orientation Theory– A goal orientation (achievement goal)goal orientation (achievement goal): a specific goal
that individuals strive to attain in achievement contexts (Urdan & Schoenfelder, 2006; Ames, 1992; Kaplan & Maehr, 2006)
• A Mastery vs. Performance Goal– A mastery goalmastery goal: a concern with developing
competence and skills– A performance goalperformance goal: a concern with demonstrating
competence to others by appearing capable or outperforming others
(Urdan & Schoenfelder, 2006; Ames, 1992; Kaplan & Maehr, 2006)
Theoretical Background
• Goal Structures– Goal structuresGoal structures: messages in the learning
environment that make certain goals salient– Examples: task, evaluation, recognition, & authority– Affect students’ perception of the class -> students’
goal orientations(Ames, 1992, Urdan & Schoenfelder, 2006; Urdan, 2004)
• Manipulation Strategies – Examples: task instructions, reasons for engaging in
an activity, and type of feedback – Students’ perception about classroom structure can
be manipulated by the instructor(Ames, 1984; Butler, 1987; Jagacinksi & Nicholls, 1987; Elliott & Dweck, 1988)
Theoretical Background
• Feedback and Achievement Goals– Different types of feedback -> different motivational
orientations– Self-evaluation & self-improvement (Butler, 1987; Butler, 1995)
• Feedback in Online Learning Environments– Communication tools embedded in learning platforms– Research on effective online teaching: importance of
providing specific, objective, and individual feedback(Lebaron & Miller, 2005; Mory, 2004; Tallent-Runnels et al., 2006)
Theoretical Background
• Normative Feedback– Prompt students to compare their performance with
their peers– Provides interpersonal norms
• Did I do better than other students in the class?
– Performance goals: social comparison
• Self-referenced Feedback– Prompt students to look back on their own
performance– Provides internal norms
• Have I improved?
– Mastery goals: self-improvement, self-development(Ames, 1992; Brophy, 2004; Kaplan & Maehr, 2006; Urdan & Schoenfelder, 2006)
Research Purpose
• To examine the effects of normative and self-referenced feedback on:
1. Students’ motivation– mastery goals– performance goals– interest in the course
2. Students’ academic performance– punctuality of assignments– essay lengths
Research Hypotheses
• H1: Students who receive self-referenced graphical feedback should report higher gains in mastery goal mastery goal orientationsorientations than those who receive normative graphical feedback
• H2: Students who receive normative graphical feedback should report higher gains in performance goal performance goal orientationsorientations than those who receive self-referenced graphical feedback
• H3: Students who receive self-referenced graphical feedback should report higher gains in interest in the interest in the coursecourse than those who receive normative graphical feedback
Exploratory Questions
• Q1: Do students become more punctual in submitting their assignment after they receive feedback about punctuality?
• Q2: Do students write longer essays after they receive
feedback about essay lengths?
Research Design
• Cross-Over design
R1 O1 T1 O2 T2 O3R2 O1 T2 O2 T1 O3
R: Random Assignment (by Groups)O: Observations (Pre, Post, & Post Post-test)T1: Normative FeedbackT2: Self-Referenced Feedback (Petersen, 1985)
• Random Assignment– Group NS NS (R1): normative feedback (T1) -> self-
referenced feedback (T2)– Group SN SN (R2): self-referenced feedback (T2) ->
normative feedback (T1)
Sample
• Sample: graduate students enrolled in an online Master's program during the summer semester of 2008
• Total N = 29
Test Sample Size
Pre-Test N = 28
Post-Test N = 25
Post Post-Test N = 23
Procedures
• Pretest– First weekFirst week of the semester– Survey
• Mastery goals
• Performance goals
• Interest
– Sample Size• Group NS: 14
• Group SN: 14
Procedures
• Treatment 1– First half of the courseFirst half of the course– Graphical feedback on how punctually students
submitted their work each week– Feedback about punctuality was given twice in week
3 and week 4 via email
• Group NSNS: Normative Feedback • Group SNSN: Self-referenced feedback
Norm
ative Feedback
Self-R
ef Feedback
Procedures
• Posttest– End of the first halfEnd of the first half of the semester– Survey
• Mastery goals
• Performance goals
• Interest
– Sample Size• Group NS: 12
• Group SN: 13
Procedures
• Treatment 2– Second half of the courseSecond half of the course– Graphical feedback on the length of their assignments
each week
– Feedback about the essay lengths was given twice in week 7 and week 8 via email
• Group NSNS: Self-referenced feedback • Group SNSN: Normative feedback
Norm
ative Feedback
Self-R
ef Feedback
Procedures
• Post-Posttest– End of the semesterEnd of the semester– Survey
• Mastery goals
• Performance goals
• Interest
– Sample Size• Group NS: 10
• Group SN: 12
Measures• Achievement Goal Orientations
– Mastery Goals: 5 items• my goal in this class is to learn as much as I can about this topic
– Performance Goals: 5 items• it is important for me to do well compared to others in this class
– Interest in the Course: 5 items• I enjoy this course very much
(Barron & Harackiewicz, 2003; Harackiewicz et al., 2000)
• Academic Performance– Punctuality of Assignment: an hourly difference between
the deadline and the actual submission time
– Essay Lengths: a difference between the recommended essay length and the actual essay length
Analyses
• H1, H2, & H3: – 2 (group NS vs. group SN) x 3 (pretest vs. posttest
vs. post-posttest) repeated measures ANOVA
• Q1 & Q2:– Q1: 2 (group NS vs. group SN) x 3 (week 2 vs. week
3 vs. week 4) repeated measures ANOVA– Q2: 2 (group NS vs. group SN) x 3 (week 7 vs. week
8 vs. week 9) repeated measures ANOVA
Results (H1: Mastery Goals)
• Finding– No significant
interaction effect between group and time (F < 1)
• Interpretation– H1 rejected
Figure 1. Mean mastery goals score by test occasion and groupNS: N->SSN: S->N time 1: pretest; 2: posttest; 3: post-posttest
Results (H2: Performance Goals)
• Finding– No significant
interaction effect between group and time (F(2, 40) = 2.07, p = .14)
• Interpretation– H2 rejected– The direction of
changes in performance goals was consistent with the hypothesis
Figure 2. Mean performance goals score by test occasion and groupNS: N->SSN: S->N time 1: pretest; 2: posttest; 3: post-posttest
Results (H3: Interest)
• Finding– No significant
interaction effect between group and time (F < 1)
• Interpretation– H3 rejected– The direction of
changes in interest was consistent with the hypothesis
Figure 3. Mean interest score by test occasion and groupNS: N->SSN: S->Ntime 1: pretest; 2: posttest; 3: post-posttest
Results (Q1: Punctuality)
• Finding– No significant interaction
effect between group and time (F(2, 54) = 2.30, p = .11)
– No significant main effects of group (F < 1) and time (F < 1)
Figure 4. Mean punctuality score by feedback type and group.NS: normative feedbackSN: self-referenced feedbacktime 1: week2 (pretest); 2: week 3; 3: week 4
Results (Q2: Essay Lengths)
• Finding– No significant interaction
effect between group and time (F < 1)
– No significant main effects of group (F < 1) and time (F < 1)
Figure 5. Mean essay lengths score by feedback type and groupNS: normative feedbackSN: self-referenced feedbacktime 1: week7 (pretest); 2: week 8; 3: week 9
Discussion
• The findings did not support the hypotheses. In other words, students’ motivation and academic performance were not affected by the types of graphical feedback they received.
• However, the direction of changes in students’ performance goals and interest in the course after feedback was consistent with the hypotheses.
Limitations
• Weak treatment– No explicit statement– Interpretation of graphs
• A cross-over design – Issue of equal treatment of subjects– Treatment– Complex analysis of effects
• Small sample size – Issue of statistical analysis
Significance
• Theoretical perspective– Experimental nature– Application of traditional motivation theory in online
learning environments
• Instructional perspective – Use of a semi-automated feedback mechanism
Thank you !!!