responsive pedagogy questionnaire - the developing and validation...
TRANSCRIPT
![Page 1: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/1.jpg)
Responsive Pedagogy Questionnaire
- the developing and validation process
Bringing Teacher Education Forward
08.06.2016
Siv M. Gamlem, Volda University College
Knut Steinar Engelsen, Stord Haugesund University College
Lars M. R. Kvinge, Stord Haugesund University College
Ann Karin Sandal, Sogn and Fjordane University College
Kari Smith, Norwegian University of Science and Technology
![Page 2: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/2.jpg)
“Responsive Pedagogy & Student Learning”- Examine relationship between teachers’ responsive
teaching (feedback practice) and pupils’ learning
outcome in Mathematics
![Page 3: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/3.jpg)
Research question
What is the relation between
responsive pedagogy (feedback
practice) and student learning
(achievements, self-efficacy and self-
regulation)?
![Page 4: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/4.jpg)
Mixed Methods (questionnaire, interview,
video observation, achievement test, -intervention)
Goal for this presentation:- To present the developing and validation process of the Responsive Pedagogy Questionnaire
![Page 5: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/5.jpg)
Responsive Pedagogy – Feedback
Self-regulation skills
Self-efficacyMathematics
Pupils (9th grade)
Le
arn
ing
ou
tco
me
![Page 6: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/6.jpg)
Responsive Pedagogy Questionnaireexamine:
- pupils’ perceptions of self regulation
skills, self-efficacy and feedback
practice in Mathematics
![Page 7: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/7.jpg)
Self-regulation• “Self-regulated learning is an active
constructive process whereby learners set goals for their learning and monitor, regulate, and control their cognition, motivation, and behaviour, guided and constrained by their goals and the contextual features of the environment” (Pintrich & Zusho,2002, p. 64).
• Three phases: 1) forethoughts, 2) performance-volitional control, 3) self-reflection (Zimmerman, 2000).
![Page 8: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/8.jpg)
Self-efficacy
• “Perceived self-efficacy refers to beliefs in one`s capabilities to organize and execute the courses of action required to produce given attainments” (Bandura, 1997,p.3).
• Self-efficacy beliefs are domain specific, and they differ according to physical and social context (Zimmerman, 2000; Bandura, 1997; Skaalvik & Skaalvik,
2004).
![Page 9: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/9.jpg)
Feedback - Responsive Pedagogy
• Hattie & Timperley (2007):
• «Where am I going?»
• «Where am I?»
• «What to do next?»
• Responsive Pedagogy is the recursive dialogue
between the learner’s internal feedback and
external feedback provided by significant others
throughout the three phases of self-regulation;
forethoughts, monitoring and reflection (The RespMath
team, 2015).
![Page 10: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/10.jpg)
A need for «bridges» – pupils’ perceptions
Achievement in
Mathematics
Feedback
practice
Self-
regulation
Self-efficacy
![Page 11: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/11.jpg)
RPQ-M builds on validated instruments• the Norwegian version of the Cross Curricular
Competences (CCC)/PISA-survey
• the Motivated Strategies for Learning Questionnaire
(MSLQ) (Pintrich et al., 1991)
• the Achievement Emotions Questionnaire -
Mathematics (AEQ-M) (Pekrun, Goetz & Frenzel, 2005)
• the Student Perception Assessment survey (SPA) (Havnes et al. 2012)
![Page 12: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/12.jpg)
![Page 13: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/13.jpg)
Theoretical model of construct in questionnaire
![Page 14: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/14.jpg)
Cluster (I): «Forethought»
• FT (18 items)
– FTSR, 4 items; FTFB, 4 items; FTSE, 10 items
– (FTFB) Forethought feedback refers to when a learner seeks answers to prospective learning goals – “where am I going?” These questions can refer to the perceptions of quality of one’s work or learning processes, depending on how they are asked and answered.4 items 4 items 10 items
![Page 15: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/15.jpg)
Cluster (II): «Performance and control»
• PC (29 items)
– PCSR, 8 items; PCFB, 13 items; PCSE, 8 items
– (PCFB) Performance and control feedback refers to when learners seek answers to monitor learning in relation to target (learning goals) – “how am I doing?” These questions can refer to the perceptions of quality of one’s work or learning processes in relation to the target/learning goals.
8 items 13 items 8 items
![Page 16: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/16.jpg)
Cluster (III): «Reflection»
• R (32 items)
– RSR, 7 items; RFB, 7 items; RSE, 18 items
– (RFB) Reflection feedback refers to when learners seek answers to retrospective and prospective learning goals –“what’s next?” These questions can refer to the perceptions of quality of one’s work or learning processes, depending on how they are asked and answered.
7 items 7 items 18 items
![Page 17: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/17.jpg)
First pilot (January 2016)
• Closed-end questions
• 81 items
• Properly organized; easy to use
- Fatigue (?)
• Item stem/readability (Johnsen & Christensen, 2012)
- «Liks» (Björnsson,1968), scale 1-60 (Score 4-45)
• 4-point rating scale (Dillman, 2007)
«agree» – «partly agree» – «partly disagree» – «disagree»
![Page 18: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/18.jpg)
First pilot-test (January 2016)
• Participants: n=10 (9th graders, one school)
– Individually answered on paper
– 2 focus-group interviews (5 participants each)
• Results:
– Rating scale: «partly agree» – «partly disagree» =
the same scale? (changed)
– Fatigue: Similar question asked «before» (add lead-in instructions in sections of questionnaire)
– Item: three removed, one divided in two
– Language: unfamiliar words (changed)
![Page 19: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/19.jpg)
Second pilot (June 2016)
• Questionnaire: 79 items
• Demographics (at the end) (Johnson & Christensen,
2012)
– gender, parents’ education
– «value & expectations from parents» (items
integrated in questionnaire)
• 4-point rating scale (Dillman, 2007)
«strongly agree» – «agree» – «disagree» – «strongly
disagree»
• Participants: n=198 (8 removed – missing data)
![Page 20: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/20.jpg)
Second pilot-test (June 2016)
• Test/establish reliability & validity: Coefficient
alpha (Cronbach’s alpha)
.56 .80 .91 .69 .77 .73 .78 .89 .90
.89 .86 .92
![Page 21: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/21.jpg)
Second pilot-test (June 2016)
• Test/establish reliability & validity: Coefficient
alpha (Cronbach’s alpha)
Self-regulation Feedback Self-efficacy
.77 .92 .95
![Page 22: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/22.jpg)
Second pilot-test (June 2016)
• Factor analysis:
– some items set are multidimensional (different
sets of item tap different construct)
• Adjust items-set; remove some items (strive for
homogeneity)
• Demographics: challenging (missing data)
– “highest education (mother/father)”
– “occupation (mother/father)”
– “grade in mathematics”
![Page 23: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/23.jpg)
Further process
• Demographic data
– item stem?, questions?, instruction?...
• Reduce items (?) (factor analysis)
• Choose design for analysis:
– «Clusters of concept» – «theorethical model»?
![Page 24: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/24.jpg)
Responsive Pedagogy – Feedback
Self-regulation
Self-efficacy skillsMathematics
RespMath Questionnaire- Pupils (9th grade); experimental group (n=1000)- Pupils (9th grade); control group (n=1000)
Learn
ing o
utc
om
e
![Page 25: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/25.jpg)
The RespMath project
• Homepage:
http://prosjektsider.hsh.
no/respmath/
• Facebook: Responsive
Pedagogy in
Mathematics
• Twitter: @RespMath
![Page 26: Responsive Pedagogy Questionnaire - the developing and validation …prosjektsider.hsh.no/respmath/files/2016/06/Gamlem-08.06... · 2016-06-15 · Responsive Pedagogy Questionnaire](https://reader033.vdocuments.us/reader033/viewer/2022042200/5e9facfc97a00f0ad4099176/html5/thumbnails/26.jpg)
- Andrade, Heidi L. (2010). Students as the Definite Source of Formative Assessment: Academic Self-Assessment and
the Self-Regulation of Learning. Nera Conference Proceedings 2010, Paper 25.
http://digitalcommons.uconn.edu/nera_2010/25
- Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioural change. Psychological review, 84(2), 191-
215.
- Björnsson, C. H. (1968). Läsbarhet. Stockholm: Liber
- Butler, D. & Winne, P. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational
Research, 65(3), 245-281.
- Hattie, J. & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112.
- Havnes, A., Smith, K, Dysthe, O., & Ludvigsen, K. (2012). Formative assessment and feedback: Making learning
visible. Studies in Educational Evaluation, 38, 21-27.
- Pekrun, R., Goetz, T. & Frenzel, A.C. (2005). Achievements Emotions Questionnaire – Mathematics (AEQ-M) User’s
manual. Department of Psychology, University of Munich.
- Pintrich, P.R. (2000). The role of goal orientation in self-regulated learning. In: M. Boekaerts, P.R. Pintrich, & M.
Zeidner (Eds.), Handbook of Self-Regulation (pp.452-502). San Diego, CA: Academic Press.
- Pintrich, P.R., Smith, D.A.F., Garcia, T., & McKeachie, W.J. (1991). A Manual for the Use of the Motivated Strategies
for Learning Questionnaire (MSLQ). Ann Arbor, MI: National Center for Research to Improve Postsecondary Teaching
and Learning.
- Usher, E.L. & Pajares, F. (2008). Sources of Self-efficacy in School: Critical Review of the Literature and Future
Directions. Review of Educational Research, 78(4), 751-796.
- Zimmerman, B. J. (2000). Self-regulatory cycles of learning. In G.A. Straka (Ed.), Conceptions of Self-Directed
Learning: Theoretical and Conceptual Considerations (pp.221-34). New York: Waxmann.
- Zimmerman, B. J. (2002). Becoming a Self-Regulated Learner: An Overview. Theory into Practice, 41(2), 64-70.
References