formative questioning in mathematics: an open or closed · pdf fileformative questioning in...
TRANSCRIPT
Formative Questioning in Mathematics: An Open or Closed Case Study?
Introduction
Despite the overwhelming success of Bloom’s Taxonomy of Learning Objectives (Anderson
& Sosniak, 1994) and Black’s and Wiliams’ (1998) extensive research on formative
assessment, questioning still requires development in the mathematics classroom “to check
and probe understanding” (Ofsted, 2012, p.34-35). In the author’s school, questioning in
mathematics has been identified as requiring improvement; this essay is a proposal for a
micro-research study to empirically investigate both the question types and the questioning
techniques which encourage mathematical thinking and participation with the aim of
identifying effective questioning in this school and provide recommendations for
improvement. In this study, question type refers to the mathematical thinking intended and
questioning technique refers to the strategies that teachers put in place for learners to think
about and respond to questions. A literature review of learning objective taxonomies and
their limitations considers those which are suitable to identify question types and levels of
complexity to probe mathematical understanding; a combination of the most relevant
taxonomies discussed will be used to classify questions for the purpose of this study. A
review of Black’s and Wiliam’s theoretical framework on formative assessment analyses
techniques to support formative questioning in mathematics. Based on the conclusions of
the literature review, research questions are presented on the question types and questioning
techniques employed in this school. The research design and methodology is described in
terms of a mixed-method approach of quantitative research methods, by means of lesson
observations and learner questionnaires, and qualitative methods, using semi-structured
interviews with teachers to give them a voice on their intentions. Ethical issues are
considered and sampling implications and analysis rationale discussed, including strategies
to increase the reliability and validity of the study.
Literature Review
Black et al. (2006) believe effective questioning is essential to develop metacognition and
self-awareness, so learners “can ask questions of each other and the focus can move from
the teacher to the pupils” (p.128). However, research shows that teachers’ questioning is
“not always well judged or productive for learning” (DfES, 2004, p.4) and highlights the need
to use “open, higher-level questions to develop pupils’ higher-order thinking skills” (ibid,
p.18). In the 1950s and 60s there were many attempts to produce a hierarchy for the
complexity of thinking skills (Gall, 1970), however it was Bloom’s (1956) Taxonomy that
experienced “phenomenal growth” (Bloom in Anderson & Sosniak, 1994, p.1) and became
widely accepted as the optimal classification (Gall, 1970).
Bloom et al. (1956) were aware of the limitation that the Taxonomy classifies observable
behaviours, so it is not explicit how learning is constructed; instead they hoped the
classification would contribute to the development of a more complete theory of learning.
However, educational changes occurred mostly at policymaker level rather than having direct
influence on teachers (Anderson & Sosniak, 1994). Other criticisms include the omission of
the term ‘understanding’ (Furst, 1994) and the hierarchy implying that ‘knowledge’ leads to
intellectual abilities (Bereiter & Scardamalia, 1999), which are addressed (Figure 1) in a
revised taxonomy (Anderson et al., 2001).
Figure 1 – Bloom’s Original and Revised Taxonomies (Image from ODU, 2013)
Interchanging the top two tiers of the hierarchy perhaps reflects the importance of student-
centred learning in 21st Century education and replacing the nouns with verbs, could indicate
active learner participation, however neither version are intended as a “constructive way of
planning and answering questions” (Morgan & Saxton, 2006, p.19), rather it is a framework
about knowledge so “helps us to see the kind of thinking we can set into action through
questions” (ibid).
Anderson et al., (2001), consider remembering and understanding to be lower-order thinking
skills, while applying, analysing, evaluating and creating are considered higher-order,
however mathematical understanding is not necessarily a linear progression (Sfard, 1991;
Gray & Tall, 1991). Watson (2007) believes that Bloom’s Taxonomy “does not provide for
post-synthetic mathematical actions, such as abstraction and objectification” (p.114) and that
it “underplays knowledge and comprehension in mathematics” (ibid) as these can be
interpreted at different levels of mathematical thought. Watson (2003) also criticises the
simplicity of open and closed questioning as opportunities to extend conceptual
understanding in mathematics are of greater importance.
An alternative taxonomy is Biggs’ and Collis’ SOLO (Structure of Observed Learning
Outcomes) which proposes a sequence of unistructural, multistructural and relational
understanding (Pegg & Tall, 2010, p.174), which Watson (2007) believes “can be used to
devise questions which make finer distinctions than the vague notions of ‘lower-order’ and
‘higher-order’” (p.115). However, while the SOLO model allows for mathematical
abstraction, Watson (2007) argues that what a teacher intends and what a learner perceives
do not necessarily agree.
Smith et al. (1996) agree that Bloom’s Taxonomy has limitations in mathematics and propose
the MATH Taxonomy (Mathematical Assessment Task Hierarchy) for constructing
examination questions (Figure 2).
Figure 2 – MATH Taxonomy (Smith et al., 1996, p.67)
This could be a possible way of analysing verbal questioning in mathematics as the groups
distinguish the hierarchy of different types of activity which require either a “surface
approach” (Smith et al, 1996, p.67) or “deeper approach” (ibid.), rather than a hierarchy of
difficulty.
Andrews et al. (2005) use seven mathematical foci to analyse teachers’ behaviour (Figure 3).
Figure 3 – Mathematical Foci (Andrews et al., 2005, p.11)
According to Watson (2007), these foci describe “the intentions of teaching through
classifying features of mathematical meaning and structure without assuming that learners
necessarily do what is intended” (p.116). If combined with the MATH hierarchy, these foci
could provide a useful framework for analysing mathematical questioning in classroom
discourse (Appendix 1).
There exist other structures which are designed specifically for classifying questioning. For
example Morgan and Saxton (2006) classify questioning in three ways: Probing what is
already known; building a context for shared understanding; and challenging students to think
critically and creatively, however the second category could contain a large array of question
types and levels of complexity. Another distinction is in product-process questioning (Mujis &
Reynolds, 2011), where the former is designed to find the result while the latter is focused
more on the procedure, however in mathematics process is not necessarily considered
higher-order thinking (Dubinsky & McDonald, 2002; Sfard, 1991).
Black and Wiliam believe that classroom discourse should be “thoughtful, reflective, focused
to evoke and explore understanding, and conducted so that all pupils have an opportunity to
think and express their ideas” (1998, p.12) and identify that thinking is inhibited by:
Learners being directed towards an expected response, discouraging their own ideas;
not enough “quiet time” (ibid, p.11) before a response is expected, more commonly
known as wait time, so teachers answer their own questions to maintain pace.
Rowe (1986) states that teachers typically pause for less than one second both after posing a
question and after a response is given, and concludes wait time is crucial to allow students to
both think and expand upon responses; similar conclusions are drawn by Black et al. (2003).
Black and Wiliam (1998) attribute these inhibitors to only a minority of the class participating
and make several recommendations:
Increasing response time;
discussing in pairs or groups first;
providing a choice of answers;
everyone write down an answer, then select a few to share.
Following this research, Assessment for Learning (AfL) materials became a focus for
schools, supported by the National Strategies (Ofsted, 2008), however Ofsted’s review of AfL
concludes that despite the resources and training, teachers still need to “develop their skills
in targeting questions to challenge pupils’ understanding, prompting them to explain and
justify their answers individually, in small groups and in whole class dialogue” (ibid, p.7).
Wiliam (2006) believes that “[t]hrough exploring and ‘unpacking’ mathematics, students can
begin to see for themselves what they know and how well they know it” (p.5) and exemplifies
the original recommendations with strategies specific to mathematics, including the use of
mini-whiteboards, generating discussion from incorrect answers and posing questions which
have multiple solutions. Watson’s and Mason’s (1998) ‘Show me…’ questions have potential
as a mini-white board strategy, to achieve rich discussion and formative feedback.
There is a wealth of additional research on questioning (e.g. Mason, 2000; Brown &
Edmondson, 2001; Wong, 2012) which is beyond the scope of this study.
Formulation of Research Hypotheses
Based on the findings from the literature review, two hypotheses have been formulated:
1. A larger proportion of questions requiring a ‘surface approach’ are used in
mathematics lessons than those requiring deeper thinking.
2. Using formative questioning techniques, supports a wider range of intended
mathematical thinking.
These are statements about the connection between variables and there are both
quantitative and qualitative methods for testing these relationships (Kerlinger in Cohen et al.,
2007); to address construct validity, the hypotheses indicate what the author believes will be
found out from the study. The hypotheses will be investigated using a combination of the
MATH taxonomy framework (Smith et al., 1996) and Andrew et al.’s (2005) mathematical
foci, supported by prompts proposed by Watson (2007) and Wiliam (2006), to analyse the
intended purpose of types of questions employed in mathematics lessons in the author’s
school and the AfL questioning techniques which support deeper thinking.
Research Design and Methodology
A mixed-method approach (Denscombe, 2007) will be used to address the hypotheses from
both the normative, positivist research paradigm, using quantitative methods for statistical
analysis of proportion of question types and techniques employed by teachers in
observations. A more interpretive standpoint utilising qualitative research through semi-
structured interviews to probe deeper into the teachers’ intent will utilise radical listening
(Clough & Nutbrown, 2007) to ensure each teacher’s voice is heard. Learner-voice will be
heard through questionnaires, the closed or open nature of which will dictate the method of
analysis. This methodological triangulation between methods should ensure this research
views things “from as widely different perspectives as possible” (Denscombe, 2007, p.135).
In addition, if the outcomes correspond then greater confidence can be had in the findings
(Cohen et al., 2007).
An intrinsic case study approach (Stake, 2005) is proposed involving a federation of two 11-
16 single-sex schools where the proportion of students who are from minority ethnic
backgrounds or speak English as an additional language is above average. The federation is
non-selective and located on the south-coast of England in a town where two out of the ten
secondary schools are grammars. A case study approach will allow depth of study
(Hitchcock & Hughes, 1995) into different questioning types and techniques; however this
approach has the limitation that any findings could be unique to the case study group
(Denscombe, 2007), however the findings are not intended to “represent the world, but to
represent the case” (Stake, 2005, p.460); it is hoped that any findings can be used to inform
planning across the federation’s mathematics departments. The context of the schools
simply provides the reader the potential for comparison with similar schools.
To ensure the results are representative of the federation, a sample of classes from both
schools will need to be selected (Cohen et al., 2007). A cluster sample of four classes with
which to observe four teachers will be dictated by the author’s availability to minimise cover
implications, hence could be considered a form of convenience sampling, however it will be
purposive to the extent that it will aim to include a variety of key stages, gender and level of
abilities (Denscombe, 2007). Questionnaires will be given to all students in each class and
from the responses a stratified sample of 30 questionnaires will be calculated to allow for
proportional groupings from the population (Robson, 2002). This multi-stage sampling will
continue with selecting students from each category by systematic sampling, an efficient form
of probabilistic sampling provided the names of the students in each category are
randomised first (Cohen et al., 2007). By randomising the data, bias should be kept to a
minimum as each member of each group will have an equal chance of being chosen.
Lesson observation data will be collected using a coding system for both questioning
techniques (Appendix 1) and for question types (Appendix 2) which combines the taxonomies
and recommendations discussed in the literature review and classifies questions into the
following categories:
Factual
Procedural
Structural
Reasoning
Reflective
Derivational
The author will take a “non-participant observer role” (Cohen et al., 2007, p.259), as the
classroom discourse will be coded leaving no time to interact with the proceedings. The fact
that the author will observe and interview the teachers involved and produce the report could
be considered a limitation to the study (BERA, 2011), and will have an effect on the validity
and objectivity of the data collected, so to test inter-observer reliability, another teacher will
code a lesson to check percentage agreement (Robson, 2002). The micro-research study
also needs to be replicable to ensure reliability (Bashir, 2008), so one of the teachers will be
observed again after a couple of weeks to test internal consistency reliability (Cohen et al.,
2007).
A questionnaire was chosen to collect the students’ thoughts as one- to-one contact with the
researcher is not necessary, so a greater number can be sampled. Also the high proportion
of closed questions can be analysed using nominal scales for questions which can be
converted to numbers, or Likert scales for questions relating to attitude or opinion, both of
which are quicker to analyse than the qualitative data from an interview. To ensure flexibility
in participants’ responses however, the last question is open. Questionnaires are also
anonymous, so students may feel more able to be honest about their experiences and to
increase the response rate the questionnaires can be returned anonymously in the envelope
provided. An initial questionnaire (Appendix 3) was piloted on four students from Key Stage
3 and two students from Key Stage 4 to increase reliability and validity (Cohen et al., 2007).
This took students between five and ten minutes to complete and problems were
encountered with questions 5, 7 and 9. Changes were made (Appendix 4) to question 5
which needed further clarification and question 7 was removed on the basis of collinearity
with question 6; question 9 was also removed on both redundancy and reliability grounds
(Cohen et al., 2007). To check test-retest reliability, the students used to pilot the data will be
used again after a period of time to ensure the results are consistent, however as the pilot
participants are known to the author, their results will only be used to measure reliability and
will not be included in the statistical analysis (ibid).
Interviews will be held with the teachers after the lessons to explore their points of view on
what the intended mathematical thinking of the various questions had been (Miller &
Glassner, 2004). This will be semi-structured (Appendix 5) to allow for greater flexibility in
adapting the questions if deemed appropriate (Robson, 2002). A disadvantage is that
interviews are time-consuming, however only four interviews are planned to be held. A
further limitation is that teachers might just say what they think the author wants to hear
which could be a threat to validity (Cohen et al., 2007). It is hoped that by informing teachers
of the focus of the observations and interviews in advance, participants will recognise the
study aims to highlight good practice and investigate how we can improve rather than to
report on individual teachers.
To follow the Ethical Guidelines for Educational Research (BERA, 2011), the teachers and
students involved will be informed of the purpose of the project, how the observations,
questionnaires and interviews will be used and the fact that any information given will remain
strictly confidential. Participant teachers will complete consent forms to confirm their
understanding of the research aims and their right to withdraw from participating at any time.
To maintain confidentiality and anonymity, the participants will be referred to by Teacher X
etc. and names will not be requested on the student questionnaires (ibid).
Analysis Rationale
The results will be analysed using both SPSS and Excel with particular focus on proportions
of question types and techniques employed and correlation between the variables. To
reduce bias, comparisons will only be made between the hypotheses’ variables and not
between ability, gender and age groups of classes as any observed differences could be
attributable to differences in characteristics of teachers as opposed to the differences in
learners. This random assignment should maximise the internal validity of the study, while
test-retest and analysis of response rate for the questionnaire and the inter-coding testing of
the observations should check the reliability. The aim of the analysis is to verify the theory
through deductive reasoning and make recommendations from the findings.
References
Anderson, L. W., & Krathwohl, D. R. (Eds.). Airasian, P. W., Cruikshank, K. A., Mayer, R. E.,
Pintrich, P. R., Raths, J. and Wittrock, M. C. (2001). A taxonomy for learning, teaching and
assessing: A revision of Bloom's Taxonomy of Educational Objectives. New York: Allyn
Bacon Longman.
Anderson, L. W., & Sosniak, L. A. (Eds.). (1994). Bloom's taxonomy: A forty-year perspective.
Ninety-third Yearbook of the National Society for the Study of Education. Chicago: University
of Chicago Press.
Andrews, P., Hatch, G. and Sayers, J. (2005), ‘What do teachers of mathematics teach? An
initial episodic analysis of four European traditions’, in D. Hewitt and A. Noyes (Eds),
Proceedings of the sixth British Congress of Mathematics Education held at the University of
Warwick, pp. 9-16
Bashir, M., Afzal, M. T., & Azeem, M. (2008) ‘Reliability and validity of qualitative and
operational research paradigm’, Pakistan Journal of Statistics and Operation Research, 4(1).
Available at http://www.pjsor.com/index.php/pjsor/article/viewArticle/59 [Accessed 11 June
2013]
BERA (2011) Ethical Guidelines for Educational Research, British Educational Research
Association, London, p.1-11. Available at: http://www.bera.ac.uk/publications/guides.php
[Accessed 16 April 2013]
Bereiter, C. & Scardamalia, M. (1999). Beyond Bloom's Taxonomy: Rethinking Knowledge for
the Knowledge Age. In Hargreaves, A., Libermann, A., Fullan, M., & Hopkins, D. (Eds.), The
International Handbook of Educational Change (pp. 675–692). Dordrecht, The Netherlands:
Kluwer Academic Publishers.
Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom
assessment. nferNelson, London.
Black, P. (2002). Working inside the black box: Assessment for learning in the classroom.
nferNelson, London.
Black, P., Harrison, C., Lee, C., Marshall, B., & William, D. (2003). Assessment for learning-
putting it into practice. Open University Press.
Black, P., McCormick, R., James, M. & Pedder, D. (2006) ‘Learning How to Learn and
Assessment for Learning: A Theoretical Inquiry’, Research Papers in Education, 21:02,
pp.119-132.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy
of educational objectives: Handbook I: Cognitive domain. New York: David McKay.
Bloom, B. S. (1994) ‘Reflections on the development and use of the taxonomy.’ Yearbook:
National Society for the Study of Education, 92(2), pp.1-8.
Brown, G. A., & Edmondson, R. (2001) ‘Asking Questions’ in Wragg, E.C. (Ed.) Classroom
teaching skills, pp.97-120, Routledge, London.
Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education. Routledge.
Denscombe, M. (2007) The Good Research Guide for small-scale social research projects,
Third Edition, First published 1998, Maidenhead, Oxford University Press.
DfES (2004) ‘Unit 7: Questioning’, Pedagogy and Practice: Teaching and Learning in
Secondary Schools, Key Stage 3 National Strategy, Crown copyright
Available at:
http://webarchive.nationalarchives.gov.uk/20110809101133/nsonline.org.uk/node/97131
[Accessed 26 May 2013]
Dubinsky, E. and McDonald, M.A. (2002) ‘APOS: A Constructivist Theory of Learning in
Undergraduate Mathematics Education Research’ in Hoton, D. The Teaching and Learning of
Mathematics at University Level, New ICMI Study Series, 2002, Vol. 7, Section 3, pp.275-
282.
Gall, M. D. (1970) ‘The use of questions in teaching’, Review of Educational Research, 40(5),
pp. 707-721.
Gray, E. and Tall, D. (1991) ‘Duality, Ambiguity and Flexibility in Successful Mathematical
Thinking’, Proceedings of PME 15, Assisi, Vol. 2 pp. 72–79.
Hitchcock, G., & Hughes, D. (1995) Research and the teacher: A qualitative introduction to
school-based research. London, Routledge.
James, M., Black, P., Carmichael, P., Drummond, M-J., Fox, A., MacBeath, J., Marshall, B.,
McCormick, R., Pedder, D., Procter, R., Swaffield, S., Swann, J., and Wiliam, D. (2007)
Improving Learning How to Learn in classrooms, schools and networks. London,
Routledge.
Mason, J. (2000) ‘Asking mathematical questions mathematically’, International Journal of
Mathematical Education in Science and Technology, 31(1), pp.97-111.
Miller, J. & Glassner, B. (2004) ‘The “inside” and the “outside”. Finding realities in interviews’
in Silverman, D (Ed) Quantitative Research, London, SAGE Publications Limited.
Morgan, N., & Saxton, J. (2006) Asking better questions. Markham, Ontario, Pembroke
Publishers.
Muijs, D., & Reynolds, D. (2011) Effective teaching: Evidence and practice, London, SAGE
Publications Limited.
ODU (2013) Richard C. Overbaugh & Lynn Schultz, Old Dominion University Webpage
http://www.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm
[Accessed 26 May 2013]
Ofsted (2008) Assessment for learning: the impact of National Strategy support, Crown
Copyright, Manchester. Available at: http://www.ofsted.gov.uk/resources/assessment-for-
learning-impact-of-national-strategy-support [Accessed 27 May 2013]
Ofsted (2012) Mathematics: made to measure, Crown Copyright, Manchester. Available at:
www.ofsted.gov.uk/resources/110159 [Accessed 25 May 2013]
Overbaugh, R.C. & Schultz, L. Old Dominion University?????
Pegg, J. & Tall, D. (2010) ‘The Fundamental Cycle of Concept Construction Underlying
Various Theoretical Frameworks’, in Sriraman, B. and English, L. (Eds.) Theories of
Mathematics Education, Berlin Heidelberg, Springer-Verlag, pp. 173-192.
Robson, C. (1993) Real world research: A resource for social scientists and practitioner
researchers. Oxford: Blackwell.
Rowe, M. B. (1986) ‘Wait time: slowing down may be a way of speeding up!’, Journal of
Teacher Education, 37(1), pp.43-50.
Sfard, A. (1991) ‘On the Dual Nature of Mathematical Conceptions: Reflections on Processes
and Objects as Different Sides of the Same Coin’, Educational Studies in Mathematics, Vol.
22, No. 1, pp. 1-36
Silverman, D. (2000) Doing qualitative research: A practical handbook, London, Sage.
Smith, G., Wood L., Coupland, M., Stephenson, B., Crawford, K. & Ball, G. (1996)
‘Constructing mathematical examinations to assess a range of knowledge and skills’,
International Journal of Mathematical Education in Science and Technology, 27:1, pp.65-77
Stake, R. (2005) ‘Qualitative case studies’, in: N. Denzin and Y. Lincoln (eds) Handbook of
Qualitative Research (third edition). London: Sage.
Watson, A. & Mason, J. 1998, Questions and Prompts for Mathematical Thinking, ATM,
Derby.
Watson, A. (2003) ‘Opportunities to learn mathematics’, Mathematics education research:
Innovation, networking, opportunity, pp. 29-38.
Available at: http://www.merga.net.au/documents/Keynote_Watson.pdf
[Accessed 27 May 2013]
Watson, A. (2007) ‘The nature of participation afforded by tasks, questions and prompts in
mathematics classrooms’, Research in Mathematics Education, 9:1, pp.111-126.
Wiliam, D. (2006). Mathematics inside the black box: Assessment for learning in the
mathematics classroom. Granada Learning.
Wong, K. Y. (2012) ‘Use of student mathematics questioning to promote active learning and
metacognition’, 12th International Congress on Mathematical Education 8 – 15 July 2012,
COEX, Seoul, Korea
Available at: http://repository.nie.edu.sg/jspui/bitstream/10497/6194/1/ICME-2012-1086_a.pdf
[Accessed 9 June 2013]
Appendix 1
Question Type (Intended Mathematical Thinking) Coding Table
QUESTION TYPE Adapted from
Smith et al. (1996) and Andrews et al.
(2005)
PROMPTS Adapted from Watson’s
analytical instrument(2007, p.119)
FORMATIVE QUESTION STEMS From Wiliam (2006)
SURFACE APPROACH QUESTION
CODING
DEEPER APPROACH QUESTION
CODING
Factual
Name
Recall facts
Give definitions
Define terms
FS FD
Procedural
Imitate method
Copy object
Follow routine procedure
Find answer using procedure
Give answer
PS PD
Structural
Show me…
Analyse
Compare
Classify
Conjecture
Generalise
Identify variables
Explore variation
Look for patterns
Identify relationships
Tell me about the problem. What do you know about the problem? Can you describe the problem to someone else? What is similar . . . ? What is different . . . ? Do you have a hunch? . . . a conjecture? What would happen if . . . ? Is it always true that . . . ? Have you found all the solutions?
SS SD
Reasoning
Justify
Interpret
Visualise
Explain
Exemplify
Informal induction
Informal deduction
Can you explain/ improve/add to that explanation? How do you know that . . . ? Can you justify . . .?
RS RD
Reflective
Summarise
Express in own words
Evaluate
Consider advantages/ disadvantages
What was easy/difficult about this problem . . . this mathematics? What have you found out? What advice would you give to someone else about . . . ?
VS VD
Derivational
Prove
Create
Design
Associate ideas
Apply prior knowledge (in new situations)
Adapt procedures
Find answer without known procedure
Have you seen a problem like this before? What mathematics do you think you will use? Can you find a different method? Can you prove that . . . ?
DS DD
Appendix 2
Questioning Technique Coding Table
QUESTIONING TECHNIQUE CODE
Use random methods to choose a student to answer (e.g. names from hat) R
Hands up H
No hands up with ‘wait time’ N
Discuss answer for a set time in pairs/groups first G
Use mini-whiteboards to write answers W
Generate discussion from mini-whiteboards W+D
Choose from a few answers (e.g. Using voting fans) V
Ask if a student agrees with another A
Identify the error M
Writing up selection of responses on board then discuss S
Odd one out O
Always/Sometimes/Never True (or equivalent) T
Problems with more (or less) than one correct solution P