why assessment matters? · 1. what counts matters but not all that matters can be counted 2. the...
TRANSCRIPT
![Page 1: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/1.jpg)
Why Assessment Matters?
Jonathan Osborne
School of Education
Stanford University
Feb 11, 2011
![Page 2: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/2.jpg)
Main Points
1. What counts matters but not all that matters can be counted
2. The intentions of the curriculum are read from the assessment items not the curriculum documents
3. Assessments operationalize constructs
4. Knowing what’s wrong matters as much as knowing what is right.
![Page 3: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/3.jpg)
3
What can not be measured?
Persistence
Curiosity
A critical disposition
A commitment to evidence as the basis of belief
Awe and Wonder
Imagination
![Page 4: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/4.jpg)
4
Assessments Operationalize Constructs
Intentions are read from the test items
Items become the embodiment of what matters
Bloom’s Taxonomy of Cognitive Skills Recall
Comprehension
Application
Analysis
Synthesis
Evaluation
![Page 5: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/5.jpg)
Cognitive Demand: National Tests at Age 14
![Page 6: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/6.jpg)
Cognitive Demand: National Tests at Age 16
![Page 7: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/7.jpg)
7
Effect of High Stakes Assessment
Narrowing of the Curriculum & Teaching to the Test
Increasing Fragmentation
More Teacher Centred Pedagogy
Au, W. (2007). High Stakes Testing and Curricular Control: A Qualitative
Metasynthesis. Educational Researcher, 36(5), 258-267.
![Page 8: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/8.jpg)
8
![Page 9: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/9.jpg)
9
PISA
![Page 10: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/10.jpg)
10
TIMSS Framework 2011
Knowing (35%) Applying (35%) Reasoning (30%)
• Recall
• Define
• Describe
• Illustrate with
Examples
• Demonstrate
Knowledge of
Scientific Instruments
•Compare, Contrast,
Classify
•Use Models
•Relate
•Interpret Information
•Find Solutions
•Explain
•Analyze
•Integrate/Synthesize
•Hypothesize/Predict
•Design
•Draw Conclusions
•Generalize
•Evaluate
•Justify
![Page 11: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/11.jpg)
11
Content Procedural
Epistemic
Three Forms of Knowledge in Science
![Page 12: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/12.jpg)
12
Procedural Procedural
KnowledgeKnowledge
Concepts of Evidence
Error
Measurement
Instrumentation
Reliability/Validity
Variables/Fair Testing
EpistemicEpistemic
KnowledgeKnowledge
Ideas-About-Science
The role of peer review
Observation
Theory
Model
Hypothesis
![Page 13: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/13.jpg)
13
![Page 14: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/14.jpg)
14
The Absence of Critique?
Construction Critiquev
![Page 15: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/15.jpg)
15
A Sample Question
Janet was asked to do an experiment to find how long it takes for some sugar to dissolve in water. What advice would you give Janet to tell her how many repeated measurements to take?
A. Two or three measurements are always enough
B. She should take 5 measurements
C. If she is accurate she only needs to measure once
D. She should go on taking measurements until she knows how much
they vary
E. She should go on taking measurements until she gets two or more
the same
![Page 16: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/16.jpg)
![Page 17: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/17.jpg)
17
Some Concluding Thoughts
1. Assessments need to be more flexible
2. The variance in assessments should be large?
![Page 18: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/18.jpg)
18
![Page 19: Why Assessment Matters? · 1. What counts matters but not all that matters can be counted 2. The intentions of the curriculum are read from the assessment items not the curriculum](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f5c132ed1cc0d528802cc7f/html5/thumbnails/19.jpg)
19
Some Concluding Thoughts
1. Assessments need to be more flexible
2. The variance in assessments should be large?
3. In the trade off between reliability and validity, validity is what matters.