1 issues in assessment in higher education: science higher education forum on scientific...
TRANSCRIPT
11
Issues in Assessment in Issues in Assessment in Higher Education: ScienceHigher Education: Science
Higher Education Forum on Scientific CompetenciesHigher Education Forum on Scientific CompetenciesMedellin-Colombia Nov 2-4, 2005Medellin-Colombia Nov 2-4, 2005
Dr Hans WagemakerINTERNATIONAL ASSOCIATION FOR THE EVALUATION OF EDUCATIONAL ACHIEVEMENT
Theme: Validity in AssessmentTheme: Validity in Assessment
What can/should be Assessed.What can/should be Assessed. How can it be AssessedHow can it be Assessed Frameworks, Curriculum and Frameworks, Curriculum and
TrajectoriesTrajectories From TIMSS to Higher EducationFrom TIMSS to Higher Education
Purpose of Assessment: Purpose of Assessment: Assessment for Assessment for
Learning/Assessment of LearningLearning/Assessment of Learning
Validity/Reliability- Primacy of Validity/Reliability- Primacy of ValidityValidity
Construct ValidityConstruct Validity Consequential ValidityConsequential Validity
TIMSS what is Assessed:TIMSS what is Assessed:Content DomainsContent Domains
Grade 4 – Life, Physical and Earth Grade 4 – Life, Physical and Earth ScienceScience
Grade 8- Biology, Chemistry, Physics, Grade 8- Biology, Chemistry, Physics, Earth ScienceEarth Science
TIMSS DesignTIMSS Design
What is Assessed: Cognitive What is Assessed: Cognitive Assessment DomainsAssessment Domains
KnowingKnowing ApplyingApplying ReasoningReasoning Scientific InquiryScientific Inquiry
Implies different strategiesImplies different strategies
KnowingKnowing 1. Recall/Recognize- statements, facts, 1. Recall/Recognize- statements, facts,
relationships, processes, concepts.relationships, processes, concepts.
2. Define- provide or identify definitions of 2. Define- provide or identify definitions of scientific terms, vocabulary, symbols, scientific terms, vocabulary, symbols, abbreviations, units, scales in appropriate abbreviations, units, scales in appropriate contexts.contexts.
3. Describe- organisms, physical materials, 3. Describe- organisms, physical materials, scientific processes that demonstrate scientific processes that demonstrate knowledge of properties, structure and knowledge of properties, structure and relationships.relationships.
KnowingKnowing
4. Illustrate with Examples – support 4. Illustrate with Examples – support or clarify statements of facts or or clarify statements of facts or concepts with appropriate examples- concepts with appropriate examples- provide specific examplesprovide specific examples
5. Use tools and procedures – 5. Use tools and procedures – demonstrate knowledge of the use of demonstrate knowledge of the use of science apparatus, equipment , tools, science apparatus, equipment , tools, procedures, measurement devices.procedures, measurement devices.
ApplyingApplying
1. Compare contrast classify- 1. Compare contrast classify- identify or describe identify or describe similarities/differences between similarities/differences between groups of organisms, materials, groups of organisms, materials, processes –distinguish, classify or processes –distinguish, classify or order objects, materials or organisms order objects, materials or organisms and processes based on given and processes based on given characteristics and properties.characteristics and properties.
ApplyingApplying
2. Use Models – use a diagram or 2. Use Models – use a diagram or model to demonstrate science model to demonstrate science concept, structure, relationship, concept, structure, relationship, process, biological or physical process, biological or physical system or cycle ( e.g.. Atomic system or cycle ( e.g.. Atomic structure, electrical circuit, water structure, electrical circuit, water cycle)cycle)
ApplyingApplying
3. Relate – relate knowledge of an 3. Relate – relate knowledge of an underlying biological or physical underlying biological or physical concept to an observed or inferred concept to an observed or inferred property, behavior, or use of objects, property, behavior, or use of objects, organisms or materialsorganisms or materials
4. Interpret information - interpret 4. Interpret information - interpret textual, tabular or graphical textual, tabular or graphical information in light of a scientific information in light of a scientific concept or principleconcept or principle
ApplyingApplying
5. Find solutions – use a science 5. Find solutions – use a science relationship, equation, or formula to find a relationship, equation, or formula to find a qualitative or quantitative solution qualitative or quantitative solution involving the direct involving the direct application/demonstration of a conceptapplication/demonstration of a concept
6. provide and explanation for an 6. provide and explanation for an observation demonstrating understanding observation demonstrating understanding of the underlying science concept, of the underlying science concept, principle, law or theory.principle, law or theory.
ReasoningReasoning
1. Analyze problems to determine 1. Analyze problems to determine the relevant relationships, concepts the relevant relationships, concepts and problem solving steps – develop and problem solving steps – develop problem solving strategiesproblem solving strategies
2. Integrate/Synthesize – provide 2. Integrate/Synthesize – provide solutions to problems that require solutions to problems that require consideration of a number of consideration of a number of different factors or related concepts – different factors or related concepts – integrate mathematical conceptsintegrate mathematical concepts
Reasoning Reasoning
3. Hypothesize/predict – combine 3. Hypothesize/predict – combine science knowledge with information science knowledge with information from observation/experience or from observation/experience or formulate questions that can be formulate questions that can be answered by investigation – answered by investigation – formulate testable hypotheses – formulate testable hypotheses – make predictions about effects of make predictions about effects of changes in biological or physical changes in biological or physical conditions in light of evidence and conditions in light of evidence and scientific understandingscientific understanding
ReasoningReasoning 4. Design Plan investigations appropriate 4. Design Plan investigations appropriate
for answering questionsfor answering questions 5. Draw conclusions – detect patterns in 5. Draw conclusions – detect patterns in
data trends – interpolatedata trends – interpolate 6. Generalize – make general conclusions 6. Generalize – make general conclusions
that go beyond experimental or given that go beyond experimental or given conditionsconditions
7. Evaluate – weigh 7. Evaluate – weigh advantages/disadvantages about advantages/disadvantages about alternative processes, materials, alternative processes, materials, alternative explanations etc.alternative explanations etc.
ReasoningReasoning
Justify – use evidence and scientific Justify – use evidence and scientific understanding to justify explanations understanding to justify explanations and problem solutions…and problem solutions…
Scientific InquiryScientific Inquiry
1. Demonstrate a formalized 1. Demonstrate a formalized approach to scientific investigationapproach to scientific investigation
Core Principles of Effective Core Principles of Effective AssessmentAssessment
1. assessment that guides and 1. assessment that guides and encourages to effective approaches to encourages to effective approaches to learninglearning
2. assessment that validly and reliably 2. assessment that validly and reliably measures expected learning outcomes, in measures expected learning outcomes, in particular higher-order learning that particular higher-order learning that characterizes higher educationcharacterizes higher education
3. assessment and grading that defines 3. assessment and grading that defines and protects academic standardsand protects academic standards
Assessment for Learning- Assessment for Learning- FormativeFormative
Validity of Assessment for learning is Validity of Assessment for learning is high when assessment procedures high when assessment procedures help the student toward achieving help the student toward achieving the real goals of the learning activitythe real goals of the learning activity
Requires, committed participants, Requires, committed participants, trust, understanding of what is being trust, understanding of what is being asked, effective feedbackasked, effective feedback
Assessment for Learning – Assessment for Learning – Formative AssessmentFormative Assessment
Task Factors – Knowledge of key Task Factors – Knowledge of key aspects and difficulties of taskaspects and difficulties of task
Criteria – students told ( and Criteria – students told ( and understand) clearly what is needed.understand) clearly what is needed.
Standards – set standards are Standards – set standards are appropriate to studentsappropriate to students
Feedback – balance of attention – Feedback – balance of attention – both strengths and weaknesses- is both strengths and weaknesses- is useful for studentuseful for student
Assessment of Learning – Assessment of Learning – Summative AssessmentSummative Assessment
Done under conditions that give Done under conditions that give confidence it is students own work.confidence it is students own work.
Marking is standardized across Marking is standardized across several of many teachersseveral of many teachers
Most up to date information possibleMost up to date information possible
Assessment of learningAssessment of learningSummative AssessmentSummative Assessment
Quality ( and appropriateness) Quality ( and appropriateness) should take precedence over should take precedence over quantityquantity
Things that can be assessed in final Things that can be assessed in final exam should be - those that can not exam should be - those that can not should not.should not.
If multiple assessments of same If multiple assessments of same material – final grade should reflect material – final grade should reflect final level of performancefinal level of performance
Assessment of LearningAssessment of Learning- Summative Assessment- Summative Assessment
Disadvantage – time sampling Disadvantage – time sampling problemproblem
May not be valid for admitting May not be valid for admitting student to next stage of educationstudent to next stage of education
Indicators of affective assessmentsIndicators of affective assessments
1. assessment which is integral 1. assessment which is integral component of teaching and learning component of teaching and learning processprocess
2. multiple roles of assessment 2. multiple roles of assessment recognized.recognized.
3. departmental/faculty policy as 3. departmental/faculty policy as guideguide
4. alignment between what is taught 4. alignment between what is taught and what is assessed.and what is assessed.
Indicators..Indicators..
5. Assessment tasks assess capacity 5. Assessment tasks assess capacity to analyze and synthesize new to analyze and synthesize new information.information.
6. Variety of assessments used6. Variety of assessments used
7. Assessment designed to assess 7. Assessment designed to assess generic skills as well as subject generic skills as well as subject specific skillsspecific skills
IndicatorsIndicators
8. steady progression of complexity 8. steady progression of complexity and demands in later yearsand demands in later years
9. student choice (paper/exam/timing)9. student choice (paper/exam/timing)
10. workloads considered10. workloads considered
11. not excessive assessment11. not excessive assessment
12. task weighted to balance 12. task weighted to balance formative and summative rolesformative and summative roles
IndicatorsIndicators
13. Grades calculated on basis of 13. Grades calculated on basis of clearly articulated learning clearly articulated learning outcomes.outcomes.
14. Students provided with 14. Students provided with explanatory feedbackexplanatory feedback
15. Tasks checked for bias15. Tasks checked for bias
16. Plagiarism minimized 16. Plagiarism minimized
Assessment formatsAssessment formats
Multiple choiceMultiple choice Extended ResponseExtended Response Individual ProjectIndividual Project ThesisThesis Laboratory workLaboratory work Group workGroup work Computer basedComputer based PresentationsPresentations Many others: oral, portfolioMany others: oral, portfolio