shawna williams bc teal annual conference may 24, 2014
TRANSCRIPT
Demystifying Second Language Assessment
Shawna WilliamsBC TEAL Annual Conference
May 24, 2014
Agenda
IntroductionAssessment terminology – definitions
Principles of Assessment
My assessment history…
Language Assessment: Principles and Classroom Practices
Assessment Terminology
Assessment Terminology
Assessment ≠ Testing
Assessment is…“appraising or estimating the level or
magnitude of some attribute of a person.”
(Mousavi, 2009)
“an ongoing process of collecting information about a given object of interest according to procedures that are systematic and substantively grounded.”
(Brown & Abeywickrama, 2010)
Assessment“A good teacher never ceases to
assess students, whether those assessments are incidental or intended.”
(Brown & Abeywickrama, 2010)
Assessment and Learning
Teaching
Assessment
Measurement
Tests
Evaluation
(Brown & Abeywickrama, 2010, p. 6)
Function of AssessmentInformal FormalIncidental,
unplanned comments
CoachingImpromptu
feedback on homeworkNonjudgmental
Systematic and planned
Give T and Ss appraisal of achievement
Tests and assignments
Function of AssessmentFormative Summative“evaluating students
in the process of ‘forming’ their competencies and skills with the goal of helping them to continue that growth process”
feedback on performance
future continuation of learning
“aims to measure, or summarize, what the student has grasped”
end of course or unitlooking back and
taking stock
(Brown & Abeywickrama, 2010, p.7)
Five Principles of AssessmentHow do you know if an assessment task is effective, appropriate, useful, or . . . “good”?
Practicality
Reliability
Validity
Authenticity
Washback
A PRACTICAL TEST . . .
A PRACTICAL TEST . . . budgetary limitsappropriate time constraintsclear directions for administrationappropriately utilizes human resourcesdoes not exceed available material resources
considers time and effort for design and scoring
(Brown & Abeywickrama, 2010, p. 26)
A RELIABLE TEST . . .
=
A RELIABLE TEST . . .consistent across two or more
administrationsclear directions for scoring/evaluationuniform rubrics for scoring/evaluationconsistent application of rubrics by
scorerunambiguous to the test-taker
(Brown & Abeywickrama, 2010, p. 27)
Reliability con’tRater Reliability
Inter-Rater Reliability
Intra-Rater Reliability
Reliability con’tRater Reliability
Inter-Rater ReliabilityIntra-Rater Reliability
Student-Related ReliabilityTest Administration ReliabilityTest Reliability
A VALID TEST . . .
A VALID TEST . . . measures exactly what it proposes to measuredoes not measure irrelevant or
“contaminating” variablesrelies on empirical evidence (performance)performance that samples the test’s criterion
(objective)useful, meaningful information about test-
taker’s abilitysupported by theoretical rationale or
argument(Brown& Abeywickrama, 2010, p. 30)
Validity con’tContent-Related EvidenceCriterion-Related EvidenceConstruct-Related EvidenceConsequential Validity (Impact)Face Validity
AN AUTHENTIC TEST . . .
AN AUTHENTIC TEST . . .language as natural as possibleitems are contextualized rather than
isolatedmeaningful, relevant, interesting
topicsthematic organizationreal-world tasks
(Brown & Abeywickrama, 2010, p. 37)
A TEST THAT PROVIDES BENEFICIAL WASHBACK . . .
A TEST THAT PROVIDES BENEFICIAL WASHBACK . . .positively influences teachers’
teachingpositively influences learners’
learninglearners can adequately preparefeedback for language developmentmore formative than summativeconditions for peak performance
(Brown & Abeywickrama, 2010, p. 38)
Applying Principles to Creation of Assessment Tools1. Practical test procedures?2. Test is reliable?3. Rater reliability?4. Content validity?5. Impact has been accounted for?6. Procedure is “biased for best”?7. Test tasks are authentic?8. Test offers beneficial washback?
9. See Brown & Abeywickrama, Chpt. 2
