the ecology of language assessment carl falsgraf, director casls
Post on 20-Dec-2015
222 views
TRANSCRIPT
The Ecology of Language
Assessment
Carl Falsgraf, Director CASLS
The Language Education EcosystemThe Language Education EcosystemThe Language Education EcosystemThe Language Education Ecosystem
CurriculumCurriculum
InstructionInstruction
Formative AssessmentFormative Assessment
Summative AssessmentSummative Assessment
REFLECTI
ON
REFLECTI
ON
RE
FLEC
TION
REFLECTION
Can one assessment do it all?
ASSESSMENT CONTINUUM
Teacher administered
Customized
Curriculum dependent
+ high validity
+ supports instruction
- low reliability
- context dependent
Expert administered
Standardized
Curriculum independent
+ high reliability
+ context independent
- low validity
- divorced from instruction
STAMP?
ProficiencyProficiency
PerformancePerformanceAchievementAchievement
OPI
Oral quiz
Service learning project
Weekly quiz
Final exam
STAMP?
Specifications of Intended Test Use (SITU)
(Norris, 2003)
•Who are the test takers?
•Who are the test score users?
•What is the purpose of the test?
•What information should the test provide?
•What are the intended consequences of using the test?
Norris (2000)
STAMP?
0
0+
1
1+
2
2+
3
3+
4
4+
5
ILR
Novice-High
Novice-Mid
Novice-Low
Intermediate-High
Intermediate-Mid
Intermediate-Low
Advanced Plus
Advanced
Superior
Benchmark 1
Benchmark 2
Benchmark 3
Benchmark 4
Benchmark 5
Benchmark 6
ACTFL
BENCHMARKS
Ass
essm
ent
STAMP 101• • By teachers, for teachers By teachers, for teachers
• Designed to inform, not just measureDesigned to inform, not just measure
• • Online: Anywhere, anytime, anybodyOnline: Anywhere, anytime, anybody
• • Computer adaptive reading/listeningComputer adaptive reading/listening
• • Computer assisted speaking/writingComputer assisted speaking/writing
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
STAMP LISTENING
STAMP READING
STAMP READING
NOELLA SPEAKING What is happening in the picture? Tell as much as you can.
G4-IV-2-ED-1 SubQ 2
0
20
40
60
80
100
1 2 3 4 5
Group Score Rank
Percent chosen
ABCD
Example of a good item
G4-IV-2-SG-7
0
20
40
60
80
100
1 2 3 4 5
Group Score Rank
Percent chosen
A
B
C
D
Example of a bad item
Level 3
Level 4
CLASS REPORT
PROGRAM REPORT
SLA Web
National 3rd/4th YR Japanese Data
Proficiency Levels by Years (Spanish - High School - 7500 students)
0% 10% 20% 30% 40% 50% 60%
Novice-Low
Novice-Mid
Novice-High
Intermediate-Low
Intermediate-Mid
0% 10% 20% 30% 40% 50% 60%
Novice-Low
Novice-Mid
Novice-High
Intermediate-Low
Intermediate-Mid
0% 5% 10% 15% 20% 25% 30% 35% 40%
Novice-Low
Novice-Mid
Novice-High
Intermediate-Low
Intermediate-Mid
0% 5% 10% 15% 20% 25% 30% 35%
Novice-Low
Novice-Mid
Novice-High
Intermediate-Low
Intermediate-Mid
1 year of study 2 years of study
3 years of study 4 years of study
NEEDS
• AUDIT
• CERTIFICATION
• PROGRAM EVALUATION
• PROGRESS CHECK
• DIAGNOSIS
• IMPROVE INSTRUCTION
• MOTIVATE STUDENTS
NCLB/STAMP
AP/OPI
STAMP
MIDTERM
LINGUAFOLIO
LINGUAFOLIO
LINGUAFOLIO
ASSESSMENTS
DEVELOPING AN ASSESSMENT PLAN
• CLARIFY GOALS
• DEFINE PURPOSES/USES
• RESEARCH EXISTING TOOLS
• DEFINE INTERNAL TOOLS
• INITIATE ASSESSMENT LITERACY PROFESSIONAL DEVELOPMENT
SAMPLE ASSESSMENT PLAN