nagc 2013 measurement and modeling of higher order thinking skills
DESCRIPTION
This session describes how a selective, public STEM high school has used the College and Work Readiness Assessment (CWRA) in efforts to model student performance and demonstrate achievement gains in critical thinking, analytical reasoning, problem solving, and written communication. In the context of both diagnostic utility and repeated measures student growth, we will describe results from over 400 students and explain how this new measure of high order thinking skills provides reliable results for advanced students. We will also explore the institutional value of having normed data for value-added measures.TRANSCRIPT
MEASUREMENT AND MODELING OF HIGHER ORDER SKILLS IN STEM TALENT DEVELOPMENT USING THE CWRAChristopher Kolar
NCSSSMST/NAGC Annual Convention 2013, Indianapolis
The challenge
Exceptional students
Exceptional claims of exceptional results
Exceptional Evidence
Standards of Significant Learning (SSLs)
Assessment at IMSA
Competence In Inquiry
Making accurate observations.
Evaluating evidence.
Challenging assumptions.
Making connections. Making sound
judgments. Communicating
powerfully.
CWRA: Performance task and document
In addition to an overall Performance Task score, subscores include:
Analytic Reasoning
Writing Effectiveness
Writing Mechanics
Problem Solving
MEASURING UP
Admissions data
Admissions Benchmarking
Prior GPA SAT-M SAT-CR SAT-We SAT-Wmc
Admissions Benchmarking
CWRA performance task score significantly correlated with all admissions metrics.
Prior GPA SAT-M SAT-CR SAT-We SAT-Wmc
MEASURING UP
Initial academic performance
First year grades in writing intensive courses: Literary Explorations, American Studies, and Scientific Inquiries (Biology)
Prior GPA SAT-CR SAT-We SAT-Wmc CWRA+
First Year Grade Point Average
First year grades in writing intensive courses: Literary Explorations, American Studies, and Scientific Inquiries (Biology): 23% of variance explained.
Prior GPA SAT-CR SAT-We SAT-Wmc CWRA+
First Year Grade Point Average
MEASURING UP
English core assessment
No reliability as outcome measures
Thesis Paragraphs Quote Usage
Analytic Reasoning
Writing Effectiveness
Writing Mechanics Problem Solving
IMSA CWRA+
THEREFORE
What this means in practice
Student Support
CWRA+ individual results as a diagnostic Further grade and retention modeling using this
data
Assessment Development
Performance task academy School improvement plan Revision of core assessment
Value added
“Small” effect size for time by junior year. Senior exit testing next.