12 th grade naep preparedness research plans and progress
Post on 21-Jan-2016
30 Views
Preview:
DESCRIPTION
TRANSCRIPT
Cornelia Orr, Executive DirectorNational Assessment Governing Board
12th Grade NAEP Preparedness Research Plans and Progress
The Board’s Work on The Board’s Work on Preparedness Has Been Preparedness Has Been
Underway for Nearly a DecadeUnderway for Nearly a Decade
2002
12th Grade Commission Established
12th Grade Commission Established
2005
Board Resolution on Reporting12th Grade PreparednessBoard Resolution on Reporting12th Grade Preparedness
2006
Grade 12 Reading& Mathematics Frameworks Revised
Grade 12 Reading& Mathematics Frameworks Revised
2009
Board Approves Preparedness Research PlanBoard Approves Preparedness Research Plan
Conduct Research Studies and Develop Preparedness ReportConduct Research Studies and Develop Preparedness Report
2010-2012
The Governing Board has demonstrated a rigorous, comprehensive, and methodical approach
to research and has not presumed the equivalence of college and career preparedness. This
research will help inform the definition of preparedness in terms of NAEP.
2July 26, 2011 Cornelia Orr, NAGB
Key Considerations for the Governing Board
Statements in NAEP reports must be shaped and supported by validity evidence (validity framework)
Recognize NAEP has limitations
Reporting on NAEP 12th grade preparedness may or may not prove to be feasible
Consider additional Grade 12 academic areas in the future
3July 26, 2011 Cornelia Orr, NAGB
Purpose: Technical Panel on 12th Grade Preparedness Research
Identify technically appropriate and valid ways to use grade 12 NAEP for reporting preparedness
Assist in planning research and validity studies
Recommend research designs and priorities for implementation in 2008, 2009, and thereafter
Focus on 12th grade reading and mathematics
4July 26, 2011 Cornelia Orr, NAGB
Technical Panel on 12th Grade Preparedness Research: Members
Michael Kirst (Chair)Stanford University
John CampbellUniversity of Minnesota
David T. ConleyUniversity of Oregon
Michael KaneNational Conference of
Bar Examiners
Mark David MillironCatalyze Learning International
Robert MislevyUniversity of Maryland
George C. Thornton, IIIColorado State University
5July 26, 2011 Cornelia Orr, NAGB
Panel’s Key Conclusions
Reporting on 12th grade preparedness seems feasible
Use multipronged approach to collect validity information
Avoid representing 12th grade NAEP as the single authoritative definition or conception of preparedness
Consider the national context related to preparedness
6July 26, 2011 Cornelia Orr, NAGB
Preparedness: Working Definitions
Preparedness for College: reading and mathematics knowledge and skills to qualify for placement into entry level college credit courses that meet general education requirements, without the need for remedial coursework in math or reading
Preparedness for Workplace Training: reading and mathematics knowledge and skills needed to qualify for a job training program without remediation in math or reading
“Preparedness” versus “Readiness”
7July 26, 2011 Cornelia Orr, NAGB
Types of Research Studies
Content alignment
Judgmental standard-setting
Statistical relationship
Survey of Postsecondary Institutions
Benchmarking Studies
8July 26, 2011 Cornelia Orr, NAGB
500
391
367
336
327
313
297
286
277
256
243
0
346
302
265
Statistical LinkingStudies
JudgmentalStudies
(Expert Panels)
Study Interrelationships
Higher EducationSurvey
Content AlignmentStudies
9July 26, 2011 Cornelia Orr, NAGB
Data for Preparedness Research
2009 NAEP 12th grade Reading and MathematicsNational and state level data 11 states voluntarily participated
Linkages to state data systems (only 1 in 2009)Advance planningConfidentiality assurance processesFollow-up for post secondary college/career status
Planning for 2013
10July 26, 2011 Cornelia Orr, NAGB
Grade 12 NAEPResults for 11 states
11July 26, 2011 Cornelia Orr, NAGB
Research Studies
Preparedness Study Status
Content Comparisons (SAT, ACT, Accuplacer, Work Keys)
Complete
Judgmental Standard Setting Underway
Statistical Relationships /Linking(SAT, Accuplacer, ACT)
Underway
Survey of Higher Ed Institutions Underway
Benchmarking Planned
12July 26, 2011 Cornelia Orr, NAGB
500
391
367
336
327
313
297
286
277
256
243
0
346
302
265
Reference Points on the NAEP Scale (Scenario 1)
College Preparedness Standards: A
College Preparedness Standards: B
Exemplar Job 1
Exemplar Job 2
Exemplar Job 3
Exemplar Job 4
Exemplar Job 5Placement in entry-level credit-bearing course
Advanced
Proficient
Basic
Below Basic
July 26, 2011 13Cornelia Orr, NAGB
500
391
367
336
327
313
297
286
277
256
243
0
346
302
265
Reference Points on the NAEP Scale (Scenario 2)
Exemplar Job 1
Exemplar Job 2
Exemplar Job 3
Exemplar Job 4
Exemplar Job 5
College Preparedness Standards: A
College Preparedness Standards: B
Placement in entry-level credit-bearing course
July 26, 2011 14Cornelia Orr, NAGB
Reference Points on the NAEP Scale (Scenario 3)
500
391
367
336
327
313
297
286
277
256
243
0
346
302
265
College Preparedness Standards: B
College Preparedness Standards: A
Placement in entry-level credit-bearing course
Exemplar Job 4
Exemplar Job 1
Exemplar Job 5
Exemplar Job 3
Exemplar Job 2
July 26, 2011 15Cornelia Orr, NAGB
Next Steps
Develop summary report of validity evidenceValidity Framework – external reviews
Develop preparedness reportsBased on national data analyses
Research-focused report
Public-focused report
Issue Preparedness Report in 201216July 26, 2011 Cornelia Orr, NAGB
Questions and Discussion
What are the results from the preparedness research so far?
The content of NAEP is similar to the content assessed on common indicators of college preparedness, that is, the ACT, SAT, and ACCUPLACER.
Some content on NAEP is similar to WorkKeys, though the content on NAEP is generally broader than the content on WorkKeys and WorkKeys measures some content that is not in NAEP.
The linking of NAEP to SAT: NAEP proficient is about at 500 on the SAT for both reading and mathematics, the college board readiness benchmark.
How can states use the NAEP preparedness research?
As a resource for state and/or consortia standard setting
To provide reference points for state readiness/preparedness definitions
17July 26, 2011 Cornelia Orr, NAGB
Other Questions?
Dr. Cornelia S. Orr
Executive Director
National Assessment governing Board
800 North Capitol Street NW, Suite 825
Washington, DC 20002-4233
www.nagb.org
18July 26, 2011 Cornelia Orr, NAGB
top related