caveon webinar series - weathering the perfect test security storm may 2015
TRANSCRIPT
Jennifer Miller, Caveon Test SecurityDennis Maynes, Caveon Test Security
Michelle Croft, ACT
Weathering the Perfect Test Security Storm in Educational
Assessments
February 25, 2015
Caveon Webinar Series presents:
Agenda for Today
• Elements of the Perfect
Storm (Jennifer Miller)
• Fallout from the storm
(Michelle Croft)
• Weathering the storm
(Dennis Maynes)
Introduction
The Perfect Storm
• Test scores are tied to school funding.• Performance evaluations are tied to test
scores.• The responsibility of test security is placed in
the hands of those who are motivated to cheat.
• Security threats and vulnerabilities are increasing.
NCLB (2001) “Adequate yearly progress”
Federal funding goes to struggling schools, but…..
…with funding come sanctions
NCLB & Funding
• Each state sets timeline and definition for AYP.• Schools held accountable for performance of all
students, not just the average performance.• Schools that do not meet AYP for two years in a
row are subject to immediate interventions.– Technical assistance– Severe corrective actions
Adequate Yearly Progress
ESEA Flexibility Waiver
• Announced in 2011 to allow flexibility from some specific NCLB requirements.
• State-developed plans where turnaround interventions are implemented at “priority” schools.
• Requires student growth data as part of educator evaluations.
Parks Middle School in Atlanta “school in need of improvement” 2001-2005
In 2006, “Unless fifty-eight per cent of students passed the math portion of the test and sixty-seven per cent passed in language arts, the state could shut down the school.”
State Assessments and School Funding
The New Yorker, 21 July 2014
Governor’s investigation in Georgia found cheating in 44 APS schools, with almost 200 educators potentially involved.
“Investigators found ‘organized and systemic misconduct’ …and said administrators created a ‘culture of fear, intimidation and retaliation.’ The administrators used data as an ‘abusive and cruel weapon’ to coax employees into crossing ethical lines”
Los Angeles Times, 6 September 2014
State Assessments and School Funding
Performance Evaluations and Test Scores
States are moving to performance-based compensation and using test scores to evaluate
teachers
Administrators(Superintendents,
etc.)
Teachers
• 2012 El Paso Independent School District.
• Lorenzo Garcia, Superintendent.• Discouraged and/or prevented low-performing
students from taking the TAKS.• 381 students 170 students• Garcia received almost $60K in bonuses.
New York Times, 13 October 2012
Use of Test Scores for Personal Gain
Test Security in the Hands of Teachers
"If the IRS notified you that your tax return had irregularities, and they wanted you to recheck it yourself, everybody will write back and say, 'I checked my taxes, and they're fine.‘”
– Greg Cizek
Test Security in the Hands of Teachers
Teachers feeling the pressure:
Survey of Educators in Michigan
• 29% felt pressure to cheat on standardized tests.
• 34% felt pressure to help students.• 21% knew other educators who
helped students.• 8% admitted to changing answer
sheets.“Survey: Nearly 30% of Michigan Teachers Report Pressure to Cheat,” Detroit Free Press, July 27, 2011
Threats are Increasing
• Cheating technology is becoming more sophisticated.• Threats and CBT.• Threats associated with
consortia membership (PARCC and SBAC).• Threats with Social Media.
thegreenhead.com
• Harm to Students
• Tarnished
Reputations• Lost Credibility
• Negative Media
Attention
•$$$$$
Fallout from the Storm
Harm to Students“As a graduating senior from a tarnished school system, I have to worry every time I apply to a college…good grades isn’t considered a reward in itself anymore because those grades come from a school system of perceived cheaters…it’s no joke for students who have to overcome the perception of a morally bankrupt school system.”
Fallout from the Storm
Jenn Steckl APS School Districtyouthradio.org27 October 2014
Negative Media Attention• APS Scandal, Atlanta Journal
Constitution, 2008.• Cheating in 44 APS schools.• Prosecution of 12 educators under
RICO• Trial ongoing…
Fallout from the Storm
Pa Schools, The Notebook, 2009• Anomalies in 89 Schools - Cheating?
• As school starts, no action against Philly principals where cheating is suspected, 8/24/2012.
• Two District administrators surrender licenses in cheating scandal, 4/3/2013.
• District to release report on cheating investigation at 19 schools, 7/22/2013.
• 138 Philadelphia educators implicated in cheating probes, 2/5/2014.
• SRC to vote on terminating three principals related to cheating investigation, 1/16/2014.
• Two former principals charged in state cheating investigation, 9/25/2014.
• Former principal now eighth educator charged in cheating probe, 1/7/2015.
Fallout from the Storm
thenotebook.org
Fallout from the Storm
District of Columbia
• USA Today article (2011).• Entire year of items released.• Solicited bids for external
investigators to increase transparency.
• Budget
• Strategy
• Implement a Comprehensive Program
Weathering the Storm
• Strategy
• Create Mission and Vision Statements
• Assess Threats and Vulnerabilities
• Review and Allocate Resources
• Establish a Communications Plan
Weathering the Storm
• Implement a Comprehensive Program
• Protection
• Detection
• Response
• Improvement
Weathering the Storm
Weathering the Storm
Comprehensive Program
Protect
Detect
Respond
Improve
“Never before have so many had so much reason to cheat. Students’ scores are now used to determine whether teachers and principals are good or bad, whether teachers should get a bonus or be fired, whether a school is a success or failure…”New York Times, 31 July 2011
25
Audience Questions
Thank you!
- Follow Caveon on twitter @caveon- Check out our blog…www.caveon.com/blog- LinkedIn Group – “Caveon Test Security”
Jennifer Miller Data Forensics Coordinator, Caveon [email protected]
Michelle CroftPrincipal Research Associate, [email protected]
Dennis MaynesChief Scientist, Caveon [email protected]