C Arlene Ash 2009 1

Can Statistics Help Improve US Elections?

Arlene Ash, PhD

Boston University School of Medicine

AAAS

February 15, 2009

C Arlene Ash 2009 2

Change the Way America Runs Its Elections

American U, Center for Democracy & Election Mgmt, June ‘07

• Inaccurate voting lists, uncounted provisional ballots, electronic voting machines that jam without a paper trail … partisan Secretaries of State who conduct the elections… claims that U.S. Attorneys were dismissed for failing to pursue alleged electoral fraud…

• A decade ago, it would have been laughable to ask if America's elections were conducted as professionally and impartially as Mexico's. Today …Mexico has a more independent, impartial, and professional system.

C Arlene Ash 2009 3

Our elections are (too) complicated• Many races and ballot questions• Many opportunities for inadvertent,

disingenuous, or malicious errors– Sometimes the “level of sloppiness” exceeds

the “margin of victory”

• Losers of close races will “cry foul”– Can statistics help evaluate such claims?

• Can statistics help us run better elections?

C Arlene Ash 2009 4

Example: FL 2006, CD-13

• About half of 240,000 votes are from Sarasota County (leans D) and half from several appended (R-leaning) counties

• Buchanan (R) wins by 369 votes

• Challenge by Jennings (D)

C Arlene Ash 2009 5

CD-13

C Arlene Ash 2009 6

The Problem in CD-13

• 18,000 “undervotes” (15% of all votes ) in Sarasota

• 2 - 3% (“normal”) levels of undervoting in all other parts of the district

• Over 15,000 excess lost votes in Sarasota County

• Let’s look at the Sarasota County ballot

C Arlene Ash 2009 7

Screen 1

C Arlene Ash 2009 8

Screen 2

C Arlene Ash 2009 9

Did 15,000 “lost votes” matter in an election where Buchanan’s

margin of victory was 369?

• If the 15,000 were distributed just like the whole population—such a loss would not threaten the outcome

• However, Jennings’ edge in Sarasota County was larger than 50 per 1000.– So these 15,000 lost votes likely represent

an (unrealized) gain of more than 750 for Jennings

C Arlene Ash 2009 10

What Happened?• Statistical modeling of the undervote shows

that more CD-13 voters wanted Jennings, and that Jennings’ likely advantage (absent the undervote) was highly statistically significant

• The courts found– No evidence of fraud– The fact that an electoral artifact that disadvantaged

Jennings’ voters more than her opponent almost certainly cost her the election was not relevant

– The undervote was due to “voter error”

• The election stood

C Arlene Ash 2009 11

Lessons (1)• Excellent, publicly-available data was key to

understanding what happened in FL CD-13– Note: the election had no voter-verifiable paper

record; only some problems could be audited• Ballot design is very important

– Ballots should be pre-tested– Voter education is important– Contemporaneous BOLD warnings – voters should

verify that (important) undervotes are intentional• Only voters in one part of CD-13 faced the

problematic ballot– The bad ballot was like a “thumb on the scale” – Failed elections deserve a remedy

C Arlene Ash 2009 12

Lessons (2)• Election workers should be helped to

avoid known problems – E.g., pre-testing bad ballots, using queuing

theory to minimize long lines

• CQI needed– Exit polling to document voting barriers

(long lines, confusing ballots, …)– Monitoring entire process (from registration

lists to post-election audits) – Both “hot” and “cold” audits needed (“Hot”

= while the election is being certified)

C Arlene Ash 2009 13

States and election auditing• When states audit, they use “%-based”

or “tiered-%” sampling rules– “Risk-limiting audits” are more efficient– States need help in establishing auditable

voting systems, and in developing good auditing laws and “legislative rules”

C Arlene Ash 2009 14

Statistical Power of 10% Audits

C Arlene Ash 2009 15

Statistical Power of a Tiered Audit in a 500-Precinct Jurisdiction

C Arlene Ash 2009 16

What Can Statisticians Contribute?

1. Help states enact sensible laws2. Survey voter perceptions and

experience 3. Advance theoretical knowledge of

efficient procedures 4. Develop protocols to evaluate the

status quo and evaluate the impact of potential “improvements”

C Arlene Ash 2009 17

Statisticians and Electoral Integrity

• American Statistical Association (ASA) seeks to advance the integrity of US elections– Steve Pierson, ASA’s Director of Science Policy is coordinating

efforts among statisticians, other voting integrity activists, and election officers, state (and federal) legislative aides

• Teach statistical basics – The why and how of random sampling – How sample size relates to accuracy– Type I vs. Type II errors (e.g., when updating voter registration lists)

• Involve statistics classes in useful research– Exit polling to identify problems with “the machinery” of voting

• Influence electoral standards– Help define the data needed for routine CQI

• Develop new statistical knowledge relevant to elections– Efficient sampling plans and stopping rules for post-election audits

C Arlene Ash 2009 18

ASA Position Statement Trustworthy elections demand integrity throughout the

entire electoral process, from voting laws and regulations to details of implementation, including maintaining voter registration lists and a secure chain of custody for voted ballots. All processes and data of US elections should be subject to statistically sound, continuous-quality monitoring and improvement. Data releases should be comprehensive and timely and follow standardized, readily analyzable formats. It is critical that the integrity of central vote tabulations be confirmed by audits of voter-verified hardcopy records in order to provide high -- and clearly specified -- levels of confidence in electoral outcomes.

C Arlene Ash 2009 19

American Statistical Association Electoral Resources

Google: “amstat election auditing” “mathaware 08 essays” “asa position electoral integrity”

• Florida’s CD-13 Election in 2006: Can Statistics Tell Us Who Won? – Arlene Ash and John Lamperti

• Roles for Statisticians in Elections– John Gardenier

• Percentage-Based versus Statistical-Power-Based Vote Tabulation Audits– McCarthy, et al. TAS, Feb 2008