what the numbers say: a digit-based test for election ... · what the numbers say: a digit-based...
TRANSCRIPT
What the Numbers Say: A Digit-Based Test
for Election Fraud Using New Data from Nigeria∗
Bernd Beber† Alexandra Scacco‡
August 2008
Abstract
Is it possible to detect manipulation by looking at electoral returns only? Drawing on workin psychology, we exploit individuals’ biases in generating numbers to highlight suspicious digitpatterns in reported vote counts. First, we show that fair election procedures produce returnswhere last digits occur with equal frequency, but lab experiments indicate that individuals tendto favor small numbers, even when subjects have incentives to properly randomize. Second,individuals underestimate the likelihood of digit repetition in sequences of random integers, sowe should observe relatively fewer instances of repeated numbers in manipulated vote tallies.Third, lab experiments demonstrate a strong preference for adjacent digits, suggesting thatnon-adjacent digits should appear with lower frequency on fraudulent return sheets. We test fordeviations in digit patterns using data from Sweden’s 2002 parliamentary election and previ-ously unavailable results from Nigeria’s 2003 presidential election and find strong evidence thatmanipulation occurred in Nigeria.
1 Introduction
Suppose you have been asked to assess how “clean” a past national election was in different areas of acountry. You have poor national-level information about the make-up of the voting population, andvirtually no information about any subnational variation. You have no results from any previouselections. Constituency maps are not available or do not exist. In essence, the only informationyou have is a list of electoral returns. This is a situation election monitors may well encounter insome developing countries, and it is a situation in which regression-based tests for outliers do notwork. Is it possible to say, with some confidence, whether results have been manipulated by lookingat the return sheets only?
We approach this problem by developing a digit-based test that exploits human biases in numbergeneration. We refer to three expectations in particular from the relevant psychology literature:First, fair election procedures should produce returns where last digits occur with equal frequency,but lab experiments have shown that individuals tend to disproportionately select small numbers,even when they have incentives to properly randomize. Second, individuals tend to underestimate∗We thank Katherine Zhang and Matt Heiman for excellent research assistance and Andrew Gelman, Adam Glynn,
Walter Mebane, and participants at previous seminar and conference presentations for valuable comments.†Ph.D. candidate, Columbia University, [email protected].‡Ph.D. candidate, Columbia University, [email protected].
1
the likelihood of digit repetition in sequences of random integers, which means that we shouldobserve relatively fewer instances of repeated numbers on manipulated vote report sheets. Third,lab experiments demonstrate a strong preference for adjacent digits, suggesting that non-adjacentdigits should appear with lower frequency on fraudulent return sheets.
Our approach to detecting electoral fraud is similar to recent work by Mebane (2006) in thatwe highlight digit patterns in reported vote counts. In contrast to Mebane’s inspection of seconddigits, however, we focus on last digits and digit pairs, which permits us to be relatively agnos-tic with respect to the underlying distribution from which vote counts are drawn. While it maywell be reasonable to assume that the second digits of electoral returns follow a Benford distribu-tion, as Mebane suggests, we show that our last digit test requires extremely weak distributionalassumptions.1
The digit-oriented procedure we propose stands in contrast to covariate-based studies, whichemploy regression methods to identify vote returns that are statistically unlikely to have beengenerated by an untainted electoral process (Mebane and Sekhon (2004) and Wand et al. (2001),who use a number of different methods). Some authors have emphasized high levels of turnoutor rapid changes in turnout or vote share as possible indicators of irregularities (Myagkov et al.,2005), but just like models that use data on constituency characteristics, this method works bestif data is available for several elections as opposed to only the one under study. The method wedevelop here can obviously be deployed in data-rich environments as well, but it can also be appliedto cases where data availability is poor (which might in fact be places where fraud is likely to occurin the first place).
We proceed as follows: We first show that last digits will occur with equal frequency for alarge class of theoretical distributions, and we argue that non-fraudulent electoral returns are likelydrawn from such a distribution. We also derive implications for pairs of last and penultimate digits.Section 2 concludes with simulations supporting our claim that the last digits of electoral resultsare likely to be distributed uniformly. Section 3 discusses common human biases in random numbergeneration. Section 4 presents empirical evidence using data from Sweden’s 2002 parliamentaryelection as well as data we retrieved from original enumeration area report sheets used by localauthorities in Plateau state in northern Nigeria during the 2003 presidential election. Section 5concludes.
2 Theoretical expectations for digit distributions
Before we can assess the extent to which observed digit distributions on electoral return sheetssuggest manipulation, we need to establish baseline expectations about how digits should be dis-tributed in a fair election. In this section, we first show that discrete distributions with certaincharacteristics yield (a) last digits that are distributed uniformly (e.g. the number 2 is as likely toappear as the number 5), and (b) sequences where the probabilities with which different numbers
1Nigrini (1999) applies Benford’s Law to detect tax and accounting fraud, and Schafer et al. (2004) use it toidentify fake surveys.
2
appear after a given digit are uniformly distributed as well (e.g. the number 2 is as likely to appearas the number 5 after the number 1). As a general rule, this result obtains if a distribution meetstwo requirements: First, the density in its tails approaches zero, and second, it can be approxi-mated in a piecewise linear fashion over intervals of a size corresponding to the base of the numeralsystem. We state and prove these requirements more formally below. We then show, by way ofsimulations, that a wide variety of theoretical distributions meets these requirements.
2.1 Conditions under which last digits are uniformly distributed
We conceptualize the numbers observed on an electoral return sheet as draws from a randomvariable X, which follows some probability distribution f . For convenience, we sometimes conceiveof f as a discretization of a continuous probability density function g, where
f(z) =∫ z+1
zg(x) dx, (1)
where z ∈ Z∗.We prove our results in general for any numeral system with base b > 1. The electoral returns
we check for manipulation are written in base 10, as one would expect.Proposition 1 states that the last digits of the draws from X are distributed uniformly if three
conditions are met: the support of g is divisible by base b of the numeral system; the density of g
at the lower bound of its support is equal to its density at the upper bound; and we can linearlyapproximate g in segments of size b. Here we assume that we can approximate g without error,an assumption relaxed by proposition 2. Proposition 3 relaxes the assumption that the support ofthe probability density function is divisible by b, although at the cost of requiring that its densityapproach 0 in the tails.
Proposition 1. Consider a discrete, non-negative random variable X with probability density f ,where f discretizes continuous probability density function g by piecewise integration over intervalsof size 1, and g has domain [s1, s2], where s2−s1 is divisible by base b of a given positional numeralsystem. Then the occurrence of numerals in the last digit of X is distributed uniformly if g can beestimated by piecewise linear approximation over consecutive intervals of size b, and g(s1) = g(s2).
All proofs are located in the appendix, unless stated otherwise. The intuition behind the prooffor proposition 1 is that if density function g takes on the same value at its lower and upper bound,then any linear change in g over a given interval of b digits (and hence any differences in the densityattributed to last digits) will eventually have to be offset by the equivalent negative change in g
over another interval of size b.Proposition 2 addresses the open question of how accurate the linear approximation of g (and
in turn of f) has to be in order for uniformly distributed last digits to occur. (The proof forproposition 1 is limited to the case in which there is no error in the linear approximation of densityfunction g, i.e. g is a piecewise linear function.)
3
Proposition 2. Suppose the discrete, non-negative random variable X has probability densityf(ab) + kad + fe(ab + d) with domain [s1, s2] for any a ∈ { s1
b , . . . , s2−bb } and d ∈ {0, . . . , b − 1},
given base b of the numeral system (i.e. suppose that we can decompose f into a piecewise linearcomponent and an “error” component fe(.) within each interval of size b). Then proposition 1 holdsif
s2−bb∑
a=s1b
fe(ab + d1) =
s2−bb∑
a=s1b
fe(ab + d2)
for any d1, d2 ∈ {0, . . . , b− 1}.
In other words, our result from proposition 1 obtains if a piecewise linear approximation under-and overestimates the true density to the same extent across possible last digits. The bias inducedby a linear approximation in the density of different last digits has to average out over the supportof the density function. One direct implication is that we can expect last digits to appear withequal frequency if the error around the linear approximation of the density function is zero inexpectation, as stated in corollary 1.
Corollary 1. Proposition 1 holds if E(fe(ab + d)) = 0 for all d ∈ {0, . . . , b− 1}.
Proof. Follows directly from proposition 2.
We can provide a similar, but less strict statement in terms of the partial derivative of thesummed approximation error with respect to d. While corollary 1 shows that last digits are uni-formly distributed if the piecewise linear approximation provides on average an unbiased estimateof the density function, corollary 2 states that the approximation can in fact be biased, as long asit is biased in the same way for draws ending in different digits.
Corollary 2. Proposition 1 holds if∂∑ s2−b
b
a=s1b
fe(ab+d)
∂d = 0 for d ∈ {0, . . . , b− 1}.
Proof. Follows directly from proposition 2.
It is apparent from this corollary that the key class of error functions fe for which proposition 1will not hold are periodic or quasi-periodic with period b. The following corollary illustrates thisinsight by pointing to multiplicatively or additively separable error functions: Last digits are notuniformly distributed if the error around the linear density approximation can be written as theproduct or sum of one function of a and b and a separate function of d.
Corollary 3. Proposition 1 does not hold if fe(a, b, d) = fe(a, b)he(d) or fe(a, b, d) = fe(a, b) +he(d), and he(d) is not constant.
We will show in section 2.2 that this is of no particular concern for a wide variety of distributionsthat could underpin fair electoral processes. It is difficult to think of reasons why the distributionof electoral results might follow a periodic distribution with period 10. Indeed, only very particular
4
(and peculiar) distributional assumptions will consistently produce last digits that are not uniformlydistributed.
Proposition 3 relaxes the assumption that the support of the distribution of electoral returnshas to be a multiple of numeral base b, but in turn it requires that the density has to approach0 for very large or very small returns. In reality, this is an issue only for small returns, becausewe cannot reasonably extend the support of f below its natural lower bound of 0. If there is anon-trivial probability of observing less than eighteen votes for a unit of interest, then proposition 3does not hold (although proposition 1 still may).
For convenience, proposition 3 explicitly imposes a restriction on the linear approximation error,which is equivalent to the restriction discussed in corollary 1.2
Proposition 3. Consider a discrete, non-negative random variable X with probability densityfunction f and domain {s1, . . . , s2}. Suppose f can be approximated by an arithmetic progressionfor any sequence containing 2b−1 elements, where b is the base of the positional numeral system, andthe approximation error follows function fe, where E[fe(z + d)] = 0 over z ∈ {s1, . . . , s2− 2(b− 1)}for any d ∈ {0, . . . , b − 1}. Then the occurrence of numerals in the last digit of X approaches auniform distribution as f(x) approaches 0 for x ≤ s1 + 2b− 3 and x ≥ s2 − 2b + 3.
The intuition behind the proof of proposition 3 is similar to the one for proposition 1. Here weshow that the total density for different last digits in sequences of size 2(b − 1) is proportional toa constant if we can linearly approximate the density function within each sequence. In the proofof proposition 1, we broke density function g into consecutive pieces of size b. Here the pieces areoverlapping, with a sequence starting at each integer, and in turn the density function’s supportno longer has to be divisible by b.
Finally, no formal proof is needed to see that if last digits are independently and uniformlydistributed, then (a) in expectation no last digit will be repeated more frequently than any otherin a series of N random draws, and (b) the expected number of repetitions (i.e. consecutive drawsof the same last digit) is N−1
b . We argue that the type of empirical data we consider lends itselfto the assumption that last digits are independently distributed. It is certainly possible that thelast digit of the total number of votes cast at a polling station is correlated with the last digit ofthe vote count at the next polling station. But if turnout is in the several hundreds, as it is in ourdata, it would take a spatial correlation of unlikely magnitude to carry through to the last digit.
Also note that if last and penultimate digits are independently distributed, and last digitsare distributed uniformly, then the expected number of pairs with digit repetition is again N−1
b ,regardless of how the penultimate digit is distributed. Even if the second-to-last digit was alwaysthe same, it would not change the fact that the last digit is a match with probability 1
b . If we thinkabout the minimum distance between penultimate and last digits more generally (for convenience,we like to visualize numerals in a circle, in which case it is easy to see that the minimum distancebetween 7 and 1, for example, is 4), we can say that this distance is 0 with probability 1
b , it is 1 with
2Note that in this case f is linearly approximated over sequences of size 2(b− 1) rather than b, which means thatproposition 3 actually places a somewhat stricter restriction on the approximation error than corollary 1.
5
probability 2b , and it is greater than 1 with probability b−3
b (for b > 2). We later use simulations toconstruct confidence intervals around these expected values, and we investigate the extent to whichreturn sheets significantly deviate from them.
2.2 Distributions for which last digits are uniformly distributed
The previous section suggested that we require very particular distributional assumptions in orderfor last digits not to be distributed uniformly. Figure 1 illustrates this point. We parameterizedeight different distributions in terms of their first two moments and simulated 2500 random drawsfrom these distributions for each of 400 sets of means and standard deviations. Figure 1 indicateswhether we could reject the null hypothesis that the random draws’ last digits were distributeduniformly: A gray circle means that we cannot rule out that last digits occurred with equal fre-quency, while a black X indicates otherwise. Blank space suggests that a distribution cannot yieldthe relevant combination of mean and variance.
All distributions for which we simulated draws quickly result in equal-frequency last digits,including the mixtures that Mebane (2006, 8–9) proposed as examples in which the second digitfollows the Benford distribution. The mean and standard deviation of the empirical data dis-cussed in this paper far exceed the values for which last digits were distributed uniformly in thesesimulations.
The simulations underline the fact that we are agnostic about the exact statistical process bywhich electoral returns are generated. One key advantage of our approach is that we need not beconvinced that election results follow any particular distribution (or mix of distributions), becausethe theoretical result of uniformly distributed last digits holds across a wide range of data-generatingprocesses.
3 Psychological biases in number generation
While many statistical distributions generate equal-frequency last digits, human beings typicallydo not. A host of studies in psychology and statistics address ways in which people perceiverandomness and their ability to produce random sequences in experimental settings (see Nickerson(2002) for a review). These studies mostly focus on binomial trials, however, and usually asksubjects to produce sequences of heads and tails to simulate results from coin tosses. We foundonly three sets of experiments in the literature that directly address the problem of generatingsequences of decimal digits. In the first such experiment, Chapanis (1953) asked 13 subjects withvarying levels of formal education in statistics and probability theory to write out sequences of2520 single digits. Subjects were asked to write down the digits 0 to 9 in “random order” on sheetsof paper, without interruption, at their own pace. Chapanis found that subjects displayed markedpreferences for certain digits, although they disagreed in their preferences. Without exception,subjects avoided repetition in their sequences, rarely creating repetitive pairs or triplets (such as111, 110, or 101). They also preferred decreasing (e.g. 987) to increasing (e.g. 123) sequences, and
6
Mean µµ
Sta
ndar
d de
viat
ion
σσ
0
10
20
30
40
50
0 10 20 30 40 50 60 70 80 90 100
●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
Binomial (for σσ2 << µµ), negative binomial (for σσ2 >> µµ),
and Poisson (for σσ2 == µµ) distributions
0
10
20
30
40
50
0 10 20 30 40 50 60 70 80 90 100
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
Discretized normal distribution, truncated at 0
0
10
20
30
40
50
0 10 20 30 40 50 60 70 80 90 100
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
Discretized uniform distribution, truncated at 0
0
10
20
30
40
50
0 10 20 30 40 50 60 70 80 90 100
●●●
●●●●●●
●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
Discretized Gamma distribution
0
10
20
30
40
50
0 10 20 30 40 50 60 70 80 90 100
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
Discretized mix of distributions(Mechanism A, Mebane 2006)
0
10
20
30
40
50
0 10 20 30 40 50 60 70 80 90 100
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
Discretized mix of distributions(Mechanism B, Mebane 2006)
Figure 1: Theoretical distributions and uniformly distributed last digits
7
choices were auto-correlated to the fourth order.In a similar experiment, Rath (1966) asked 20 university students to each produce 2500 random
digits by filling in ten one-page grids of 250 rectangles each. The students were told that they couldleave the experiment as soon as they filled in all of the sheets. Rath found three clear patterns insubjects’ digit choices. First, subjects significantly preferred small numbers (1, 2, and 3) over bothlarger numbers (5, 7, 8, and 9) and zero. Second, he found very strong biases against repetitivepairs of digits, and third, subjects exhibited strong biases in favor of adjacent pairs of numbers(such as 12, 23).
Boland and Hutchinson (2000) echo these findings in an experiment with 458 university students,where each was asked to produce random sequences of 25 single digits. Again, subjects preferredsmall digits (particularly 1, 2, and 3) and avoided others (5, 6, 8, and 0). They also find anespecially strong effect for the avoidance of repetition. A striking 70 percent of respondents failedto repeat a single digit in their 25 digit-sequence, whereas this would be expected to occur only8 percent of the time using a random number generator. Finally, they find strong preferences foradjacent digits.
The literature thus suggests three key findings: (1) digits are not selected with equal frequency,(2) repetition is avoided, and (3) serial sequences are preferred. The first two findings in particularare consistent with a larger theoretical and experimental literature on cognitive biases in probabilityperception, such as the “gambler’s fallacy” (where people expect the second draw of a signal tobe negatively correlated with the first) and beliefs that very small samples resemble the parentpopulations from which they are drawn (Tversky and Kahneman, 1972). Drawing on these studies,we test for the three patterns identified above.
4 An empirical assessment of electoral fraud
4.1 Establishing an empirical baseline
We establish an empirical baseline, a null result against which our later analysis can be compared,using data from the 2002 parliamentary elections (Riksdagsval) in Sweden.3 We analyze data at theward level (valdistrikt), where the 5976 wards in our dataset are nested within 290 municipalities(kommun) and 21 counties (lan). As far as we can tell, there were no suspicions of electoral fraud orreturn sheet manipulation raised with respect to this election, and our theoretical expectation is thatlast digits will be distributed uniformly. Figure 2 shows last digit frequencies across all wards forthree different return sheet columns: votes for the Social Democratic Party (SocialdemokratiskaArbetareparti, SAP), votes for the Moderate Party (Moderata Samlingspartiet, MSP), and thenumber of registered voters (antal rostberattigade). Horizontal lines indicate the expected value of.1 as well as the lower and upper confidence bounds. Our expectation is confirmed: Each numeralappears roughly as often as any other.
3The data is available at http://www.val.se/val/val 02/radata/radataslut.html. Retrieved on March 1, 2008.
8
SAP results
0
.08Mean
.12
.22
0 1 2 3 4 5 6 7 8 9
MSP results
0
.08Mean
.12
.22
0 1 2 3 4 5 6 7 8 9
Share of last digitsNumber of registered voters
0
.08Mean
.12
.22
0 1 2 3 4 5 6 7 8 9
Figure 2: Frequencies of last digits, Sweden 2002
Similarly, we do not expect to observe a lack of digit repetition or an improbable glut of adjacentdigits. Figure 3 provides evidence to that effect. Each point represents one of the municipalities,which are sorted by size (i.e. ward count) along the horizontal axis. Point size is proportional toturnout, which we measure as the share of registered voters who cast a ballot. We consider herepairs of last and penultimate digits found in the columns for the number of registered voters, theSAP vote count, and the MSP vote count within each municipality.
The vertical axis gives the quantity of interest. For the first graph, it denotes the extent towhich the last and penultimate digits within a given municipality are the same, relative to the lowerconfidence bound. Municipalities marked in black above the dashed line at 0 have suspiciously fewrepetitions. The second graph shows the extent to which digits are adjacent in a given municipality,relative to the upper confidence bound. Points above 0 have worryingly many pairs of adjacentdigits. The third graph displays the degree to which we observe pairs of non-neighboring digits,relative to the lower confidence bound. The black dots indicate municipalities with suspiciouslyfew pairs of non-adjacent digits.
There are a small number of municipalities that are seemingly suspicious, but this is the result ofthe fact that we plot unadjusted 95% confidence bounds for a test of many hypotheses—one for eachmunicipality. Since we plotted just short of 200 municipalities (in order to facilitate comparisonwith our analysis of Nigeria’s 2003 election), it is not surprising that a small number of them will liebeyond the 95% confidence interval purely by chance. Again, we cannot reject the null hypothesisof a “clean” election.
4.2 Data and results from Nigeria
We now use our digit-based test to examine electoral returns from Nigeria. In particular we analyzedata at the polling station level for Plateau state, which is located in the “middle belt” region of thecountry. We were able to retrieve this data in 2006, and it is, to our knowledge, the first time thatpost-colonial election data at this level of aggregation has been available outside of Nigeria. The
9
5th pctl of simulated frequency with which last two digitsare identical, minus empirical frequency
510
2050
100
300
1000
−.2
−.10
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
● ●
●
●●
● ●
●
● ●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●●
●●
●●
●●
●● ●
●
●● ●
●●
●
●
●
●
●●
●
●
●
●●
● ●●●
●
●
●
●
●●●
●
●
●
●
●
●
●●●●
●●
●●
●
●
● ●
●
●●
●
●
● ●
●
● ●
●● ●●
●●
● ●
●
●
●
●
● ● ●
●
●
●●
●
●
●
●
●●
●●
●●●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●●
●● ●
●●
●●
●●●
●●
●
● ●
●
Empirical frequency minus 95th pctl of simulated frequency withwhich distance between last and penultimate digit is one
Num
ber
of w
ards
in m
unic
ipal
ity (
logg
ed)
510
2050
100
300
1000
−.4
−.3
−.2
−.10.1
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
● ●
●●
●
●●
●
●
●●
● ●
● ●
●
● ●
●
●
●
● ●
●
●●
●
●
●
●●● ● ●
●●
●
●
● ●
● ●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●● ●
●●
●
●
●●
●
●●●
●
●●
●
●●
●● ●
●● ●
●
●
●
●●
● ●●
●●
●
●●
●
●
● ●●
●
●
● ●●
●●
●●
●
●
●
●
● ● ●
●
●
●
●
●
●
●●
●
●
● ●
●
●●
●
●
●
●
●●
●●
●●
● ●●●
●●
●
Point size is proportional to turnout
Mul
tiple
col
umns
5th pctl of simulated frequency with which distance between lastand penultimate digit is greater one, minus empirical frequency
510
2050
100
300
1000
−.4
−.3
−.2
−.10
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●●
●
●
●
●
● ●
●
● ●
●●
●● ●
●
●
●
● ●
●
●
●
●
●
●
●●
●● ●
●●
●
●
● ●
●●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●● ●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
● ●●
●● ●
●
●
●
●
●●
●
●
● ●
●●
●
●
●●● ●●
●
●●
●
●●
●
●
● ●●
●
●
●
●●
●
●
● ●
●
●
●
●
● ●●
● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●● ●●
●●
●
Fig
ure
3:L
ast
and
penu
ltim
ate
digi
ts,
Swed
en20
02
10
body of data we gathered consists of a nearly complete record of the 2003 presidential, gubernatorial,and parliamentary elections for Plateau and, to some extent, neighboring Kaduna state. We hereanalyze presidential election returns for Plateau state.
All results were entered from original, handwritten electoral ward report sheets used by localauthorities, and we focus on the ward as our target of analysis. Figure 4 provides an example ofsuch a ward-level return sheet, highlighting the digits we will analyze below. For the vast majorityof sheets, the handwriting suggests that only a single official entered the results, so these returnsseem as plausible a collection of data as any to assess whether we can discern fraudulent sheetsby leveraging individuals’ psychological biases. We analyze results from 198 wards, where eachward contains an average of about 13 polling stations. Ward size varies significantly: Wards in thebottom decile contain no more than five polling stations each, while those in the top decile are eachcomprised of more than 23 polling stations. In turn, the power of our analysis varies substantiallyacross wards.
It is no secret that electoral fraud occurred during the 2003 elections in Nigeria and that someward results were manipulated wholesale. At the time of the 2007 elections, The Economist reportedthat in Awka, Anambra state, “barely any polling stations were provided with a results-sheet [. . . ];presumably these were being filled in elsewhere” (The Economist, April 21, 2007). Similarly, “thehead of the Catholic Bishops Conference [. . . ] cited massive fraud and disorganisation, includingresult sheets being passed around to politicians who simply filled in numbers as they chose whilebribed returning electoral officers looked away” (AFP, April 25, 2007). We believe the fact thatwe know fraud occurred in Nigeria makes the country an especially useful case for validating ourdigit-based approach. We have used data from Sweden to illustrate that our digit-based test doesnot ring alarm bells in an apparently non-fraudulent environment. Data from Nigeria will help usestablish that our test is in fact sensitive to real manipulation of election results.
In the Nigerian context, covariate-based tests for electoral fraud face serious challenges. Verylittle data is available at the sub-state level, and none at the ward level. Census data is notoriouslyunreliable—the most recent census prior to 2003 took place in 1991 and was marred by blatantfraud and large-scale violence—and few would argue that it accurately reflects the facts on theground. In this environment, monitors and outside observers need other tools to assess patterns offraud.
A digit-based approach helps monitors and observers in two other ways as well. First, someareas, particularly in Nigeria, are unsafe for election monitors to visit, so tools for detecting fraudother than direct observation are needed. Second, if observers help curtail fraud techniques such asintimidation and ballot box stuffing at the polling station level, they will also need tools to validatethat elites have not simply shifted to manipulating return sheets.
Why does it make sense to look at ward-level manipulation in Plateau state, in particular? Ina paper on the 2003 elections, Darren Kew groups Nigeria’s states into three broad categories offraudulence (Kew, 2004):
• In a handful of states, including Lagos state, minor violations occurred across polling stations,
11
Fig
ure
4:E
xam
ple
ofa
war
d-le
vel
retu
rnsh
eet,
wit
hhi
ghlig
hted
colu
mns
ofin
tere
st
12
but “most did not detract from holding serious elections, with results at polling stations andwards that were generally accurate” (157).
• Another dozen states, saw “fairly credible elections, but with some instances of rigging in rurallocations. Moreover, their results grew increasingly questionable as they moved through theLGA [local government area] and state collation centers” (157). Kew includes Plateau statein this middle category, which he describes as a context in which “localities made a pretenseof voting properly at the ballot box and then altered the numbers at the LGA level” (163).
• In another third of Nigeria’s states, leaders of the incumbent People’s Democratic Party(PDP) “did not even attempt to erect a facade of credibility. Ballot boxes were brazenlystuffed in the presence of observers” and, in some cases, “voters were intimidated and keptaway” (161). For example, at a polling station in Adamawa, Kew himself watched as a PDPagent “stood by the inkpad where voters made their choices, showing them where to vote forthe PDP” (158). Further types of fraud documented in these worst-offender states includedwidespread underage voting and outright physical intimidation of voters and monitors atpolling booths.
The second group of states is precisely the context where our application—looking for manip-ulation at local collation centers rather than at the polling stations themselves—is most useful.The small number of states in the first category may have had little fraud at all, while in states inthe third category, manipulation of digits at local collation centers is likely to be overshadowed byblatant fraud at polling stations.
PDP results
0
.08Mean
.12
.22
0 1 2 3 4 5 6 7 8 9
Total vote counts
0
.08Mean
.12
.22
0 1 2 3 4 5 6 7 8 9
Share of last digitsNumber of registered voters
0
.08Mean
.12
.22
0 1 2 3 4 5 6 7 8 9
Figure 5: Frequencies of last digits, Nigeria 2003
We first examine the frequencies with which different numerals appear in the last digit. Figure 5provides digit frequencies across all wards for three different return sheet columns: votes receivedby the PDP, total vote counts, and the number of registered voters. The difference to figure 2is striking. For all three columns, we observe significant deviations from the uniform distribution
13
(which are marked in black), in particular for the numeral 0. This strongly suggests that electoralreturns were indeed manipulated.
Another way to test whether last digits depart significantly from the uniform distribution isto measure the extent to which digit frequencies vary. Asymptotically, they should not vary atall; for the number of polling stations in the sample, we expect a standard deviation of about.006, with only a 5% chance of a standard deviation of more than .009. Figure 6 shows that lastdigit frequencies in all three columns vary significantly more than they should if results were notmanipulated. For total vote counts, digit frequencies have some six times the standard deviationthat we would expect in a “clean” election.
Standard deviation of share of last digits
Mean: .00695%: .009
0
.04
.01 PDP
.036 Total votes
.023 Registered voters
Figure 6: Standard deviations for last digit frequencies
Figures 5 and 6 present aggregate results, but we can also use our test to uncover variationacross wards. Figures 7 and 8 again show last digit frequencies, but for a random sample of abouta third of the total number of wards. Figure 7 looks at the return sheet column containing totalvote counts, while figure 8 plots digit frequencies across all three columns of interest. Frequenciesmarked with a black rather than gray line exceed the dashed 95% confidence bound.
A substantial number of wards produce deviations from the uniform distribution, many morethan we would expect even when testing such a large number of hypotheses. In suspicious wards,zeros in particular are overabundant. We argue that this pattern suggests fraud, but one couldargue that many zeros simply indicate benign laziness as election officials are rounding to the nearestmultiple of ten to facilitate their election day accounting. One piece of evidence lends some supportto this alternative hypothesis: Figures 7 and 8 sort wards from smallest (fewest polling stations) to
14
Inde
x
Pro
port
ions
of l
ast d
igits
for
tota
lpo
lling
sta
tion
vote
s, b
y w
ard,
sor
ted
by n
umbe
r of
pol
ling
stat
ions
, with
unad
just
ed 9
5% c
onfid
ence
bou
nd
Inde
x
NA
011
02
46
8
Inde
x
NA
.5
02
46
8
Inde
x
NA
.5
02
46
8
Inde
x
NA
.5
02
46
8
Inde
x
NA
.5
02
46
8
Inde
x
NA
.6
02
46
8
Inde
x
NA
.5
02
46
8
Inde
x
.5
02
46
8
Inde
x
NA
.5
02
46
8
Inde
x
NA
.5
02
46
8
Inde
x
NA
.5
02
46
8
Inde
x
NA
.5
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.4
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
.3
NA
.3
NA
.3
NA
.3
NA
.3
NA
.3
NA
.3
NA
.2
NA
.2
NA
.2
Fig
ure
7:D
igit
freq
uenc
ies
ina
rand
omsa
mpl
eof
war
ds,
war
dsw
ith
addi
tion
erro
rshi
ghlig
hted
15
Inde
x
Pro
port
ions
of l
ast d
igits
for
mul
tiple
retu
rn s
heet
col
umns
, by
war
d, s
orte
dby
num
ber
of p
ollin
g st
atio
ns, w
ithun
adju
sted
95%
con
fiden
ce b
ound
Inde
x
NA
0.51
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.3
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
Inde
x
NA
.2
02
46
8
.2
NA
.2
NA
.2
NA
.2
NA
.2
NA
.2
NA
.2
NA
.2
NA
.2
NA
.2
Fig
ure
8:D
igit
freq
uenc
ies
ina
rand
omsa
mpl
eof
war
ds,
war
dsw
ith
addi
tion
erro
rshi
ghlig
hted
16
largest (most polling stations), and it appears that officials in larger wards are more likely to fill areturn sheet with zeros. A logit regression of a ward-level indicator for suspiciously distributed lastdigits (which equals 1 if we can reject the null hypothesis of uniformly distributed last digits forany of the three return sheet columns in a given ward) on ward size yields a statistically significantand positive coefficient (.05, with standard error .02).
There are two reasons, however, why we are not convinced by this argument. First, we are morelikely to reject the null of no manipulation in large wards for the simple reason that they are large,whether or not ward size is a predictor of digit modification. The more polling stations a wardcontains, the tighter the 95% confidence bound around the uniform distribution of last digits, andthe fact that suspicious digit distributions occur more frequently in large wards could be spuriousto the fact that we face greater uncertainty in evaluating small wards. (Note that zero is in factthe most common numeral for a number of smaller wards, even if its frequency does not surpassthe relevant 95% confidence bound.)
Second, we observe an excess of zeros in total vote counts in particular, but there are only fewpolling stations (<.8%) in which all vote columns end in zero. (There are 19 such polling stations,and in all but one of them only votes for the two main parties ANPP and PDP were recorded.)But this implies that all other excess zeros, if they are the result of officials innocently roundingtotal vote counts, should go together with addition errors. This is not the case: Figures 7 and 8highlight wards in which we observe such addition errors particularly often, i.e. for more than 20%of the polling stations, and these are not the same wards as the ones in which we see suspicious digitdistributions. In fact, a logit regression yields a negative, although insignificant coefficient (-0.43,with standard error 0.42). If anything, it seems, officials manipulating return sheets try to makesure that the numbers add up, which ironically makes their return sheets all the more suspiciousgiven the otherwise high error rate.
Figures 9 and 10 show a measure of the frequency of the most common last digit only, but theydo so for all wards. Each point represents a ward, its horizontal position indicates its most frequentlast digit, its vertical position shows the extent to which the last digit’s frequency falls above orbelow the 95% confidence bound, and the point size is proportional to ward size. As before we findsuspiciously non-uniform distributions of last digits in a substantial number of wards.
These two figures also provide information on ward turnout, by way of point shades. The graphsillustrate that there is no significant correlation between turnout and suspicious digit distributions,and a logit regression of an indicator of potential manipulation on turnout produces a statisticallyinsignificant coefficient (0.71, with standard error of 1.56). Turnout is dubiously high in a numberof areas, with about one in seven wards reporting turnout among registered voters to be higherthan 95%, but these areas are generally not the same as the areas in which we find suspicious digitdistributions.
We think there are at least two reasons for this disconnect. First, in the absence of informationon demographics, we compute turnout as the ratio of total votes cast to registered voters. In andof itself, this procedure leaves us with an inflated estimate of turnout (the median ward turnout
17
Num
ber
of r
egis
tere
d vo
ters
Mos
t fre
quen
t las
t dig
it on
war
d re
turn
she
et
Share of digit proportion that exceedsupper bound of 95% confidence interval
01
23
45
67
89
−.4
−.20.2
● ●
●
●
● ● ●●
●●
● ● ●●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
● ●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●●
●●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●●
●
●
●
●●●
●●
●
● ●●
●
●
●
●●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●●●
●●
●●
●
●
●
●●
●●
●
●
●●
●●●
●●
●●
●●
●
●
●
●●
●●
●
●
●●
●●
●●
●●
●
●●
●●
●●
●●
●
●●
●●
●
●
●●
●●
●●
●
●
●●
●●
●
●● ●
●
●●
●
●
●●
33%
Tur
nout
100%
Poi
nt s
ize
is p
ropo
rtio
nal t
onu
mbe
r of
pol
ling
stat
ions
Bar
kin
Ladi
, Lob
iring
Bok
kos,
Ric
ha
Jos−
Nor
th, A
bba
Na
She
hu
Jos−
Nor
th, G
agar
e
Jos−
Nor
th, J
enta
Apa
ta
Jos−
Nor
th, N
arag
uta
B
Jos−
Nor
th, J
os J
araw
a
Jos−
Nor
th, T
udun
Wad
a/ K
abon
g
Man
gu, J
anna
ret
Pan
kshi
n, P
anks
hin
Cen
tral
Riy
om, B
um
Fig
ure
9:D
evia
tion
from
confi
denc
ebo
und
for
mos
tco
mm
onla
stdi
git
18
Mul
tiple
col
umns
on
retu
rn s
heet
Mos
t fre
quen
t las
t dig
it on
war
d re
turn
she
et
Share of digit proportion that exceedsupper bound of 95% confidence interval
01
23
45
67
89
−.2
−.10.1.2
● ●
●●● ●
●●
●●
●●
●
●
●
●●
●
●●
● ●●
●●●
●●
●
●●●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
● ●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●
●●
●●
●
●●
●●
●
●
●●
●
●●
●
●●
●●
●
●●
●
●
● ●●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
33%
Tur
nout
100%
Poi
nt s
ize
is p
ropo
rtio
nal t
onu
mbe
r of
pol
ling
stat
ions
Man
gu, K
ombu
n
Riy
om, B
um
Riy
om, S
haru
butu
Was
e, B
asha
r
Was
e, K
umbu
r
Was
e, M
avo
Was
e, W
ase−
Tof
a Fig
ure
10:
Dev
iati
onfr
omco
nfide
nce
boun
dfo
rm
ost
com
mon
last
digi
t
19
is about 85%), which limits the extent to which high turnout can usefully serve as an indicatorof election fraud. This problem is aggravated by the fact that the registration process in Nigeriaprior to the 2003 elections “was marred by a number of severe irregularities” (European UnionElection Observation Mission, 2003, 21), which may have prevented some 30%-40% of eligiblevoters from registering (although Nigeria’s Independent National Election Commission maintainedthat the register accounted for virtually all adult Nigerians). With registration figures artificiallydepressed, particularly among the less persevering, who would presumably have been least likelyto turn out to vote if they had been registered, our estimated turnout figures are inflated further.
Second, the lack of correlation between dubiously high turnout figures and our digit-basedmeasure arguably points to the fact that these indicators are sensitive to different types of fraud. Ameasure focusing on last digits captures manipulation of return sheets by the individual writing innumbers. High turnout figures could be the result of underage or forced voting, ballot box stuffing,or cheating on report sheets. We gladly isolate a specific type of fraud, which could potentially betraced to named individuals, but are not surprised to find that other mechanisms of fraud couldplay an important role in areas for which we cannot reject the null of a “clean” election.
Figure 9 suggests a similar narrative with respect to differences between urban and rural areas:A number of questionable ward sheets come from Jos-North, a densely populated urban localgovernment area. More generally, we find dubious digit sequences not only in rural, but alsoin many urban areas, and a logit regression of our indicator of potential manipulation on anindicator for urban areas, coded on the basis of a 2003 map of Plateau state (Satod CartographicConsultants, 2003), even yields a positive, although statistically insignificant, coefficient.4 Thisstands in contrast to the observations of election monitors in 2003, who generally concluded thatelectoral fraud affected rural areas most severely. Most observers noted that flagrant violations ofrules at the polling stations were much more likely to occur in rural as opposed to urban areas,because international election observers were far more prevalent in cities (Kew, 1999). But thissuggests that in rural areas, manipulation of return sheets was likely to be unnecessary in areaswhere fraud occurred. In larger urban areas (like the city of Jos), on the other hand, manipulationof election results was more likely to occur behind closed doors, at ward-level collation centers.Arguably, different types of fraud can serve as substitutes for one another: In urban areas, digitmanipulation was more likely to occur on ward sheets than in rural areas, where such fraud wasunnecessary.
Finally, we assess the extent to which digit pairs exhibit repetition or adjacency across wards.Figures 11 and 12 use data for Nigeria, but are equivalent in design to figure 3. Surprisingly, we findno solid evidence of return sheets with too few digit repetitions, nor do we detect an overabundanceof adjacent digits. (Recall that we can expect a small number of wards to exceed the confidencebound purely by chance.) We do, however, find that in a large number of wards, digit pairs do notoften enough bridge a distance of more than one. When we look at several columns for each ward
4Recall that ward size is positively and significantly correlated with our indicator of suspiciously distributed lastdigits, a result that is consistent with this argument.
20
5th pctl of simulated frequency with which last two digitsare identical, minus empirical frequency
25
1020
5010
0
−.5
−.4
−.3
−.2
−.10
●● ●● ●
●
●
●
●
●
●●
●
●
●
●●● ●●
●
●
●
●● ●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●●
● ●
●
●●
●
●
●
●●
●
●●
●
●
●
● ●●
●
●
● ●
●
●
●●
●
●
●
● ●
●
●
●
●
●●
●● ●
●●
●●
●● ●●●
●●
●●●
●●
●●
●●
●●
●●
●●
●●● ●● ●●
●●
●●
●●
●●●
● ●●
●●●
●●
●●●
●●●
●●●
●●
●●
●●
●● ●
●●
●● ●●
●
Empirical frequency minus 95th pctl of simulated frequency withwhich distance between last and penultimate digit is one
Num
ber
of p
ollin
g st
atio
ns in
war
d (lo
gged
)
25
1020
5010
0
−.7
−.6
−.5
−.4
−.3
−.2
−.10.1.2.3.4.5
●
● ●●
●
●● ● ● ●
●
●
●●
●
●
●
●
●●
●
●● ● ●
●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●●
● ●●
●
●●● ●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ● ●●● ●
● ●
●●
●
●
●
●●
●
● ●
●● ●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
● ●
●●
● ●
●
●●
●
●
●● ●
●
● ● ●●
●
●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●●
● ●
●
●
●●
●
●
●●●
●●
●●
●●
●●
● ●●
●●
●●●
●●
●
●●
●
●●
Point size is proportional to turnout
PD
P r
esul
ts
5th pctl of simulated frequency with which distance between lastand penultimate digit is greater one, minus empirical frequency
25
1020
5010
0
−1
−.9
−.8
−.7
−.6
−.5
−.4
−.3
−.2
−.10.1.2.3
●
●●●●
●● ● ●●
●●
●
●●
●
●●
●●● ●●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
● ●●
●
●●● ●
● ●● ●
●●
●
●●
● ● ●
●
●
●
●
●
●
●
●
●
●
●● ●
● ● ●● ● ●
●●
●●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
● ●
●●●
●
●
●
●
●
●
●
●●●
●
● ●
●
●
● ●
●●
●●
●
●●
●
●
●
● ●
●
● ● ●●
●
●●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●●●
●
●
●●
●●
●●
●●
●
● ●●
●
●●
●
●
●
Fig
ure
11:
Las
tan
dpe
nult
imat
edi
gits
21
5th pctl of simulated frequency with which last two digitsare identical, minus empirical frequency
1020
5010
020
0
−.3
−.2
−.10
●
●●●
●
●● ● ●
●
●
●
●●
●
●
●
●
●● ●●
●
●
●
●
● ●
●
● ● ●
●●
●
●
●
●
●
●●
●● ●
●
●●
●
●
●●●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
● ●
● ● ● ●●●
● ●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
● ●●
●
●
● ●
●
●
●●
●●
●
●●
●●
●
●● ●
●
● ●●●
●
● ●
●
●
●
●
● ●
● ●
●
●
●
●
● ●● ●
●
●●
●
●
●●
●
●●
●
●
●
● ●
●
●
●● ●
●
●●
●●
● ●●●
●●
●● ●
●●●
●
●
● ●
Empirical frequency minus 95th pctl of simulated frequency withwhich distance between last and penultimate digit is one
Num
ber
of p
ollin
g st
atio
ns in
war
d (lo
gged
)
1020
5010
020
0
−.5
−.4
−.3
−.2
−.10
●● ●●
●
●● ● ●●
●●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●●
●
●
●
●● ●
●
●
●
● ●
●
●
●
●
● ●●
●
●●
●
●
●●●●
●
●
●
●
●●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
● ●
●● ● ●● ●
● ●
●●
●
●
●
●
●
●
● ●●
●●
●
●
●●●
●
●●
●
●
●●
●
●
●●
●
●
● ●
●
●
●
● ●
●
●
●●
●●
● ●
●
●●
●
●
●
●● ●
●● ●●
●
●●
●
●
●
●
●●●●
●
●
●
●
● ●
● ●
●
●
●
●
● ●
●
● ●
●
●
●
● ●
●
●
●
●●●
●
●
●●●●
●
●
●● ●
Point size is proportional to turnout
Mul
tiple
col
umns
5th pctl of simulated frequency with which distance between lastand penultimate digit is greater one, minus empirical frequency
1020
5010
020
0
−.6
−.5
−.4
−.3
−.2
−.10.1
●
● ●●
●● ●● ●
●●
●
●
●
●
●●
●
●
●●● ●●
●
●●
●
●
●
●
●●●
●●
● ●
●●
●
●
● ●●
● ● ● ● ●●●
●
● ●
●●
●
●
●
●● ●
●
●●
●
●
●●
●
●
●●
● ●
●● ●●● ●● ●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●●● ●
●
● ●
●
●
●
● ●
●
●●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
● ●
●● ●
●●
●
●●●
●
● ●●
●
●
●
●●●
●●
●●
●●
●
●●
●
●
● ●●
●
●●●
●
●
Fig
ure
12:
Las
tan
dpe
nult
imat
edi
gits
22
return sheet (PDP votes, number of registered voters, and total vote count), we can identify 17wards in which pairs of non-neighboring digits occur suspiciously infrequently.
How can we interpret the fact that pairs of adjacent numerals are sufficiently rare, digit rep-etition is as common as it should be, and yet we find a good number of return sheets on whichpairs of non-adjacent numerals are lacking? We argue that this is, to some extent, a reflection ofthe statistical power associated with each measure. Even for relatively small wards, the expectednumber of pairs of non-adjacent digits is fairly large, at least in comparison to the expected numberof pairs of repeated or adjacent digits, and so the 95% confidence bound is relatively unforgiving.Small wards need to lack digit repetition or exhibit adjacency to a very substantial degree in orderfor us to be confident enough to reject the null hypothesis of a fair election, while a less extremeinsufficiency of non-neighboring digits could push the ward beyond the 95% confidence bound.
5 Conclusion
This paper derived and applied a method to detect manipulation of electoral return sheets. Weshowed that we can expect the last digits of electoral results to occur with equal frequency givena wide range of distributional assumptions, and we then emphasized the fact that humans tend tobe biased in the production of random numbers: They tend to select small digits, avoid repetition,and favor adjacent numerals. If we find that digit patterns deviate from our theoretical expectationin a way that reflects these biases, we suspect that a return sheet has been manipulated. We useddata from Sweden and Nigeria to show that our approach is sensitive to fraud but produces a nullresult in a non-fraudulent environment.
We see two avenues in particular for future research. First, we hope to broaden our analysis toinclude additional countries in order to show the applicability of our method beyond the Nigeriancontext. Second, we plan to develop additional measures in an attempt to validate the approachpresented here. The fact that this method is implementable in data-poor environments is anadvantage, but it also poses a challenge as we try to verify that we are in fact detecting what weclaim to be measuring.
6 Appendix
Proof of Proposition 1. For convenience, let 0 denote the last digit of s1. If the last digit of X isdistributed uniformly, the difference in density with which different numerals occur must on averagebe zero. Formally,
s2−bb∑
a=s1b
(f(ab + d1)− f(ab + d2)) = 0 ∀d1, d2 ∈ {0, . . . , b− 1}. (2)
If g can be approximated linearly over consecutive intervals of size b, each starting at somea ∈ { s1
b , . . . , s2−bb }, we have
23
g(ab + d) = g(ab) + kad , and so (3)
g(ab) + kab = g((a + 1)b) (4)
for any d ∈ {0, . . . , b−1}, with g(ab) constant over the given interval and ka denoting the linearcoefficient for that interval.
From (1) and (3) it follows that
f(ab + d) =∫ ab+d+1
ab+dg(x) dx
=∫ ab+1
abg(x) dx +
∫ ab+d+1
ab+1g(x) dx
= f(ab) + (g(ab) + kax)|ab+d+1ab+1
= f(ab) + kad. (5)
Using (5), we can rewrite (2) as
s2−bb∑
a=s1b
(f(ab) + kad1 − f(ab)− kad2) = 0
⇔ (d1 − d2)
s2−bb∑
a=s1b
ka = 0 (6)
It now remains to be shown that∑ s2−b
b
a=s1b
ka = 0.Recall from (4) that we can write
g(s2) = g(s1) + b
s2−bb∑
a=s1b
ka
Since g(s1) = g(s2) and b > 0, this implies∑ s2−b
b
a=s1b
ka = 0.
Proof of Proposition 2. Recall that proposition 1 holds if equation (6) is true. Given probabilitydensity f(ab) + kad + fe(ab + d), and recalling equation (2), we rewrite (6) as
24
(d1 − d2)
s2−bb∑
a=s1b
(ka + fe(ab + d1)− fe(ab + d2)) = 0
⇔
s2−bb∑
a=s1b
fe(ab + d1) =
s2−bb∑
a=s1b
fe(ab + d2). (7)
Proof of Corollary 3. Suppose to the contrary that proposition 1 holds if d is additively separablefrom fe. Then (7) can be written as
s2−bb∑
a=s1b
fe(ab + d1) =
s2−bb∑
a=s1b
fe(ab + d2)
⇔ he(d1) = he(d2),
which is not true if he(d) is not constant over all d ∈ {0, . . . , b − 1}. Similarly we can show thatproposition 1 does not hold if d is multiplicatively separable from fe.
Proof of Proposition 3. Consider any sequence {z, . . . , z+2(b−1)}, where z ∈ {s1, . . . , s2−2(b−1)}.Let this sequence be denoted q, and let Q denote the set of all such sequences of size 2b − 1 onthe domain of f . We can approximate f in this sequence by arithmetic progression, which yieldsf(z +d′) = f(z) +kzd
′+ fe(z +d′), where kz is the common difference of successive elements of thesequence, fe is some function that gives the error in approximation, and d′ ∈ {0, . . . , 2(b−1)}. Sincewe want to assess the average relative densities with which last digits appear across all sequences ofsize b, let’s average f across all sequences of size b inside q. There are b unique sequences of size b
wholly contained in {z, . . . , z+2(b−1)}. Note that each last digit d ∈ {0, . . . , b−1} appears exactlyonce in each sequence of size b, each number z + d appears in d + 1 sequences, and correspondinglyeach number z + b + d that is contained in q appears in b− (d + 1) sequences. We can then writethe sum of weighted densities for numbers ending in d (i.e. the numbers z + d and z + b + d) as
(d + 1)f(z + d) + (b− (d + 1))f(z + b + d)
= (d + 1)(f(z) + kzd + fe(z + d)) + (b− (d + 1))(f(z) + kz(b + d) + fe(z + b + d))
= f(z) + kz(b2 − b) + (d + 1)fe(z + d)− (b− (d + 1))fe(z + d + b).
In expectation we have E[fe(z+d)] = 0, and so by taking expectations we are left with E[f(z)]+kz(b2 − b). Note that this density is not a function of d, i.e. in expectation it is identical forall d ∈ {0, . . . , b − 1}. Thus last digits of the random variable X ′ are uniformly distributed in
25
expectation, where X ′ has probability density f(x) weighted by the probability with which x
is included in an arbitrary sequence of length b in q. In other words, we have shown that the(unnormalized) density function f(x)h(x, q) produces a uniform distribution of last digits, whereh(x, q) gives the probability that number x is included in any sequence of size b in q. It remains tobe shown that
∑q∈Q h(x, q) is proportional to a constant (i.e. does not vary with x), or equivalently,
that∑
q∈Q f(x)h(x, q) can be normalized to f(x).Function h(x, q) is clearly not constant within the sequence {z, . . . , z + 2(b − 1)}, since the
number of sequences of size b that include x varies with the position of x relative to z. But thereare 2b − 1 sequences in Q that include x, and x is in a different position relative to z in each ofthese sequences. For any x ∈ {s1 + 2(b− 1), . . . , s2 − 2(b− 1)}, summing over Q then yields
∑q∈Q
f(x)h(x, q) = f(x)∑q∈Q
h(x, q)
= f(x)
(b−1∑d=0
(d + 1) +b−1∑d=0
(b− (d + 1))
)
= f(x)b2b−1∑d=0
((d + 1)− (d + 1))
= f(x)b2
∝ f(x).
This leaves x ∈ {s1, . . . , s1 +2b−3; s2−2b+3, . . . , s2}, that is x at the boundaries of the domainof f . For x at the lower bound, we have
∑q∈Q
h(x, q) =x−s1∑d=0
(d + 1) for x ∈ {s1, . . . , s1 + b− 1}, and
∑q∈Q
h(x, q) =b−1∑d=0
(d + 1) +x−(s1+b−1)∑
d=0
(b− (d + 1)) for x ∈ {s1 + b, . . . , s1 + 2b− 3},
where the sum of h(x, q) over all elements of Q varies with x. This follows equivalently for x atthe upper bound.
Hence we can normalize∑
q∈Q f(x)h(x, q) to f(x) only if f(x) = 0 for x ∈ {s1, . . . , s1 + 2b −3; s2 − 2b + 3, . . . , s2}. In other words, the density attributed to x at the upper and lower boundsof the domain determines the extent to which f(x) is different from the (normalized) density∑
q∈Q f(x)h(x, q) and thus the extent to which last digits may follow a non-uniform distribution.For the relevant density at the lower bound of x we have
26
f(s1) + . . . + f(s1 + 2b− 3) =f(s1) + f(s1 + 2b− 3)
2(2b− 2)
= (b− 1)(f(s1) + f(s1 + 2b− 3))
Similarly we can compute the density over x ∈ {s2 − 2b + 3, . . . , s2}. It follows that as
(b− 1)(f(s1) + f(s1 + 2b− 3) + f(s2 − 2b + 3) + f(s2))→ 0
or, less generally, as f(x) approaches 0 for x ≤ s1 + 2b− 3 and x ≥ s2 − 2b + 3, the last digitsof random variable X approach a uniform distribution.
27
References
Philip J. Boland and Kevin Hutchinson. Student selection of random digits. Statistician, 49(4):519–529, 2000.
Alphonse Chapanis. Random-number guessing behavior. American Psychologist, 8:332, 1953.
European Union Election Observation Mission. Nigeria 2003 elections: Final report, 2003.
Darren Kew. The 2003 elections: Hardly credible, but acceptable. In Robert I. Rotberg, editor,Crafting the New Nigeria: Strengthening the Nation. Boulder: Lynne Rienner, 2004.
Darren Kew. Democrazy: Dem go craze, o: Monitoring the 1999 elections. Issue: A Journal ofOpinion, 27(1):29–33, 1999.
Walter R. Mebane, Jr. Election forensics: Vote counts and Benford’s law. Working paper, 2006.
Walter R. Mebane, Jr. and Jasjeet S. Sekhon. Robust estimation and outlier detection for overdis-persed multinomial models of count data. American Journal of Political Science, 48(2):392–411,2004.
Mikhail Myagkov, Peter Ordeshook, and Dimitry Shakin. Fraud or fairytales: Russia and Ukraine’selectoral experience. Post-Soviet Affairs, 21(2):91–131, 2005.
Raymond S. Nickerson. Perception and production of randomness. Psychological Review, 109(2):330–357, 2002.
Mark J. Nigrini. I’ve got your number: How a mathematical phenomenon can help CPAs uncoverfraud and other irregularities. Journal of Accountancy, 187(5), 1999.
Gustave J. Rath. Randomization by humans. American Journal of Psychology, 79(1):979–103,1966.
Satod Cartographic Consultants. Satod street guide and administrative centers, Jos, Plateau state,2003.
Christin Schafer, Jorg-Peter Schrapler, Klaus-Robert Muller, and Gert Wagner. Automatic iden-tification of faked and fraudulent interviews by two different methods. German Institute forEconomic Research Discussion Paper, 2004.
Amos Tversky and Daniel Kahneman. Subjective probability: A judgment of representativeness.Cognitive Psychology, 3:430–454, 1972.
Jonathan N. Wand, Kenneth W. Shotts, Jasjeet S. Sekhon, Walter R. Mebane, Jr., Michael C.Herron, and Henry E. Brady. The butterfly did it: The aberrant vote for Buchanan in PalmBeach County, Florida. American Political Science Review, 95(4):793–810, 2001.
28