detecting voter fraud in an electronic voting context: an ...detecting voter fraud in an electronic...
TRANSCRIPT
Detecting Voter Fraud in an Electronic Voting Context:An Analysis of the Unlimited Reelection Vote in Venezuela
Ines Levin∗ Gabe A. Cohn† Peter C. Ordeshook‡ R. Michael Alvarez‡
June 23, 2009
AbstractBetween December 2007 and February 2009, Venezue-lans participated twice in constitutional referenda wherethe elimination of presidential term limits was one ofthe most salient proposals. Assuming voter preferencesdid not change significantly during that period, the ‘re-peated’ character of these elections provide us with an ex-cellent opportunity to apply forensic tools designed to de-tect anomalies and outliers in election returns in electionswhere electronic voting technologies were used. Similartools were first applied by Myagkov et al. ([20], [21],[22], [23]) to the study of electoral fraud in Russia andUkraine, and were effective in the isolation of potentialcases of manipulation of electoral returns. The case ofVenezuela is different because there exists no widespreadagreement about the integrity or otherwise fraudulent na-ture of national elections, and because it is a nation whereelectronic voting technologies are used. Unless electoralfraud takes place in exactly the same manner in each elec-tion, an analysis of the ‘flow of votes’ between electionscan be used to detect suspicious patterns in electoral re-turns. Although we do not find evidence of pervasive elec-toral fraud compared, for instance, to the Russian case,our analysis is useful to detect polling places or regionsdeviating considerably from the more general pattern.
1 IntroductionHaving an electoral process with a high degree of integrityis important for the maintenance of a well-functioningrepresentative democracy. All stakeholders must believethat an election was free from fraud and malfeasance in or-der for the regime that takes power after a contested elec-tion to have legitimacy. Elections that lack integrity often
∗Graduate student, Division of Humanities and Social Sciences, Cal-ifornia Institute of Technology, Pasadena, CA 91125, MC 228-77; cor-responding author. E-mail:[email protected]. Tel:(626)395-4289†Undergraduate student, Division of Engineering & Applied Science,
California Institute of Technology, Pasadena, CA 91125.‡Professor of Political Science, Division of Humanities and Social
Sciences, California Institute of Technology, Pasadena, CA 91125.
lead to internal political conflict, and to external politicalpressures. An example of such an election occurred in2004 in the Ukraine, where electoral fraud produced mas-sive protests as well as pressure by foreign governments,resulting in an eventual revote.
While examples of clear and widespread electoral fraudare relatively rare, concerns have arisen in recent yearsregarding the detection of fraud as new voting technolo-gies and procedure are tested and deployed. These elec-toral innovations often involve relatively untested tech-nologies and procedures, which sometimes appear to in-teract to produce potentially problematic outcomes. Thismay have been the case in Florida’s 13th Congressionaldistrict election in 2006, when a very high undervote ratewas observed in parts of the district in Sarasota County,where electronic voting machines were used. While re-cent research focuses on how the voting machines dis-played the ballot for this race, considerable debate aroseabout whether there were problems with the electronic de-vices themselves (see Frisina, Herron, Honaker and Lewis[12]).
The need to assure the integrity of an electoral pro-cess has produced a variety of calls for better post-electionanalysis of election administration, sparking a number ofnew research efforts regarding official post-election audit-ing procedures.1 Our research is closely related to theseefforts, as we study the issue of post-election statisticaldetection of election anomalies and outliers. Of course,the statistical analysis of election returns and other elec-tion statistics has a long history, with scholars attemptingto identify statistically the systematic factors impactingoutcomes. Only recently, though, have scholars begun toapply their substantive and methodological tools to looknot for the systematic explanatory factors of voting andelection outcomes, but also for outliers and anomalies notaccounted for by a classical statistical model.2
In this paper we discuss some of these new statisticaltools for post-election forensics, with an application to re-
1See, for example, Aslam, Popa and Rivest [4] or Hall [14], and thereferences therein.
2See Part Three of Alvarez, Hall and Hyde [2] for discussion of manyof these different statistical tools.
1
cent elections in Venezuela. We use the Venezuelan casefor a number of reasons: Recent elections have been im-portant and concern widely-followed referenda on impor-tant constitutional and political issues; we want to applythese statistical tools to a nation where electronic votingis used; finally, while there have been studies of electionfraud in a number of Latin American nations, the tech-niques we present here have not been applied to LatinAmerican cases.
This essay proceeds as follows. In the next section wediscuss briefly the research literature on election fraud,and the cross-cutting research literature on electronic vot-ing. We then turn to a discussion of the Venezuelan case,as well as a general presentation of post-election statisticalforensic analyses. Thereafter we examine the results fromrecent Venezuelan elections, and conclude with a discus-sion of the merits of these post-election statistical forensictools for fraud detection and routine auditing of electionadministration practices.
2 Election Fraud and ElectronicVoting
Despite the frequent use of the term “election fraud”, thereis little guidance in the research literature about exactlywhat constitutes “election fraud.” For example, Alvarez,Hall and Hyde [2] note that notions of fraudulent elec-toral practices vary across time and political jurisdictions,and that behaviors that in one nation at a particular timeare thought of as fraudulent may be perfectly legal andacceptable in other nations at other points in time. TheUnited States Election Assistance Commission (USEAC)recently recognized this problem in a report on “electionfraud”, noting that it has“learned that these terms meanmany things to many different people” ([29], page 11);as a consequence, the USEAC used the term “electioncrimes” in their study instead of “election fraud.”
Regardless of how one defines “election fraud” acrossnations or time, there are clear cases where deliberate andillegal electoral manipulation have been observed. LatinAmerica, our region of focus, has experienced allegationsof election fraud in the past, and there has been substan-tial research on these allegations in some Latin Ameri-can cases. Two Latin American nations in particular havebeen widely studied. One is Costa Rica, which was ex-amined closely by Lehoucq and Molina [17]. Lookingacross nearly fifty years of Costa Rican history (made pos-sible in part by that country’s longstanding democracy),they found that election fraud varied depending on po-litical competition, political institutions, and sociologicaltrends. Another Latin American case that has seen sub-stantial research is Mexico, where during the PRI’s pe-riod of one-party rule, that party’s tactics included the
manipulation of voter registries, multiple voting, ballotbox stuffing and other forms of voter intimidation (Law-son [18]). Others have studied the transition from PRI-dominance to competitive elections, with a focus on howpolitical institutions like electoral courts can help mitigatethe manipulation of elections (Eisenstadt [10]). Predomi-nantly, though, research on election fraud in Latin Amer-ica focuses on systems employing “traditional” technolo-gies, usually paper-based voting registration and ballot-ing systems. As the examples from Costa Rica and Mex-ico demonstrate, paper-based registration and ballot sys-tems are not fraud-proof, instead they have many differenttypes of vulnerabilities that have been exploited in pastelections.
Missing from the literature are studies of the poten-tial for election fraud that arise from the use of elec-tronic voter registration and balloting systems (e.g., Al-varez and Hall [1]). In the United States, a widespreadmovement toward the greater use of electronic voting fol-lowing the 2000 presidential election has proceed with fitsand starts, and at this moment in time has stalled or evenreversed itself (Alvarez and Hall [1]). Much of the de-bate about electronic voting systems in the United Statesconcerns security and reliability; specifically, allegationsthat malicious tampering with the software and hardwareis possible and difficult to detect (e.g., Kohno et al. [16]).These allegations have been the focus of substantial popu-lar press, and a growing body of research (State of AlaskaDivision of Elections [27],[28]; California Secretary ofState [7]; Ohio Secretary of State [24]).
The electronic voting system currently used inVenezuela was first introduced for the 2004 presidentialrecall referendum, and has since been employed in fivenational elections — three constitutional referenda, onepresidential election, and one parliamentary election. Inaddition, it is used in regional and municipal races. Thetouch-screen system is manufactured by Smartmatic, andis an example of an electronic voting device commonlyknown as a “direct-recording electronic” (DRE) votingmachine. The Carter Center [8] issued an observationreport of the 2006 presidential election that contains agreat deal of technical discussion of the electronic vot-ing system used by Venezuela, and we draw heavily fromthat report as it provides one of the best English-languagesources about the technical details of the voting system inVenezuela.
The Carter Center [8] analysis covers in detail many ofthe security features of the Smartmatic voting devices; infact, they devote an entire section of their report to a dis-cussion of the security features of these voting systems.Some of those features include encryption of voting infor-mation, randomization of information to deter the recon-struction of the sequence of voting, disabling unnecessaryphysical access ports, chain-of-custody procedures, and
2
the security of the system’s paper receipts. Overall, theCarter Center report positively evaluated the voting sys-tem security, though it did recommend stronger physicalsecurity and chain-of-custody measures for future elec-tions. The report specifically concluded that “The sys-tem used in Venezuela needs to be secured (and audited)it its entirety” ([8], page 29). However, following the2006 presidential election, the Carter Center report noteda number of issues associated with the audit and it’s pro-cedures ([8], pages 40-42).
Thus, the research reported in the sections that followoffers another set of post-election tools for election of-ficials, stakeholders, and other interested parties to use,and that thereby expands the more limited post-electionaudit procedures employed thus far in recent Venezuelannational elections. A post-election audit of only a smallrandom sample of voting machines from polling stationsmay uncover some problems with the administration ortechnology used in an election; the post-election forensictools we employ here are complementary to the randomaudit, as they can help discover other anomalies or prob-lems with the conduct of a particular election. The ad-vantage of our tools, on the other hand, is that they aredesigned to operate over the complete set of election re-turns.
3 Previous Research
The forensic tools applied here to detect election fraudusing aggregate official election statistics were initiallydeveloped to study election fraud in post-Soviet states,notably Russia and Ukraine (Myagkov, Ordeshook andShakin [21], [22], [23]). Briefly, those tools seek to dis-cover the anomalies in the data occasioned by explicitforms of fraud, such as ballot box stuffing and falsifiedelection protocols that introduce specific types of hetero-geneity into the data. Indeed, the search for ‘artificially’created heterogeneity both defines our methodology andestablishes the prerequisites for its application.
We know that election returns from any national elec-tion exhibits their own natural forms of heterogeneity.Both turnout and a candidate or party’s support will, gen-erally and quite naturally, vary with demographic param-eters such as urbanization, income levels, ethnic compo-sition and even simply region. Our tools, then, seek tocontrol for these parameters and then to look for other,potentially suspicious, sources. Of course, whether wedeem these other sources suspicious depends on what weknow about the election at hand. For instance, does theelection involve a ‘favorite son’ who can be expected towholly legitimately garner special support in some re-gions but not in others? Did the election concern an is-sue that had previously been unimportant — an issue that
divided the electorate differently than before? Naturally,a concern with such things requires that in the search forsuspicious heterogeneity, we must also look across elec-tions so that we can answer questions such as ‘Did a pat-tern suddenly emerge in the data that cannot be explainedby demographic shifts?’
In contending with these substantive issues along withthe various methodological issues associated with thetreatment of aggregate data, Russia and Ukraine proveda useful laboratory for the development and testing of ourforensic tools. Briefly, in Russia, Myagkov, Ordeshookand Shakin [23] had reasonably good priors as to wherespecifically election fraud was most likely to arise — no-tably in its ethnic republics, such as Tatarstan, Bahkor-tostan, Dagestan and Ingueshtia. Each region is controlledby political bosses (republic presidents) who generallywin elections with numbers mimicking a Soviet past; bothturnout and support for the incumbent in excess of ninetypercent. In addition, it is not unusual to find subregions(rayons) in which the majority of precincts report 100 per-cent turnout with 100 percent of the vote going to the re-gional or national incumbent or to the Kremlin sanctionedparty. Data from these regions should readily signal fraudand should distinguish themselves from data from otherregions.
Ukraine, in turn, offered Myagkov et al. [23] nearlythe perfect social science experiment in 2004 from theperspective of testing their forensic tools. Followingan inconclusive first round presidential ballot in Octo-ber, the November runoff pitted two candidates, ViktorYushchenko and Viktor Yanukovich, with sharply differ-ing ideologies and bases of support. Yushchenko’s sup-port, with his pro-NATO and reformist agenda, camelargely from Western Ukraine, whereas Yanukovich’ssupport cam from the more industrialized and largely pro-Russian East (along with Crimea). Yanukovich also re-ceived the strong support of the incumbent regime alongwith Russia’s Vladimir Putin.
The second November runoff election, however, wasmarred by massive and readily documented irregulari-ties — irregularities of such an extent that, in additionto a massive uprising among the population termed ‘TheOrange Revolution’ that saw upwards of a half millionUkrainians permanently camped out in protest in Kyiv’scentral square, Ukraine’s Supreme Court invalidated theresult and called for a new runoff in December. As aconsequence of the Court’s decision and the self-evidentnature of the fraud (along with, doubtlessly, behind thescenes diplomatic maneuvering), Yanukovich lost the en-dorsement of then president Kuchma (thereby signal-ing to regional political bosses that ‘special efforts’ onYanukovich’s behalf were no longer necessary), and sev-eral thousand election observers from Ukraine’s diaspora(notably, from Canada and the United States) poured into
3
the country to monitor the re-runoff. Ukraine, then, pre-sented the authors with a situation in which they had twoelections, one month apart, contested by the same twocandidates, with the same issues, among the same elec-torate but with considerably fewer opportunities and in-centives for fraud in the second runoff. And since theyhad good priors as to where fraud was most blatant in theNovember runoff, they could test their indicators with theexpectation that those indicators would signal fraud in onecase but not in the other and that the measures they pro-vided of the magnitude of fraud should correspond to theestimates offered by a variety of objective observers andjournalistic accounts. Suffice it to say, those indicatorsperformed as designed and suggested the presence of be-tween 2 and 3 million ‘suspect’ votes in the November2004 Ukrainian presidential runoff.3
4 The Case of Venezuela
We started the Venezuela analysis without strong priorsabout the types and location of potential election anoma-lies or potential fraud, or even expectations about whetherthere were problems in recent elections. The electoralcontext in this country is considerably different from thatdescribed in the Russian and Ukrainian cases. First, in thethree most recent national elections, there are few pollingplaces with 100 percent turnout where the official posi-tion, or any other alternative, wins close to 100 percentof the vote. Even in regions with strong support for thegovernment, the opposition wins a ‘reasonable’ positiveshare of the vote. Second, international observers such asthe European Union Election Observation Mission (EU-EOM) positively evaluated recent electoral processes. Forexample, in their report on the 2006 presidential contest,the EUEOM congratulated the National Elections Coun-cil (CNE), as well political actors and social movements,for “creating conditions to hold elections that are accept-able to all stakeholders” and argued that “the electronicvoting system established in Venezuela is efficient, se-cure, and auditable” (EUEOM [11], page 2). Election ob-servers pointed to minor irregularities and suggested im-provements to the electoral system, but they did not raiseserious allegations of electoral malfeasance. Although lo-cal organizations — such as SUMATE, as well as the op-position — made great efforts to monitor and report inci-dents of fraud, there exists no proof of widespread irreg-ularities in recent elections. The most notable case of anelection where Chavez’s position won a disproportionateshare of the vote is the 2005 parliamentary contest, wherethe Movimiento V Repblica (MVR) obtained 116 out of
3Similar tools were applied to the 2004 Russian presidential election,2007 Russian parliamentary election, and 2007 Ukrainian election (seeMyagkov et al. [22], [23]).
167 seats in Venezuela’s National Assembly. This lop-sided victory occurred, however, because the oppositionboycotted the election, alleging there were not enoughguarantees for a free and fair vote. Unsurprisingly, theopposition’s participation and vote share was small andoverall turnout only reached 25.3 percent.
Some of the main opposition arguments about the non-democratic character of the government relate to the con-stitutionality of the articles included in the 2007 and 2009constitutional referenda. The 2007 referendum consistedof, among other things, the removal of presidential termlimits, abolishing the autonomy of the central bank, grant-ing the president control over international currency re-serves, expropriation of large land estates, and reducingthe work day from eight to six hours. People opposedto Chavez’s ‘reforms’ argued this last provision was in-troduced merely as a sweetener to attract votes. In De-cember 2007, citizens voted on two bundles of constitu-tional changes, one of them (bundle A) containing all thereforms originally proposed by Chavez, including the un-limited reelection proposal. Paradoxically, it was the op-position boycott of the 2005 election that gave Chavez anoverwhelming majority in the National Assembly and al-lowed Chavez to readily include his favored proposals inthe 2007 referendum. Still, Chavez’s position lost by asmall margin, which shows that the Venezuelan case dif-fers significantly from the Russian or Ukrainian cases pre-viously summarized.
After the 2007 defeat, Chavez accepted the outcome butpublicly warned that this was just a temporary defeat —he said “no pudimos, por ahora”, meaning “for the mo-ment, we couldn’t” making allusion to the phrase he usedwhen surrendering after the 1992 attempted coup d’tat.After the referendum, banners appeared throughout Cara-cas showing the phrase “por ahora” (“for the moment”).Moreover, while the constitution does not allow a sim-ilar constitutional reform to be put to a vote twice dur-ing the same National Assembly period, Chavez came upwith a new proposal which excluded most of the 2007 re-forms, but still included the unlimited reelection proposal,together with the elimination of reelection terms for otherpublic offices, and managed to get approval from the Con-stitutional Court for holding a new referendum in Febru-ary 2009. In this context, it is reasonable to wonder towhich extent the integrity of the electoral process wouldbe preserved in 2009.
The data about electoral returns we use here to addressthis issue comes from two sources. First, we downloadedthe 2007 and 2009 referendum results from the CNE web-page using a Python script to store the data in a spread-sheet. Later, we found it convenient to compare the refer-endum outcomes with those from the 2006 presidentialelection. The data corresponding to this last race wasdownloaded from the ESDATA website, which contains
4
spreadsheets with official election results at the lowestlevel of aggregation available.4 In addition, in differentsections of our analysis we classify polling places, or con-trol for socio-demographic differences, using census in-formation. This data was downloaded from the web pageof the National Statistics Institute (INE).5
5 MethodologyBriefly, our forensic indicators fall into three categories.The first looks for anomalies in the distribution of turnoutand the relationship between turnout and a candidate orparty’s reported share of the eligible electorate, the sec-ond looks for anomalies in the patterns of the numbers —the digits — employed in official protocols, and the lastcategory applies various econometric techniques to esti-mate the flow of votes between elections.
With respect to the first category, suppose we in facthave a homogeneous data set (e.g., polling stations orprecincts) in the sense that turnout varies within it forpurely random reasons, for reasons having little if any-thing to do with demographic parameters or with thosefactors that impact a candidate or party’s level of support.In this case the overall distribution of turnout across thedata should be approximately normally (and in particu-lar, unimodally) distributed, with some districts reportinghigh turnout, some reporting low turnout, but the over-whelming majority reporting turnout rates at or near theaverage. Now, however, suppose we introduce a specificform of heterogeneity into the data by literally stuffingthe ballot boxes of a subset of precincts with votes for aspecific candidate. The turnout distribution of that sub-set will be moved to the right so as to create an elongatedtail for the overall distribution. And if we stuff enoughballots into those boxes, we can actually create an over-all bimodal distribution. Bimodal distributions of turnout,then, are one of the ‘red flags’ we can use when lookingfor fraudulently cast votes.
Next, suppose we look at the relationship betweenturnout and a candidate’s share of the eligible electorate.Again, if turnout is unrelated to a party’s relative support,that relationship (i.e., the slope of the line relating turnout,T, to share of the eligible electorate, V/E) should approx-imately equal the party’s overall share of the vote in oursample of the data. In other words, suppose turnout in-creases by 100 people. Then if turnout is unrelated to theparty’s support relative to the opposition, that party shouldenjoy a share of those 100 additional voters equal approx-
4The ESDATA web site is http://www.esdata.info/venezuela. Tocheck the accuracy of the data available in this location, we comparedtheir 2009 referendum database, with the one we constructed with thedata downloaded from the CNE, and results were 100% consistent.
5The INE webpage is www.ine.gov.ve, and the section we down-loaded the data from is called “Sıntesis Estadıstica Estatal 2008.”
imately to its general share of the vote. Surely it shouldnot gain more than 100 votes, nor should it experience anyloss in votes. Thus, regression estimates of the relation-ship between V/E and T in homogeneous data should fallin the interval [0,1], so once again, regression estimatesoutside of this interval serve as another indicator of po-tentially fraudulently reported votes.
The second category of forensic indicators examinespatterns in digits in official election returns (Berber andScacco [5], Shpilkin [26]).6 Suppose for example thatelection protocols are filled out with little or no regardfor ballots actually cast. And suppose, moreover, as isactually the case in places such as Russia, that there arefew penalties for committing fraud, provided only that thefraud benefits the incumbent regime. In this case we canreadily imagine a heuristic for filling out protocols — say,officially reported turnout — in which numbers are simplyrounded off to 0 or 5. Looking at the distribution of lastdigits, then, can serve as an additional piece of forensicevidence. But suppose those who would commit fraud at-tempt to be more sophisticated and deliberately avoid theover-use of 0’s and 5’s. So consider instead the last twodigits of official tabulations. It is an experimentally veri-fied fact that if we ask people to write sequences of ran-dom numbers, they will tend to write down paired num-bers (e.g., 2 2 or 3 3 or 7 7) less frequently than we wouldactually expect in a purely random sequence (Chapanis[9], Rath [25], Boland and Hutchinson [6]). A fourth ‘redflag’, then, is a distribution of last and next-to-last digitsthat departs significantly from what we would expect froma purely random process.
An additional forensic indicator entails estimating the‘flow of votes’ from one election to the next. Here, ofcourse, we are attempting to estimate where the votesof one party or candidate (or position on a referendum)went to in successive elections with the expectation thatno party or candidate should win more than 100% of thevote of some earlier candidate or party. Needless to say,treating aggregate data so as to obtain estimates of thissort encounter any number of problems associated withecological correlation. Nevertheless, a variety of econo-metric techniques have been developed for treating suchissues, and their application constitutes our final forensicindicator. Specifically, we estimate Goodman regressionsof the following form:
y1i = β11x1i + β12x2i + β13(1− x1i − x2i) (1)y2i = β21x1i + β22x2i + β23(1− x1i − x2i) (2)
6There are other scholars who also study patterns in digits to detectelectoral fraud. For instance, Mebane [19] looks for deviations fromthe ‘second-digit Benford’s law’. In this paper, we follow a differentapproach, by searching for patterns of non-random behavior in the dis-tribution of the last-two digits.
5
y3i = β31x1i + β32x2i + β33(1− x1i − x2i) (3)
where each yi represents a share of the eligible elec-torate we want to explain - yes, no and abstention pro-portions in the case of referenda, and the xi’s indicate theproportions of people voting for the government’s favoritealternative, for the opposition’s favorite alternative, or ab-staining in a previous election. Each β coefficient repre-sents the proportion of a previous vote share that ‘flows’to each alternative in the most recent election. Therefore,these coefficients should lie between 0 and 1. Further, as-suming no change in the electorate, any voter must eitherabstain, or vote for one of the available alternatives. Thus,coefficients lying in a similar column — i.e., those associ-ated with the same xi, should add to one across the threeequations. If this holds, the nine β coefficients should addto three, predicted yi’s should lie between zero and one,and the sum of the predicted yi’s should add to one.
Still, since we do not use individual level data, aggre-gation bias may lead to estimated coefficients that are outof bounds. The usual solution applied in the literature hasbeen to use an estimation method restricting coefficientsto lie within ‘theoretically prescribed’ bounds (King [15]).Myagkov et al. [23], however, argue that electoral fraudmay lead to negative or above-one coefficients even inthe absence of aggregation issues, so restricting the co-efficients is not desirable because it does not allow usto detect cases of electoral fraud. Still, to correctly as-sess the cause of an out-of-bounds coefficient, we need totake steps to mitigate the ecological inference problem.Myagkov et al. [23] use a semi-parametric procedure,which consists of estimating the model within clusters ofsimilar regions, and then constructing a weighted averageof the coefficients to obtain the overall results. In this pa-per, we employ a method with a similar aim. Specifically,we estimate Goodman regressions with random effects,where coefficients are allowed to vary by region, or tovary by levels of census variables.7 This procedure canbe thought of as the intermediate step between a completepooling approach where the model is estimated at the na-tional level without random effects, and a no-pooling ap-proach where the model is estimated independently withineach region.
Finally, an additional method we use to detect anoma-lies in electoral results follows a test done by Alvarez andKatz [3]. The authors first fit a model to the 1998 gov-ernor and senate election results in the state of Georgia,using the results of the 1996 presidential election as ex-planatory variables. Then, they use the estimated coef-ficients to predict the 2002 election results. In that way,they seek to detect if outcomes deviate significantly fromexpectations. In the case of Venezuela we first fit the yesshare of the vote in the 2007 referendum, using the pro-
7Estimations were done by MLE, using the lme4 R package.
portions of the eligible electorate abstaining or support-ing Chavez and Rosales in the 2006 presidential electionas explanatory variables — controlling for a small set ofsocio-demographic variables, and then use the estimatedcoefficients and 2007 electoral results to predict the yesshare of the vote in the 2009 referendum. The idea is tocheck whether a subset of the polling places deviates con-siderably from our forecast.
Of course, if fraud occurs in both elections in preciselythe same way and in the same localities, this method willnot detect anomalies. This fact, though, merely servesto emphasize that our forensic indicators, taken togetheror separately, are precisely that — indicators. None ofthem, separately or together, can be said to ‘prove’ theexistence or absence of fraud. Much as in a criminal in-vestigation, they merely provide evidence, out of whichwe, as analysts, most form a coherent view of an election— a view that is consistent with that evidence. There isno black box into which we plug official election returnsand out of which comes a determination as to whether anelection was or was not tainted by significant fraud. Ourindicators provide us with alternative ways of looking atofficial election returns and must, necessarily, be com-bined with some substantive expertise about the societyand election at hand. Only the substantive expert can an-swer such questions as: Were adequate controls in placefor heterogeneity in the population; are there alternativeexplanations for why, for instance, increases in turnoutbenefited only one candidate or party; did fraud take moresubtle forms so as to be undetectable by our indicators;did issues arise in one election but not in others that givethat election a distinctly different character; or did onecandidate or another enjoy administrative advantages thatoccasion suspicious patterns in the data but that cannotbe classified as illegal fraud? Only after these questionsare asked and answered can we begin drawing definitiveconclusions from our forensic indicators.
6 Results
Most of the analysis done for individual elections wasdone using ‘mesa’ (table) level data. However, since thenumber of tables changes within one same polling placefrom election to election, the cross-election analysis wasdone comparing polling-place level results. Specifically,we used polling place IDs — constant across elections, toexactly match 8,815 polling places which were open dur-ing the three elections of interest. The average number ofvoters and eligible voters per polling place in 2009 was1,230 and 1,744, respectively.
It was very important for our analysis to count with2007 turnout figures. Still, this information is not avail-able at the CNE’s web site. To solve this issue, we esti-
6
mated polling place level registration numbers for Decem-ber 2007 by interpolating between the December 2006and February 2009 official registration figures for eachpolling place. Turnout rates were computed by taking thetotal number of votes — including both valid and invalidvotes, as a percentage of the total number of registeredvoters. According to the news reports, turnout in the 2007constitutional referendum was approximately 56 percent.Based on our own estimations, turnout in that election sit-uated at 58 percent — with average and median figures of57 percent and 59 percent, respectively. These figures aresubstantively smaller than the 75 percent and 70 percentofficially reported for the 2006 presidential and 2009 con-stitutional referendums, respectively — or 74 percent and69 percent in our sample of matched polling places. Fig-ure 1 compares the distribution of turnout across the threeelections. It shows that turnout was largest and had a rela-tively low variance in 2006, then it decreased considerablyin 2007, and finally increased in 2009. The only slightanomaly we observed in this completely pooled data issome thickness in the right tail of the 2006 turnout distri-bution.
Figure 1: Distribution of Turnout in Recent Elections
The official result of the 2006 presidential election wasa 64.8 percent Chavez victory, relative to the total num-ber of valid votes. If instead we consider the proportionof the eligible electorate, Chavez vote share was equal to45.6 percent. Then, in the 2007 referendum, the no optionwon the election corresponding to both bundles of con-stitutional articles. Specifically, the no vote share stoodat 50.7 percent in the case of bundle A. If we considerthe proportion of the eligible electorate, the vote share re-ceived by the no option — in bundle A, was equal to 29.7percent. Last, the yes option won the 2009 referendumwith a 54.9 percent share of the vote — or 37.4 percent interms of the eligible electorate. Figure 2 shows the resultsof the 2006 presidential election, and the 2007 and 2009
constitutional referendums, using ternary plots. The uppervertices correspond to abstention, the lower left verticescorrespond to the alternative supported by the Chavismo,and the lower right vertices correspond to the alternativesupported by the opposition. Any point within these tri-angles indicates an election outcome — where the pro-portions associated with the different alternatives add to1. To interpret these figures, notice that standing at anyvertex, lines parallel to the opposite side represent similarvote shares associated with the vertex’s label. Also, voteshares decrease linearly as we move away from the vertex.Since the centers of the triangles are equidistant from allextremes, they represent a situation with the exact sameproportion of votes for each alternative.8 These figuresshow that while the 2009 distribution of outcomes is verysimilar in shape to the one observed in the 2006 presiden-tial election, the 2007 constitutional referendum outcomelooks like a special case of relatively low turnout.
Figure 3 shows the distribution of turnout by state. Ingeneral, state-level distributions follow the national pat-tern observed in Figure 1. The most noticeable deviationsfrom the normal distribution occur in the 2006 presiden-tial election — consistent with the thick right tail observedin the distribution of national turnout, with some statesexhibiting relatively large second modes to the right — inparticular, this happens in Cojedes and Vargas. This laststate shows slightly multimodal distributions in all threeelections. Then, in 2007, there is a set of states whoseturnout distribution does not look normal, but slightlyskewed and with thick tails — we refer to Bolivar, DeltaAmacuro, Guarico and Zulia. Finally, in the case ofthe 2009 referendum, most densities look unimodal andtightly clustered around some point, and are not skewedto any side — although Barinas and Tachira exhibit smallhills to the left, and Vargas to the right.
As mentioned in the previous section, both alternativesshould benefit partially from an increase in turnout — or,at least, none should get hurt. A failure of this hypoth-esis raises a ‘red flag’ about artificially inflated turnoutfavoring one of the alternatives. To test it we plot the re-lationship between turnout and the shares of the vote foreach alternative, by state, for sufficiently similar pollingplaces. The criteria for constructing subsets of similarpolling places by state, were splitting the data in two, de-pending on whether the yes option won the previous elec-tion in that polling place (see Figure 4), or whether theopposite outcome took place (see Figure 5). In the caseof those regions where the yes won in 2007, we observeslightly negative slopes in the no shares in the states ofCojedes and Delta Amacuro. Also, in those regions where
8Actually, the abstention percentage represented in the ternary plotsincludes both actual abstention as well as invalid vote. In the case of the2006 presidential election and 2009 constitutional referendum, invalidvotes stood at 1.0% and 1.2%, respectively.
7
Figure 2: Recent Election Outcomes
the no won in 2007, we observe negative yes slopes in sev-eral states — Bolivar, Carabobo, Cojedes, Lara, Merida,Portuguesa, Trujillo and Yaracuy. The most extreme casesare Carabobo and Cojedes, where the yes share falls con-siderably as turnout increases. In general, observed fig-ures for the 2009 election are mostly consistent with ourhypothesis. A similar analysis of the 2007 election resultsis also consistent with the hypothesis that both shares ofthe eligible electorate benefit partially from an increasein turnout. However, we do observe deviations from ourexpectation in some states, which warns about potentialelectoral fraud in some of the regions under considera-tion. Since our hypothesis rests on the assumption of ho-mogeneity across observations — i.e. observations whichonly differ in turnout, deviations may also be due to het-erogeneity not accounted for by our bivariate regressionsand scatter plots.
Next, we proceeded to analyze the patterns of digits ofthe ‘mesa’ (table) level electoral results. Figure 6 showshistograms with the density of the last digit for each alter-native in the three elections of interest — the 2006 presi-dential election, the 2007 constitutional referendum (bun-dle A), and the 2009 constitutional referendum. In addi-tion, we also include the density of the last digit corre-sponding to the outcomes of bundle B of the 2007 con-stitutional referendum. We do not observe any obviousnon-random pattern — such as 0/5 rounding, or avoidanceof zeros and fives. All last digits have approximately thesame 10 percent incidence. Also, Figure 7 shows matrixplots of the density of the last two digits. Each cell corre-sponds to a combination of a next-to-last (row) digit, anda last (column) digit, with darker tones indicating largerincidence. If individuals unconsciously avoid paired num-bers such as 2 2, 3 3, and so on, then we should observewhite diagonals. However, this pattern does not show upin any occasion — and neither does the opposite patternwhere people focus on 2 2, 3 3, and so on. In general, noneof the matrices shows an obvious non-random pattern inthe distribution of last two digits. A shortcoming of thistool is that electronic fraud may be designed in such a waythat manipulation of the last digits is completely random.If this were the case, this particular indicator would not beuseful to detect fraud.
After that, we turn to the analysis of the flow of votesbetween different elections. Figure 8 shows the propor-tion of the yes vote in the eligible electorate in 2009, as afunction of the same variable in 2007. We observe thatthe proportion of support for the yes increases in mostpolling places, especially where the yes vote was smallin 2007. Even though most polling places are clusteredaround the regression line, some observations are far off.Since deviations may be due to the aggregations of resultsfrom dissimilar regions, we built Figure 9 showing com-parable graphs by state. Again, state-level results show
8
patterns similar to those observed at the national level,though outcomes are more tightly clustered around the fit-ted regression line. Some states, such as Aragua, DeltaAmacuro, and Distrito Capital exhibit small clusters ofpolling places where the proportion of yes votes increasedconsiderably in 2009 relative to most voting centers.
To further learn about the flow of votes between elec-tions, we estimated a set of Goodman equations similarto the one represented by equations (1) through (3). Ta-ble 1 shows the results with complete pooling, where eachequation is estimated by OLS. The model fits the data well— with each equation explaining more than 95% of thevariance in the dependent variable. Still, three of the co-efficients lie out of bounds. To mitigate the potential ag-gregation bias, we re-estimated similar equations specify-ing random coefficients by region. In total, we considered366 regions — which in most states correspond to munic-ipalities, except for the Distrito Capital and Vargas, whereour regions correspond to ‘parroquias.’ In this case, westill find that two of the average coefficients are negative,but closer to the [0,1] interval compared to the completepooling approach. The main result following from this ta-ble is that voters who supported the yes option in 2007,overwhelmingly support the yes option in 2009 — this isnot surprising because the 2007 reform proposal was evenmore radical than the 2009 reform proposal. Similarly,those voters who supported the no alternative in 2007,overwhelmingly choose the no alternative in 2009. Noneof these results is surprising.
The most interesting result corresponds to the flow ofvotes of from those who abstained in 2007. Suppose allof the increase in turnout observed between both electionswas artificial participation created by the government toinflate the yes results. Then, the coefficient in the up-per right corner should be relatively large. However, weobserve a different pattern. Approximately 56 percent ofthose who abstained in 2007 continue to abstain in 2009.And among those who choose to participate, two thirdssupport the yes position, and one third support the no po-sition. Thus, even though the yes alternative was the mainbeneficiary of the increase in turnout, the no alternativealso received a fairly large proportion of the new vote.Also, the random coefficients model is useful because itallows us to detect regions deviating significantly from av-erage estimates. Figure 10 shows that the random effectsassociated with some of the regions were significantly dif-ferent from the average effects. For instance, in the Ped-ernales municipality of Delta Amacuro, the proportion ofthe 2007 yes share received by the yes alternative in 2009is equal to 146 percent, very out of bounds, and way abovethe estimates from any of the remaining regions. Also,in the case of Acevedo municipality in the state of Mi-randa, the proportion of the 2007 no vote share receivedby the no alternative is equal to 111%, an out-of-bounds
and relatively large estimate in comparison to other re-gions. Table 3 shows the results of a similar estimationwith non-nested random effects by average age, genderand proportion of rural population, for the first equation.According to the deviance criterion — a measure of er-ror — the model with random coefficients by region fit-ted the data better than this second model. However, thenon-nested alternative allows us to learn about the deter-minants of the effects. For instance, according to panel (c)in Figure 11, the proportion of the 2009 yes vote originat-ing from those abstaining or voting yes in 2007, decreaseswith the proportion of rural population, while the oppositehappens with the proportion received from those voting noin 2007.
Table 1: Flow of Votes between 2007 and 2009Complete pooling, OLS estimation
Yes 07 No 07 Abst 07Yes 09 0.99 -0.09 0.27No 09 -0.15 1.02 0.13Abst 09 0.14 0.05 0.58Vertical sum 0.98 0.98 0.98
Table 2: Flow of Votes between 2007 and 2009Random effects by region, M.L.E.
Yes 07 No 07 Abst 07Yes 09 0.97 -0.08 0.28No 09 -0.11 0.94 0.14Abst 09 0.13 0.10 0.56Vertical sum 1.00 0.96 0.98
Table 3: Flow of Votes between 2007 and 2009Random effects by demographic indicators, M.L.E.
Yes 07 No 07 Abst 07Yes 09 1.00 -0.07 0.27No 09 -0.15 0.97 0.16Abst 09 0.13 0.09 0.56Vertical sum 0.98 0.99 0.99
Table 4 shows the results of a Goodman regressionanalysis with random coefficients by region, to explainthe flow of votes between the 2006 presidential electionand the 2009 constitutional referendum. The main resultis that a substantial proportion (20 percent) of those whovoted for Chavez in 2006 abstained in the 2009 referen-dum. Also, 94 percent of those who voted for Rosales in2006 choose the no alternative in the 2009 constitutionalreferendum, and 11 percent abstain. Second, 72 percent ofthose who abstained in 2006, abstain again in 2009, whilethe rest (20 percent), support the yes alternative. Table 5
9
shows the results of a similar analysis to explain the flowof votes between the 2006 and 2007 elections. In this case,we find that 36 percent of those who voted for Chavez in2006, decide to abstain in 2007. Also, among those votingfor Rosales in 2006, 87 percent vote no in 2007, while 19percent abstain. Last, 77 percent of those who abstainedin 2006, also do in 2007, while 16 percent support the yesalternative.
Table 4: Flow of Votes between 2006 and 2009Random effects by region, M.L.E.
Chavez 06 Rosales 06 Abst 06Yes 09 0.74 -0.09 0.24No 09 0.04 0.94 0.04Abst 09 0.20 0.11 0.72Vertical sum 0.98 0.96 1.00
Table 5: Flow of Votes between 2006 and 2007Random effects by region, M.L.E.
Chavez 06 Rosales 06 Abst 06Yes 07 0.60 -0.08 0.16No 07 0.03 0.87 0.08Abst 07 0.36 0.19 0.77Vertical sum 0.99 0.98 1.01
Our final indicator corresponds to a comparison of theobserved yes vote share in 2009, with a forecast of the yesvote share for 2009. Predictions were computed using theestimated coefficients of a model first fitted to the 2007election results — with predictors including the 2006 out-come, plus a set of socio-demographic control variables,which explained 98 percent of the variance in the 2007yes vote share. Figure 13 shows the result of this com-parison. Most of the polling places exhibit larger thanexpected vote shares, but we do not observe any group ofpolling places located away from the rest of the data —most points are clustered around a line with slope close toone and positive intercept. The model did worst amongthose polling places where the yes position was predictedto obtain between 60 percent and 80 percent of the vote— instead, many of those locations exhibited a yes voteshare between 80 percent and 100 percent.
7 ConclusionsIn this paper, we used a set of statistical tools to detectoutliers and anomalies in election returns in two recentconstitutional referendums in Venezuela where electronicvoting technologies were used. Studying the integrity ofthese elections is challenging because manipulation of thesoftware used for electronic voting could lead to types of
fraud difficult to detect with some of our forensic indica-tors. Still, the elimination of the presidential term lim-its stood out as one of the main proposals in both con-stitutional referendums. Assuming there were no consid-erable socio-demographic changes in the population be-tween both elections, it would be suspicious if the 2009share received by the yes alternative changed dramaticallyin polling places where the same position lost the electionin 2007. Even with a well designed electronic fraud, con-siderable changes between both elections may be detectedwith a careful analysis of the flow of votes.
Our main observation is that most of the new votesreceived by the yes position came from polling placeswith large abstention in 2007. Still, a third of those whoabstained in the previous election participated and sup-ported the no position, implying that not all the increasein turnout was artificially created by the government toget the reform approved. Further, the new distribution ofturnout, as well as the election outcome, was very simi-lar to that observed in the 2006 presidential election, sug-gesting the 2009 election was one where turnout recov-ered and people voted relatively similar to how they didin 2006. Of course, if fraud took place in 2006 and it wascarried out in the same way as in 2009, then we wouldnot be able to detect it using our flow of votes analysis.But this begs the question of why would the governmentreplicate the same type of manipulation in 2009, and notin 2007? If they were capable of successfully conductingfraud in 2006, they could also have used it to approve theradical reforms proposed in 2007.
Even though we did not find evidence of widespreadelectoral fraud taking place in the last referendum, we didobserve anomalies and outliers in different steps of ourstudy. For instance, the state of Delta Amacuro showedboth anomalies in the turnout distribution, as well as inthe relationship between vote shares and turnout. Further,a municipality in the same state was prominent for hav-ing a very large and out of bounds coefficient associatedwith the share of the yes vote in 2007 ‘flowing’ to theyes vote in 2009. However, this is no proof that electoralfraud took place in Delta Amacuro. As we warned in themethodological section, our indicators do not constitute,by themselves, proof that electoral fraud did or did nottake place in any particular region, or in the country as awhole. They should be interpreted by experts with knowl-edge about the characteristics and heterogeneities of thedifferent areas, and employed as a complement to the in-formation arising from pre- and post-election audits, aswell as reports from election observers.
10
References[1] Alvarez, R.M. and T.E. Hall. 2008. Electronic elec-
tions: the perils and promises of digital democracy.Princeton University Press.
[2] Alvarez, R.M., T.E. Hall, and S.D. Hyde, editors.2008. Election Fraud: Detecting and DeterringElectoral Manipulation. Washington, DC: Brook-ings Institution Press, pages 149-61.
[3] Alvarez, R.M. and J.N. Katz. 2008. Detecting Elec-toral Fraud: The Case of 2002 General Election inGeorgia. In R.M. Alvarez, T.E. Hall and S.D. Hyde,editors. Election Fraud: Detecting and DeterringElectoral Manipulation. Washington, D.C.: Brook-ings Institution Press, pages 162-81.
[4] Aslam, J.A., R.A. Popa and R. L. Rivest. 2008.On Auditing Elections When Precincts Have Differ-ent Sizes. Proceedings, 2008 USENIX/ACCURATEElectronic Voting Technology Workshop.
[5] Berber, B. and A. Scacco. 2008. What the numberssay: A digit based test for election fraud using newdata from Nigeria. Paper Presented at the AnnualMeeting of the APSA, Boston, Mass.
[6] Boland, P.J. and K. Hutchinson. 2000. Student Selec-tion of Random Digits. Statistician 49(4): 519-29.
[7] California Secretary of State. 2007. Top-to-Bottom Review. http://www.sos.ca.gov/elections/elections_vsr.htm (June 22,2009).
[8] Carter Center. 2007. Observing the 2006 Presiden-tial Elections in Venezuela. Final Report of the Tech-nical Mission.
[9] Chapanis, A. 1953. Random-Number Guessing Be-havior. American Psychologist 8:332.
[10] Eisenstadt, T.A. 2002. Measuring electoral courtfailure in democratizing Mexico. International polit-ical science review 23(1): 47-68.
[11] European Union Election Observation Mis-sion. 2006. Final Report: Presidential ElectionsVenezuela 2006.
[12] Frisina, L, M.C. Herron, J Honaker, J.B. Lewis.2008. Ballot formats, touchscreens, and Under-votes: A Study of the 2006 Midterm Elections inFlorida. Election Law Journal, 7(1): 25-47.
[13] Gelman, A., and J. Hill. 2007. Data Analysis Us-ing Regression and Multilevel/Hierarchical Models.New York: Cambridge University Press.
[14] Hall, J.L. 2008. Improving the Security, Trans-parency, and Efficiency of California’s 1%Manual Tally Procedures. Proceedings, 2008USENIX/ACCURATE Electronic Voting Technol-ogy Workshop.
[15] King, G. 1997. A solution to the ecological inferenceproblem. Princeton University Press.
[16] Kohno, T. A. Stubblefield, A.D. Rubin and D.S.Wallach, 2004. Analysis of an electronic voting sys-tem. IEEE Symposium on Security and Privacy.
[17] Lehoucq, F.E. and I. Molina Jimnez. 2002. Stuffingthe ballot box: fraud, electoral reform, and democ-ratization in Costa Rica. Cambridge: CambridgeUniversity Press.
[18] Lawson, J.C.H. 2000. Mexico’s Unfinished Transi-tion: Democratization and Authoritarian Enclaves.Mexican Studies/Estudios Mexicanos, 16 (2): 267-87.
[19] Mebane, W.R., Jr. 2008. Election Forensics: TheSecond-Digit Benfords Law Test and Recent Amer-ican Presidential Elections. In R.M. Alvarez, T.E.Hall and S.D. Hyde, editors, Election Fraud: Detect-ing and Deterring Electoral Manipulation. Wash-ington, D.C.: Brookings Institution Press, pages162-81.
[20] Myagkov, M, P.C. Ordeshook and D. Shakin. 2005.Fraud or Fairytales: Russia and Ukraine’s ElectoralExperience. Post-Soviet Affairs, 21 (2):91-131.
[21] Myagkov, M, P.C. Ordeshook and D. Shakin.2007. The Disappearance of Fraud: The Forensicsof Ukraine’s 2006 Parliamentary Elections. Post-Soviet Affairs, 23(3): 218-39.
[22] Myagkov, M, P.C. Ordeshook and D. Shakin. 2008.On the Trail of Fraud: Estimating the Flow of Votesbetween Russia’s Elections In R.M. Alvarez, T.E.Hall and S.D. Hyde, editors. Election Fraud: Detect-ing and Deterring Electoral Manipulation. Wash-ington, D.C.: Brookings Institution Press, pages182-200.
[23] Myagkov, M, P.C. Ordeshook and D. Shakin. 2009.The Forensics of Election Fraud: Russia andUkraine. New York: Cambridge University Press.
[24] Ohio Secretary of State. 2007. EVEREST: Evalua-tion and Validation of Election-Related Equipment,Standards and Testing. http://www.sos.state.oh.us/SOS/upload/everest/14-AcademicFinalEVERESTReport.pdf(June 22, 2009).
11
[25] Rath, G.J. 1966. Randomization by Humans. Amer-ican Journal of Psychology 79(1): 97-103.
[26] Shpilkin, S. 2008. Analysis of Russian Elections asreported in TimesOnline, April 18. See also http://freakonomics.blogs.nytimes.com/2008/04/16/russian-election-fraud/(June 22, 2009).
[27] State of Alaska Division of Elections. 2007. Stateof Alaska Election Security Project Phase 1 Report.http://www.elections.alaska.gov/Documents/SOA_Election_Security_Project_Phase_1_Report_Final.pdf(June 22, 2009).
[28] State of Alaska Division of Elections. 2008. Stateof Alaska Election Security Project Phase 2 Report.http://www.elections.alaska.gov/Documents/SOA_Election_Security_Project_Phase_2_Report.pdf (June 22,2009).
[29] United States Election Assistance Commis-sion. 2006. Election Crimes: An Initial Re-view and Recommendations for Future Study.http://www.eac.gov/program-areas/research-resources-and-reports/copy_of_docs/reports-and-surveys-2006electioncrimes.pdf/attachment_download/file (June 22,2009).
12
AppendicesA Tables and Figures
13
Figu
re3:
Dis
trib
utio
nof
rurn
outi
nre
cent
elec
tions
.T
hebl
ue,r
edan
dgr
een
dens
ities
corr
espo
ndto
turn
outi
nth
e20
06pr
esid
entia
lele
ctio
n,20
07co
nstit
utio
nal
refe
rend
uman
d20
09co
nstit
utio
nalr
efer
endu
m,r
espe
ctiv
ely.
14
Figu
re4:
Rel
atio
nshi
pbe
twee
ntu
rnou
t(ho
rizo
ntal
axis
)and
vote
shar
esof
elig
ible
elec
tora
te(v
ertic
alax
is)i
npo
lling
plac
esw
ithvi
ctor
yof
the
yes
alte
rnat
ive
in20
07.B
lue
dots
indi
cate
vote
shar
esfo
rthe
yes
optio
n,an
dgr
een
dots
indi
cate
vote
shar
esfo
rthe
noop
tion.
Lin
esco
rres
pond
tofit
ted
biva
riat
elin
earr
egre
ssio
nsbe
twee
ntu
rnou
tand
vote
shar
es.
15
Figu
re5:
Rel
atio
nshi
pbe
twee
ntu
rnou
t(ho
rizo
ntal
axis
)an
dvo
tesh
ares
ofel
igib
leel
ecto
rate
(ver
tical
axis
)in
polli
ngpl
aces
with
vict
ory
ofth
eno
alte
rnat
ive
in20
07.B
lue
dots
indi
cate
vote
shar
esfo
rthe
yes
optio
n,an
dgr
een
dots
indi
cate
vote
shar
esfo
rthe
noop
tion.
Lin
esco
rres
pond
tofit
ted
biva
riat
elin
earr
egre
ssio
nsbe
twee
ntu
rnou
tand
vote
shar
es.
16
37
Fig
ure
6:
Dis
trib
uti
on o
f L
ast
Dig
it
Cha
vez
06
Ros
ales
06
Yes
09
No
09
Yes
07
(bu
ndle
A)
Yes
07
(bu
ndle
B)
No
07 (
bun
dle
A)
No
07 (
bun
dle
B)
Figu
re6:
Dis
trib
utio
nof
the
last
digi
t
17
38
F
igu
re 7
: D
istr
ibu
tion
of
Las
t T
wo
Dig
its
Not
e: I
n ea
ch f
igur
e, t
he h
oriz
onta
l di
men
sion
cor
resp
onds
to
the
last
dig
it, a
nd t
he v
erti
cal
dim
ensi
on c
orre
spon
ds t
o th
e ne
xt t
o la
st d
igit
. Eac
h ce
ll r
epre
sent
s th
e de
nsity
of
the
last
two
digi
ts, w
ith
dark
er to
nes
indi
cati
ng la
rger
inci
denc
e.
Cha
vez
06
Ros
ales
06
Yes
09
No
09
Yes
07
(bu
ndle
A)
Yes
07
(bu
ndle
B)
No
07 (
bun
dle
A)
No
07 (
bun
dle
B)
Figu
re7:
Dis
trib
utio
nof
last
two
digi
ts.I
nea
chfig
ure,
the
hori
zont
aldi
men
sion
corr
espo
nds
toth
ela
stdi
git,
and
the
vert
ical
dim
ensi
onco
rres
pond
sto
the
next
tola
stdi
git.
Eac
hce
llre
pres
ents
the
dens
ityof
the
last
two
digi
ts,w
ithda
rker
tone
sin
dica
ting
larg
erin
cide
nce.
18
Figure 8: Relationship between yes shares in 2009 and yes shares in 2007. The red line represents the fitted resultsof a linear regression between the proportion of yes votes in the 2007 constitutional referendum (bundle A), andthe proportion of yes votes in the 2009 constitutional referendum. The green line represents a situation where theproportion of yes votes in both elections is the same (45◦ line).
19
Figu
re9:
Rel
atio
nshi
pbe
twee
nye
ssh
ares
in20
09an
dye
ssh
ares
in20
07.
The
red
line
repr
esen
tsth
efit
ted
resu
ltsof
alin
ear
regr
essi
onbe
twee
nth
epr
opor
tion
ofye
svo
tes
inth
e20
07co
nstit
utio
nalr
efer
endu
m(b
undl
eA
),an
dth
epr
opor
tion
ofye
svo
tes
inth
e20
09co
nstit
utio
nalr
efer
endu
m.
The
gree
nlin
ere
pres
ents
asi
tuat
ion
whe
reth
epr
opor
tion
ofye
svo
tes
inbo
thel
ectio
nsis
the
sam
e(4
5◦lin
e).
20
41
Fig
ure
10:
Goo
dm
an R
egre
ssio
n C
oeff
icie
nts
by
Mu
nic
ipal
ity8
(dev
iati
ons
from
mea
n)
8 I
n th
e ca
se o
f th
e C
apit
al D
istr
ict a
nd V
arga
s st
ate,
reg
ions
cor
resp
ond
to “
parr
oqui
as”.
Figu
re10
:Goo
dman
regr
essi
onco
effic
ient
sby
regi
on(d
evia
tions
from
the
mea
n),w
ithde
pend
entv
aria
ble
equa
lto
the
yes
prop
ortio
nof
the
elig
ible
.
21
42
Figure 11: Goodman Regression Coefficients
(deviations from mean)
a- Random coefficients by age quantiles
b- Random coefficients by gender9
c- Random coefficients by rurality quantiles
9 1 indicates more than 50% female.
Figure 11: Goodman regression coefficients by demographic differences (deviations from the mean), with dependentvariable equal to the yes proportion of the eligible. In the second panel, ‘1’ indicates more than 50% female.
22
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
● ●
●
●
●●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
● ●
●●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●●●
●
●●● ●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●● ●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●
● ●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●●
●●
●
● ●
●●
● ●
●●●
●
●
●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●●
●
● ●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●●●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●●
● ●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
● ●●
●● ●
●●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●● ●●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
● ●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●●●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
● ●
●●
●
●
●●
●●●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●
●●
●
●
●●●
●
●
●
●
●
●●
●●●●●
●
●
● ● ●
●●
●
● ●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●●
●●
●
●●●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●● ●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●
●● ●
●●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●●
●
●●
●●
●●
●
● ●●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●●
●
●
●●●●
●●
●
●
●
●
●●
●●
●
●
●●
●●
●
● ●
●
●●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●● ●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
● ●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●● ●●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●●●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
● ●
●
●
●
●
●●●
●
●●
●
●●
●
●●
●
●
●●
●
● ●
●
●
●
●● ●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
● ●
●
●●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
● ●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●●
●
●
●●
●●
● ●●
●
●●●
●
●
●
●●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
● ●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●● ●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
● ●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●● ●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
● ●●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●●
● ●●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●●
●
●
●
●
●●●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●●
●●
●
●
●●●●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●●●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
● ●
●●●
●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●●
●
●●
●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●
●
●
●●●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
● ●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
●●●
●
●
●
●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
● ●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●●
●●
●
●
●
●●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●
●●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
● ●● ●
●●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
● ●●●●
●
●
●● ●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●● ●
● ●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●●●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●● ●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●● ●
●●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●●
●
● ●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
● ●
●
●
●
● ●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●●
●●
● ●●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●●
●
●
●
●●
●●
● ●
●
●
●
●
●
●
●
●●● ●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●●●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●● ●
●● ●●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
●●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●●
●●●
●●
●●
●
●
●
●●
●●
●
● ●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●●
●
●●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
●●●●
●
●
●
●●
●●
●
●●●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
predicted_yes09
pyes
09_2
Figure 12: Forecasted 2009 results versus actual 2009 results. Dependent variable: yes votes as a proportion of totalvotes. Predictions were made based on the 2007 yes and no shares of the eligible electorate, using the estimatedcoefficients corresponding to an OLS regression where the dependent variable was the 2007 referendum yes share,and the independent variables were the 2006 Chavez and Rosales shares of the eligible electorate, plus a set of socio-demographic variables.
23