Security Analysis of the Estonian Internet Voting System

J. Alex Halderman1 Harri Hursti Jason Kitcat2 Margaret MacAlpineTravis Finkenauer1 Drew Springall1

1 University of Michigan, Ann Arbor, MI, U.S.A.2 Open Rights Group, U.K.

Technical report – May 2014

For additional materials and contact information, visit estoniaevoting.org.

1. INTRODUCTIONSeveral countries have experimented with casting votes

over the Internet, but today, no nation uses Internet voting forbinding political elections to a larger degree than Estonia [38].When Estonia introduced its online voting system in 2005,it became the first country to offer Internet voting nation-ally. Since then, it has used the system in local or nationalelections five times, and during recent elections 20–25% ofparticipating voters cast their ballots online [24].

Nevertheless, the system remains controversial. Many Es-tonians view Internet voting as a source of national pride, butone major political party has repeatedly called for it to beabandoned [29]. Although Estonia’s Internet Voting Com-mittee maintains that the system “is as reliable and secureas voting in [the] traditional way” [18], its security has beenquestioned by a variety of critics, including voices within thecountry (e.g. [43, 46]) and abroad (e.g. [52]).

For these reasons, the Estonian Internet voting (I-voting)system represents a unique and important case study in elec-tion security. Its strengths and weaknesses can inform othercountries considering the adoption of online voting, as wellas the design of future systems in research and practice.

In this study, we evaluate the system’s security using acombination of observational and experimental techniques.We observed operations during the October 2013 local elec-tions, conducted interviews with the system developers andelection officials, assessed the software through source codeinspection and reverse engineering, and performed tests on areproduction of the complete system in our laboratory. Ourfindings suggest the system has serious procedural and ar-chitectural weaknesses that expose Estonia to the risk thatattackers could undetectably alter the outcome of an election.

The weakness of the Estonian system stems from its basicdesign. Most e-voting schemes proposed in recent yearsuse cryptographic techniques to achieve end-to-end (E2E)verifiability [9]. This means that anyone can confirm thatthe ballots have been counted accurately without having totrust that the computers or officials are behaving honestly.In contrast, Estonia’s design implicitly trusts the integrity ofvoters’ computers, server components, and the election staff.

Rather than proving integrity through technical means, Es-tonia relies on a complicated set of procedural controls, butthese procedures are inadequate to achieve security or trans-parency. During our in-person observations and in reviewingofficial videos of the 2013 process, we noted deviations fromprocedure and serious lapses in operational security, whichleave the system open to the possibility of attacks, fraud, anderrors. Transparency measures, such as video recordings andpublished source code, were incomplete and insufficient toallow outsider observers to establish the integrity of results.

The threats facing national elections have shifted signifi-cantly since the Estonian system was designed more than adecade ago. Cyberwarfare, once a largely hypothetical threat,has become a well documented reality [44, 51, 55, 56], andattacks by foreign states are now a credible threat to a na-tional online voting system. Given that Estonia is an EU andNATO member that borders Russia, it should not discountthe possibility that a foreign power would interfere in itselections.

To test the feasibility of such attacks, we reproduced theI-voting system in a lab environment and played the role ofa sophisticated attacker during a mock election. We wereable to develop client-side attacks that silently steal voteson voters’ own computers, bypassing safeguards such as thenational ID smartcard system and smartphone verificationapp. We also demonstrate server-side attacks that target theimplicitly trusted counting server. By introducing malwareinto this server, a foreign power or dishonest insider couldalter votes between decryption and tabulation, shifting resultsin favor of the attacker’s preferred candidate.

We conclude that there are multiple ways that state-levelattackers, sophisticated online criminals, or dishonest insiderscould successfully attack the Estonian I-voting system. Suchan attacker could plausibly change votes, disrupt elections,or cast doubt on the integrity of results. These problems aredifficult to mitigate, because they stem from basic architec-tural choices and fundamental limitations on the security andtransparency that can be provided by procedural controls. Forthese reasons, we recommend that Estonia discontinue theI-voting system.

Figure 1: I-voting client software

2. BACKGROUNDOur observations and analysis focus on the Estonian I-

voting system as it was used for the 2013 municipal elections(“KOV2013”) [20]. In these elections, Internet voting wasavailable for seven days, from October 10–16, and the mainin-person polling took place on election day, October 20.Results were declared that evening. According to officialstatistics, 133,808 votes were cast online, corresponding to21.2% of participating voters [24].

In this section, we review the design and operation of the I-voting system. Figure 2 gives an overview of the interactionsbetween the main system components.

2.1 National ID CardsAn essential building block of the I-voting system is Es-

tonia’s national ID infrastructure [13]. Estonian national IDcards are smartcards with the ability to perform cryptographicfunctions. With the use of card readers and client software,Estonians can authenticate to websites (via TLS client au-thentication [48]) and make legally binding signatures ondocuments [14]. The cards are popularly used for onlinebanking and accessing e-government services [19]. In theI-voting system, voters use their ID cards to authenticate tothe server and to sign their ballots.

Each card contains two RSA key pairs, one for authen-tication and one for digital signatures. Certificates bindingthe public keys to the cardholder’s identity are stored on thecard and in a public LDAP database [15]. The card does notallow exporting private keys, so all cryptographic operationsare performed internally. As an added safeguard, each keyis associated with a PIN code, which must be provided toauthorize every operation.

2.2 I-Voting Server InfrastructureThe majority of the I-voting server source code is published

to a GitHub repository 2–3 weeks prior to the election [28].The server infrastructure is configured in a public ceremonyone week before the election and consists of four machines:

Voting Client

Vote ForwardingServer

Vote StorageServer

Verification App

Vote CountingServer

Log server

DVD

HTTPS(Client Auth) HTTPS

HSM

ElectionPrivate Key

ElectionPublic Key

ElectionPublic Key

QR CodeD

ata

Cen

ter

Vot

er C

ompu

ter

Vot

er S

mar

tpho

ne

Internet

Pub

lic C

ount

NationalID Card

Figure 2: I-voting system overview — Major componentsof the system, and how information flows between them.

Vote forwarding server (VFS/HES) The VFS is the onlypublicly accessible server. It accepts HTTPS connectionsfrom the client software, verifies voter eligibility, and acts asan intermediary to the backend vote storage server, which isnot accessible from the Internet.

Vote storage server (VSS/HTS) The VSS is a backendserver that stores signed, encrypted votes during the onlinevoting period. Upon receiving a vote from the VFS, it con-firms that the vote is formatted correctly and verifies thevoter’s digital signature with the help of an external OCSPserver.

Log server This server is an internal logging and moni-toring platform that collects events and statistics from theVFS and VSS. The source code and design have not beenpublished. While this server is not publicly accessible, it canbe accessed remotely by election staff.

Vote counting server (VCS) The VCS is never connectedto the network and is only used during the final stage ofthe election. Officials use a DVD to copy encrypted votes(with their signatures removed) from the VSS. The VCS isattached to a hardware security module (HSM) that containsthe election private key. It uses the HSM to decrypt the votes,counts them, and outputs the official results.

2

2.3 Voting ProcessesThe I-voting system uses public key cryptography to pro-

vide a digital analog of the “double envelope” ballots oftenused for absentee voting [22]. Conceptually, an outer en-velope (a digital signature) establishes the voter’s identity,while an inner envelope (public key encryption) protects thesecrecy of the ballot. Once each voter’s eligibility has beenestablished, the signature is stripped off, leaving a set ofanonymous encrypted ballots. These are moved to a physi-cally separate machine, which decrypts and counts them.

Voting At the start of each election, the election authoritypublishes a set of voting client applications for Windows,Linux, and Mac OS, which can be downloaded from https://valimised.ee. The client is customized for each election andincludes an election-specific public key.

Figure 3a shows the protocol for casting a vote. To be-gin the process, the voter launches the client applicationand inserts her ID card. She enters the PIN associated withher authentication key, which is used to establish a client-authenticated TLS connection to the election server. Theclient verifies the server’s identity using a hard-coded cer-tificate. The server confirms the voter’s eligibility basedon her public key and returns the list of candidates for herdistrict [11].

The voter selects her choice c and enters her signing keyPIN. The client pads c using OAEP and randomness r, en-crypts it with the election public key, and signs the encryptedvote with the voter’s private key. The signed and encryptedvote is transmitted to the server, which associates it with anunguessable unique token x and returns x to the client. Theclient displays a QR code containing r and x.

As a defense against coercion, voters are allowed to votemultiple times during the online election period, with only thelast vote counted. All earlier votes are revoked but retainedon the storage server for logging purposes. While the votingclient indicates whether the user has previously voted, it doesnot display the number of times. The voter can also overrideher electronic vote by voting in person on election day.

Verification The voter can confirm that her vote was cor-rectly recorded using a smartphone app provided by the elec-tion authority [23, 25, 26], as seen in Figure 4. This protocolis shown in Figure 3b. The app scans the QR code displayedby the voting client to obtain r and x. It sends x to the electionserver, which returns the encrypted vote (but not the signa-ture) as well as a list of possible candidates. The app uses rto encrypt a simulated vote for each possible candidate andcompares the result to the encrypted vote received from theserver. If there is a match, the app displays the correspondingcandidate, which the voter can check against her intendedchoice. The server allows verification to be performed up tothree times and up to 30 minutes after casting.

Tabulation Figure 3c shows the sequence that occurs atthe conclusion of an election. After online voting has ended,the storage server processes the encrypted votes to verify the

Voting Client Election ServersTLS Client Auth

Verify if eligible voterFind set of candidates C

C

Voter picks candidate c ∈Cr←{0,1}160

b← EncPKelect(Padr(c))σ ← SignSKvoter

(b)v := (b,σ)

v

Assign v vote ID xx

Display QR code: (x,r)

(a) Vote casting process

Verification App Election Servers

Scan QR code (x,r)x

Find ballot b with vote ID xFind set of candidates C

b, C

if ∃ c′ s.t. b = EncPKelect(Padr(c′)):Display c′

Voter checks: c′ ?= c

else: Display error

(b) Vote verification process

Storage Server Counting Server

B←{}For each vote v :

(b,σ) := vVerifyPKvoter

(b,σ)B← B∪{b}

B

For each c ∈C :counts[c]← 0

For each b ∈ B :c← DecSKelect(b)counts[c]← counts[c]+1

Output counts

(c) Vote counting process

Figure 3: I-voting system protocols

3

Figure 4: Verification app — A smartphone app allows vot-ers to confirm that their votes were correctly recorded. Wepresent two strategies that an attacker can use to bypass it.

signatures and remove any revoked or invalid votes. Officialsthen export the set of valid votes after stripping off the sig-natures, leaving only anonymous encrypted votes. These areburned to a DVD to transfer them to the counting server.

The counting server is attached to the HSM that containsthe election private key. The server uses the HSM to decrypteach vote and tallies the votes for each candidate. Officialsexport the totals by burning them to a DVD. These results arecombined with the totals from in-person polling stations andpublished as the overall results of the election.

3. OBSERVATIONSThe first part of our analysis follows an observational

methodology. Four of the analysis team (Halderman, Hursti,Kitcat, and MacAlpine) visited Estonia as officially accred-ited election observers during the October 2013 municipalelections and witnessed the operation of the I-voting servers.During that time, they also met with election officials andthe I-voting software developers in Tallinn and Tartu. Later,we closely reviewed published artifacts from the election:the server source code [28], written procedures [16], andnearly 20 hours of official videos that recorded the I-votingconfiguration, administration, and counting processes [17].Ultimately, we identified a range of problems related to poorprocedural controls, lapses in operational security, and insuf-ficient transparency measures.

3.1 Inadequate Procedural ControlsWhile the Internet Voting Committee has published exten-

sive written procedures covering many steps in the electionprocess [16], we observed that some procedures were not con-sistently followed and others were dangerously incomplete.

For example, tamper-evident seals are used on the serverracks in the data center1. Election staff were unsure when1We note that tamper-evident seals like those used in Estonia areeasily compromised [34–37], and their use in elections has beencalled into question in other countries [4].

asked what would happen if the seals were found to be com-promised. Procedures for handling anomalous conditions thatcould imply an attack appear to be inadequately specified ordo not exist.

Anomalous situations that did occur during the 2013 elec-tion were handled in an ad hoc manner, sometimes at thediscretion of a single individual. On multiple occasions, weobserved as data center staff restarted server processes to re-solve technical glitches, and repeated failed commands ratherthan troubleshooting the root cause of the issues. Similarissues were observed during tabulation during poll workers’attempt to boot the vote storage server. The machine reportederrors stating that the drive configuration had changed—apossible indication of tampering. Instead of investigating thecause of the alert, staff bypassed the message.

Some procedures appeared to change several times overthe observation period. For example, observers were initiallyallowed to film and photograph inside the server room, butwere prohibited the next day because of the unsubstantiatedclaim of “possible electronic interference.” In a similarlyabrupt change in procedure, observers were required to leavetheir mobile phones outside the data center after multiple dayswhere this was not the policy. Rewriting the rules on the flysuggests that the procedures had not been adequately thoughtout or were insufficiently defined for staff to implement themconsistently.

Even when procedural safeguards were clear, they werenot always followed. For example, procedure dictates thattwo operators will be present when performing updates andbackups [11]. Second-operator procedures like this are com-monplace in situations, such as voting, where the outcomemust be robust to a single point of failure. On October 14 weobserved that a lone staff member performed these tasks. Thesame staff member arrived with update disks and left withbackup disks. Without a second operator present, the securityof the system relies on the integrity of a single staff member.

3.2 Lax Operational SecuritySince the I-voting system treats parts of the server infras-

tructure as implicitly trusted, the processes used to install andconfigure those servers determine the security of the election.We witnessed numerous serious lapses in operational securityboth during our on-site observations and in the official videosreleased by the Internet Voting Committee.

Pre-election setup Several problems can be seen in theofficial videos of the pre-election setup process, which takesplace in the National Electoral Committee’s offices in Tallinn.

The videos show election workers downloading softwarefor use in the setup process from a public website over anunsecured HTTP connection (Figure 6). They also explicitlydisabled certificate checking while downloading a program,via HTTPS, whose purpose was to verify the integrity of theelection application source code. A network-based man-in-the-middle attacker could have compromised either of theseapplications.

4

In other instances workers unintentionally typed passwordsand PINs in view of the camera. These included personalnational ID card PINs and server root passwords (Figures 7and 8). Similar problems were present during daily main-tenance operations in the data center. Physical keys to theserver room and rack were revealed to observers; these keyscould potentially be duplicated using known techniques [41].

The most alarming operational security weakness duringpre-election setup was workers using an “unclean” personalcomputer to prepare election client software for distribution tothe public. As seen in Figure 5, the desktop has shortcuts foran online gambling site and a BitTorrent client, suggestingthat this was not a specially secured official machine. Ifthe computer used to prepare the client was infected withmalware, malicious code could have spread to voters’ PCs.

Daily maintenance We observed other operational secu-rity weaknesses during daily maintenance operations thattook place during the voting period. The I-voting servers arehosted at a government data center in Tallinn, and workersgo there to perform operations directly from the server con-soles. While there were security video cameras at the datacenter, there appeared to be no 24/7 security personnel norany definitive information on who monitored the cameras.

Standard practice during the daily maintenance operationsis for workers to log in to the election servers under the rootaccount and perform operations at the shell. Logging in asroot is contrary to security best practice, as it simplifiesmany attacks, eliminates user-based privilege separation foroperator functions, and increases the risk of human error.

Unencrypted daily backups were casually transported inworkers’ personal backpacks. DVDs holding updated voterlists from the population register were handled in a similarlycasual way after having been created, we were told, by amember of staff at their own computer. We did not observeany audit trail or checks on the provenance of these DVDs,which were used daily at the heart of the I-voting system.

Tabulation The tabulation process at the end of the elec-tion was also concerning. After the votes were decrypted onthe counting server, an unknown technical glitch preventedworkers from writing the official counts and log files to DVD.Instead, they elected to use a worker’s personal USB stick totransfer the files to an Internet-connected Windows laptop.This USB stick had been previously used and contained otherfiles, as shown in Figure 10. This occurred despite protestfrom an audience member and deviated significantly from thewritten procedure, adding multiple potential attack vectors.Malware present on the laptop could potentially have alteredthe unsigned ballots, or malware on the USB stick could havebeen transferred to the trusted counting server.

These instances illustrate a pattern of operational securitylapses on the part of the workers who operate the I-votingsystem. This is particularly alarming given the high degreeof trust the I-voting system design requires of the electionservers, client software, and the election workers themselves.

3.3 Insufficient TransparencyWhile the I-voting system involves many trusted compo-

nents, we have been impressed by the initial steps towardstransparency implemented by election officials, including al-lowing in-person public observation, publishing videos ofoperator tasks, and releasing large parts of the system sourcecode. However, these measures are incomplete and insuffi-cient to fully establish the integrity of election results.

One limitation is that these measures cannot show whethermalicious actions were performed on the servers or hard disksbefore recording and observation commenced. To illustratethese limitations, we conducted an experimental server-sideattack on a reproduction of the system, as detailed in Sec-tion 5.2. We have published a series of videos2 matchingthe procedures in the official videos step-by-step, but wherethe result of the election is dishonest because of malwaresurreptitiously introduced before the start of the pre-electionsetup process.

The official video records often did not capture everythinggoing on in the facilities. On many occasions, there weremultiple machines or screens in use simultaneously but onlya single camera. On Monday, October 14, we were physicallypresent in the server room when one of the servers producedabnormal output, which appeared to be a failure of the updateoperation. The official video recording was following theother display, and the operator, upon seeing the error message,quickly flushed it from the screen.

Although we attempted to follow these simultaneous op-erations during our in-person observations, on occasion theoperators appeared to be deliberately evading us. In anotherinstance when an error appeared on the server console, anelection worker quickly cleared the display and then asked usto rotate out of the room and let other observers in, allowinghim a block of observation-free time.

An auditor from a major international consulting firm hadbeen hired by the Internet Voting Committee, and his reportalso documents procedural and operational shortcomings [49].However, the auditor’s role was chiefly to observe, and hewas not provided with the access needed to confirm secureoperation of the system.

Election officials have made large parts of the server soft-ware open source [28]. This is a positive measure as it allowsindependent review and assessment—indeed, our study madeextensive use of the code. However, security-critical piecesof code are missing from the published sources, including theentire client application and code that is executed on everyserver machine (see Section 4.1).

Officials told us that the client source is not released (andfurthermore, the client binary is obfuscated) because theyare concerned that attackers might modify it and distributea trojan lookalike. Creating client-side malware is entirelyfeasible without the source, as we show in Section 5.1. Atthe same time, keeping source files secret prevents members

2See https://estoniaevoting.org/videos/.

5

Figure 5: Unsecured build systems — Operators used a PCcontaining other software, including PokerStars.ee, to buildthe official voting client for distribution. This risks infectingthe client with malware spread from the unsecured PC.

Figure 6: Insecure software downloads — Operators down-loaded software over insecure connections for use in pre-election setup. An attacker who injected malicious code intothese downloads might be able to compromise the process.

Figure 7: Keystrokes reveal critical passwords — Videosposted by officials during the election show operators typing,inadvertently revealing critical system passwords.

Figure 8: Video shows national ID PINs — During pre-election setup, someone types the secret PINs for their na-tional ID card in full view of the official video camera.

Figure 9: Posted Wi-Fi credentials — The official video ofthe server setup process reveals Wi-Fi credentials, which havebeen posted on the wall.

Figure 10: Personal USB stick — Against procedures, anofficial used a USB stick, containing personal files, whenmoving the official election results off of the counting server.

6

of the public from understanding what they are being askedto run on their computers, and it increases the risk that anycentrally introduced malicious changes to the official clientwill go undetected.

As these transparency measures are practiced today, thevideos, public observation, and open source components riskproviding a false sense of security. It is plausible for the sys-tems to be corrupted prior to videotaping, for media used toupdate systems to be maliciously modified, or for unpublishedpieces of software to contain errors or malicious code—allinvisible to the public under current transparency measures.

3.4 Vulnerabilities in Published CodeThe published portions of the I-voting server software [28]

contain 17,000 lines of code, with 61% in Python, 37% inC++, and the remainder in shell scripts. The codebase isquite complex, with a large number of external dependencies,and exhaustively searching for vulnerabilities would be achallenging task well beyond the scope of this project. Weunderstand that volunteers from the Estonian security com-munity have already audited it—a testament to the virtues ofpublishing code.

Nonetheless, we uncovered some additional vulnerabilitiesin the process of conducting our other experiments. We havedisclosed these issues to the Internet Voting Committee andwill not release details publicly until after the 2014 election.We do not consider these specific vulnerabilities to be severeproblems, but they are a reminder that open source cannotguarantee the absence of vulnerabilities [40].

4. EXPERIMENTAL METHODOLOGYIn order to further investigate the security of the I-voting

system, we set up a copy of the system in our lab, repro-ducing the software and configuration that was used for theOctober 2013 elections. Compromising a real election wouldhave involved numerous legal and ethical violations, but ourlaboratory setup allowed us to play the role of attacker in ourown mock election without hazard.

4.1 Mock Election SetupTo reproduce the I-voting servers, we used the source code

published on GitHub by the election authority [28]. We setup the servers by following published configuration docu-ments [16] and matching step-for-step the actions performedby election workers in the official videos [17]. As part ofthis process, we generated our own public keys for the webserver’s TLS certificate as well as for the election’s encryp-tion key.

Some components were missing from the published servercode, but we attempted to recreate them as faithfully aspossible. First, the software for the log server, the ivote-monitor package, was not made available; we operated astandard rsyslog [2] server instead. Additionally, there wasno source provided for the evote_post.sh script, whichruns on every server during installation of the packages.

We attempted to reproduce its functionality based on out-put shown during server configuration in the official videos.

In real election, Estonia uses a hardware security mod-ule (HSM) in order to handle the election private key anddecrypt votes. Since we did not have compatible hardwareavailable, we emulated the HSM in software using OpenSSLand Python. Since this is a deviation from the fielded setup,we ensured that none of our attacks depend on vulnerabilitiesin the HSM itself.

Similarly, we set up our own certificate authority andOCSP responder as a stand-in for the ID card PKI. We as-sumed for purposes of this study that those pieces of theinfrastructure are secure.

For the client software, we started with the official votingclient from the 2013 election, which we downloaded fromthe election website [21] in October. For convenience, wefocused on the Linux version of the client. Since the electionpublic key and server certificate are hard-coded into the client,we needed to patch it in order to replace these with the keys ofour mock election and server. Similarly, we used the officialsource code for the Android-based verification app [27] andmodified it to communicate with our server.

Because we did not have access to actual Estonian ID cardsfor testing, we had to virtualize them. By setting up our ownlocal certificate authority, we were able to generate identitiesfor voters and infrastructure for the PKI. We replaced the IDcard on the client with a software-based emulator that speaksthe protocol expected by the voting client application. Onceagain, we ensured that the success of our attacks does not relyon the changes we made; we treated the emulated ID cardsas secure black boxes for purposes of our analysis.

4.2 Threat ModelAfter setting up the mock election, we attempted to com-

promise it, allowing ourselves the resources and capabilitiesof a sophisticated but realistic attacker. This attacker could bea foreign state, a well-funded criminal organization, or a dis-honest election insider. These kinds of attackers are difficultto defend against, but they represent a serious and realisticthreat to modern elections given the enormous financial andpolicy consequences at stake.

Since the time the Estonian system was introduced, cy-berwarfare has become a well documented reality. Chineseespionage against U.S. companies [44], U.S. sabotage ofIran’s nuclear enrichment program [51], and attacks by theU.K. against European telecommunications firms [56] are justa few examples. Estonia itself suffered widespread denial-of-service attacks in 2007 that have been linked to the Rus-sia [55]. An increasing number of nations possess offensivecomputer security capabilities [47], and investment in thesecapabilities is reported to be growing at a significant rate [30].

Such an attacker would have powerful capabilities. Weassume they could obtain a detailed knowledge of the I-votingsystem’s operation, which can be gleaned from publishedsources and reverse engineering (as we did), from insider

7

knowledge, or by compromising systems used by the softwaredevelopers and election officials. We assume that if reverseengineering is required, the attacker has sufficient human andtechnical resources to accomplish this on a short timescale.Such capabilities are assumed to be within the grasp of a stateintelligence agency.

For client-side attacks, we assume that the attacker has theability to deliver malware to voters’ home computers. Thiscould be done externally to the voting system, either by pur-chasing pre-existing criminal resources, such as a botnet, orby buying or discovering zero-day vulnerabilities in popularsoftware. Another route would be to compromise the votingclient before its delivery to voters, either by a dishonest in-sider who can alter the software or by other attackers who cancompromise the computers used to build it. The operationalsecurity problems documented in Section 3.2 suggest thatthis is a practical mode of attack.

5. ATTACKSWe used our reproduction of the I-voting system to exper-

iment with a range of attacks. The I-voting system placessignificant trust in client and server components, makingthese highly attractive targets for an attacker.

While certain server operations are protected by cryptog-raphy (e.g., cast votes cannot be decrypted on the front-endweb server, since it lacks the requisite private key), in otherinstances the servers are completely trusted to perform hon-estly and correctly when handling votes. Similarly, whilethe smartphone verification app gives voters some ability tocheck that the client software is behaving honestly, there aremajor limitations to this safeguard that can be exploited tohide malicious client behavior.

We experimentally verified that these trusted componentsare vulnerable by conducting two sets of demonstration at-tacks against them in our mock election setting. The firsttype are attacks on the client within reach of a financiallycapable attacker, in which an attacker can change votes ina retail manner for large numbers of individual voters. Thesecond kind are server-side attacks within the reach of a well-resourced state-level attacker or dishonest insider, in whichan attacker could change the wholesale results of the entireelection through an exploit on the vote counting server.

5.1 Client-side AttacksThe voter’s client machine is a trusted component of the

I-voting system, so malware on the client could potentiallytamper with the voter’s ballot. There are several ways thatan attacker might try to infect a sufficient number of Esto-nian clients to alter election results in a close race. One isto rent out bots from pre-existing botnets. Botnet operatorsfrequently rent them on the black market, and these can betargeted to a specific country or region [12]. A second waywould be to discover or purchase a zero-day exploit againstpopular software used in Estonia. While this would be expen-sive, it would not be out of reach for a state-level attacker—

Figure 11: Malware records secret PINs — Estonians usetheir national ID smartcards to sign and cast their ballots. Wedeveloped demonstration client-side malware that capturesthe smartcard PINs and silently replaces the user’s vote.

several companies specialize in selling zero-day exploits togovernments [31]. A third strategy would be to infect theofficial I-voting client before it is delivered to voters.

If the attacker’s goal is merely to disrupt the election, farfewer infected system would be required. Estonian law allowselection officials to cancel online balloting if a problem isdetected [32]. In that case, Internet voters would have theirelectronic votes canceled and be required to go to the pollson election day. This is a useful emergency measure, but itrequires election officials to both detect the attack and make asnap decision about its severity. Suppose an attacker infecteda small number of clients with vote-stealing malware, allowedsome of them to be detected, and designed the attack such thatit was difficult to quickly determine the size of the infectedpopulation. Under these circumstances, officials might becompelled to cancel the online portion of the election, yet theattacker would need relatively few resources.

In order to investigate how an attacker could modify votesthrough the client application, we implemented two experi-mental attacks. Both assume that the attacker has used oneof the techniques noted above to initially infect the clientmachine. Each attack uses a different mechanism to defeatthe smartphone verification app (see Section 2.3), which isthe only tool available to voters to detect whether their ballotchoices have been manipulated by client-side malware.

Each attack involves a hidden malicious process that runsalongside the I-voting client application and tampers withits execution. In order to develop them, we first needed toreverse engineer parts of the client software used in 2013.The client is closed source, and the developers took measuresto complicate reverse engineering. The executable is obfus-cated using the UPX packing tool. Strings, public keys, andother resources are hidden by XORing them with the outputof a linear congruential generator. These measures did notsignificantly complicate the construction of our attacks. Weused the UPX application with the -d switch to unpack the

8

binary and used the IDA Pro disassembler and Hex-Raysdecompiler to reverse the portions necessary for our attacks.

Ghost Click Attack In the first attack, we use malwareon the client machine to silently replace the user’s vote witha vote in favor of an attacker-selected candidate. At a highlevel, the malware silently sniffs the victim’s PIN during theiroriginal voting session. The real vote is cast, and everythingappears normal, including the verification smartphone appif the voter uses it. Then, the malware waits until it is toolate to verify again—either until the 30 minute time limit haspassed or until after the user closes the client software andthe QR code can no longer be scanned.

At that point, the malware checks whether the voter’s IDcard is still present in the computer. If so, it opens a copy ofthe I-voting client in a hidden session and, through keystrokesimulation, submits a replacement vote. In the case thatthe ID card has already been removed, the malware remainsdormant until the card is inserted again. Since Estonian IDcards are utilized for a variety of applications, many votersare likely to use their cards again within the one-week onlinevoting period.

In our implementation, the malware attaches to the vot-ing client process and captures PINs by setting breakpointsusing ptrace [1]. Upon reaching a breakpoint, it reads thePIN from the client’s memory and stores it for future use.Although our implementation runs in userspace, a kernel-level rootkit could be used to make it even more difficult todetect [33]. The malware could be extended to sniff PINsopportunistically any time voters use their ID cards, such aswhen logging into a bank.

Bad Verify Attack The Ghost Click attack defeats theverification app, but applying it on a large scale would leadto a suspiciously high number of replacement votes. We alsoexperimented with a stealthier but more complicated style ofattack that targets the verification app directly.

The verification app is premised on the notion that thesmartphone is an independent device that would be unlikely tobe compromised at the same time as the client PC. Therefore,it should be impossible for malware on the PC to change aballot and hide that fact during verification.

However, smartphones today are not well isolated fromusers’ PCs, as there is typically regular communication be-tween the two devices. Users frequently plug their phonesinto their PCs to charge them or to transfer files. User contentis regularly synchronized between devices through GoogleDrive, Dropbox, and other cloud services. Android even al-lows users to remotely install applications on their phonesfrom their PCs through the Google Play Store web inter-face, and there are similar measures for Apple and Microsoftplatforms [45].

As a result of this convergence, there are abundant meansby which malware on the PC could attempt to infect the user’sphone. If the attacker can do this, they can create a dishonestverification app that colludes with malware on the PC to foolthe voter.

To experiment with such an attack, we implemented tan-dem PC and smartphone malware. The PC malware deter-mines which candidate the voter actually selected and modi-fies the QR code so that it encodes the chosen candidate inthe padding randomness. The malicious app behaves justlike the real verification app, except that it displays what-ever candidate is embedded in the QR code, rather than thecandidate for whom the vote was actually cast. This lets thePC malware arbitrarily change the vote that gets submittedwithout being detected by verification or causing a suspiciousnumber of replacement votes.

This form of attack adds complexity, due to the need tocompromise both devices. If used on a large scale, it carriesan elevated possibility of detection, since some users mayattempt verification with devices owned by others. However,it illustrates that as PCs and smartphones continue to con-verge, it will become increasingly unsafe to treat them asindependent devices.

5.2 Server-side AttacksThe integrity of the count depends on the correct operation

of the counting server and its HSM, which are the only com-ponents with the ability to decrypt votes. Similarly, ballot se-crecy depends on the counting server to not leak informationabout the correspondence between encrypted and decryptedvotes. An attacker who injects malicious software into thecounting server can violate these critical security properties.

Although the counting server is not connected to a network,there are a number of other means by which it might beattacked. State-level attackers are known to employ firmware-based malware [5], for instance, which could be used to infectthe BIOS or hard disk before delivery to election officials.Sophisticated attackers are known to target component supplychains and distribution infrastructure [39].

We were informed during the observation mission that theInternet Voting Committee has a bid process every election torent the servers that they use. We are concerned that attackerscould credibly target the machines through this process.

Another route would be to compromise the server softwarebefore it is installed at the beginning of the election. Wepursued this strategy in our experiments.

Injecting malware Despite procedural safeguards [16], anattacker who strikes early enough can introduce maliciouscode into the counting server by using a chain of infectionsthat parallels the configuration process. During pre-electionsetup, workers use a development machine, which is config-ured before setup begins, to burn Debian Linux installationISOs to DVD. These DVDs are later used to configure allelection servers. If the machine used to burn them is com-promised—say, by a dishonest insider, an APT-style attackon the development facility, or espionage—the attacker canleverage this access to compromise election results.

We experimented with a form of this attack to successfullychange results in our mock election setup. We first createda modified Debian ISO containing vote-stealing malware

9

intended to execute on the counting server. The tainted ISO isrepackaged with padding to ensure that it is identical in sizeto the original. In a real attack, this malicious ISO could bedelivered by malware running on the DVD burning computer,by poisoning the mirror it is retrieved from, or by a network-based man-in-the-middle.

Defeating integrity checking During the setup process,election workers check the SHA-256 hash of the ISO fileagainst the SHA256SUMS file downloaded via anonymousFTP from debian.org. Since FTP does not provide cryp-tographic integrity checking, a network-based man-in-the-middle could substitute a hash that matched the maliciousISO. However, this hash would be publicly visible in videosof the setup process and might later arouse suspicion.

An attacker who had compromised the DVD burner com-puter could achieve greater stealth. To demonstrate this, weimplemented a custom rootkit that defeats the hash verifica-tion. Our rootkit is a kernel module that hooks system callsin order to cause the hash verification to succeed and theoriginal ISO’s hash to be printed. Hash checks applied in thisway are only a minor speedbump under our threat model.

Vote-stealing payload After passing the hash check, thetainted ISOs are used to install the OS on all election servers,spreading the infection. When the new OS boots, the malwarechecks whether the machine is configured as the countingserver, in which case it launches a vote-stealing payload.

During the counting process, this payload acts as wrapperaround the process responsible for using the HSM to decryptvotes. As real votes are decrypted, it inspects them on thefly and replaces them with votes for the attacker’s preferredcandidate. (In our demonstration, we change 100% of thevotes, but it would be straightforward to implement a moresubtle algorithm that manipulates an arbitrary fraction.) Thealtered votes are then counted and released as the officialresults. Such as attack would be unlikely to be detected, asthere are no audit mechanisms to check the accuracy of thedecryption.

It took less than two man-weeks to devise, implement, andtest this server-side vote-stealing attack. The majority of thistime was spent finding an appropriate and sufficiently silentway to insert the trojan into the OS installation procedures,due to unfamiliarity with the Debian package system andinstallation process.

We note that this is far from the only means by which anattacker could inject a malicious payload into the servers.Several other pieces of closed-source software of unknownor untrusted provenance could be vehicles for attacks. Theseinclude the evote_post.sh script, missing from the serversource code repository, which runs on all servers, as well asthe driver software for the HSM, which is a closed-sourceapplication manually installed to the counting server from aDVD. These programs all touch critical, trusted portions ofthe I-voting system, yet they are not reviewable by the publicand not integrity checked through any visible procedures.

In addition, although the official videos show that electionworkers build the I-voting server software packages fromthe published code, they first install prebuilt copies of thesepackages from an unknown source in order to retrieve nec-essary dependencies. While the prebuilt packages are laterremoved, if they are malicious, then merely installing themcould trigger execution of attacker code on the supposedly“clean” build machine.

The key lesson from this attack is that it is extremely dif-ficult to ensure the integrity of code running on a criticalsystem, particularly when faced with the possibility of so-phisticated attacks or dishonest insiders. If any element inthe lineage of devices that handle the software installed onthe counting server is compromised, this could jeopardize theintegrity of election results [54].

6. DISCUSSIONThough we have spent the majority of this report discussing

weaknesses and risks, we would be remiss if we failed toacknowledge the great lengths that the I-voting system de-velopers, security staff, and officials go to in their efforts toprotect the election system.

One core strength of the I-voting system is Estonia’s na-tional ID card infrastructure and the cryptographic facilitiesit provides. While the ID cards cannot prevent every impor-tant attack, they do make some kinds of attacks significantlyharder. The cards also provide an elegant solution for remotevoter authentication, something few countries do well.

The Internet Voting Committee’s willingness to releasesource code is a very positive step for transparency. Thisshows confidence in the software’s developers and demon-strates officials’ desire to work with the security communityat large. Providing access to the source allows many partiesto analyze it—not only international researchers like us butalso the domestic security community, who have an evengreater interest in the system’s secure operation. For thesereasons, we urge the committee to go further and release thesource code to the I-voting client and the missing portions ofthe server code discussed in Section 4.1.

Finally, we commend the Internet Voting Committee fortheir dedication to improving the election system. Sinceits inception in 2005, the system has undergone significantchanges. From the switch to a standalone client, to the de-ployment of the log server that enhances forensic and moni-toring capabilities, to the addition of the verification app, theI-voting system has not stood still. Yet as we have argued,even these and an array of other useful safeguards are notenough to secure Estonia’s online elections in the face of adetermined and well-resourced modern attacker.

Opportunities for Innovation While the risks of Internetvoting are clear, the benefits are uncertain. Many Estonianssupport I-voting because they believe there is widespreadfraud in the country’s paper-based system. Whether or notthese concerns are founded, the I-voting system can do littleto help, since nearly 80% of votes are still cast on paper.

10

Fortunately, there are safe and effective ways to apply newtechnology to secure paper-based voting.

In recent years, researchers have developed methods thatcan dramatically increase the security of paper ballots. Sta-tistical risk-limiting audits [7, 8, 42, 53] can minimize therisk of error or fraud during tabulation. Cryptographic tech-niques that achieve end-to-end verifiability [6, 10, 50] enableindividual voters to verify that every vote has been countedaccurately. Estonia has an opportunity to be the first countryin the world to adopt these technologies on a national scale.

7. CONCLUSIONSCompared to other online services like banking and e-

commerce, voting is an exceedingly difficult problem, due tothe need to ensure accurate outcomes while simultaneouslyproviding a strongly secret ballot. When Estonia’s I-votingsystem was conceived in the early 2000s, it was an inno-vative approach to this challenge. However, the designersaccepted certain tradeoffs, including the need to trust the cen-tral servers, concluding that although they could take stepsto reduce these risks through procedural controls, “the fun-damental problem remains to be solved” [3]. More than adecade later, the problem remains unsolved, and those risksare greatly magnified due to the rapid proliferation of state-sponsored attacks.

As we have observed, the procedures Estonia has in placeto guard against attack and ensure transparency offer insuf-ficient protection. Based on our tests, we conclude that astate-level attacker, sophisticated criminal, or dishonest in-sider could defeat both the technological and procedural con-trols in order to manipulate election outcomes. Short of this,there are abundant ways that such an attacker could disruptthe voting process or cast doubt on the legitimacy of results.Given the current geopolitical situation, we cannot discountstate-level attacks targeting the system in future elections.

Due to these risks, we recommend that Estonia discontinueuse of the I-voting system. Certainly, additional protectionscould be added in order to mitigate specific attacks, but at-tempting to stop every credible mode of attack would add anunmanageable degree of complexity. Someday, if there arefundamental advances in computer security, the risk profilemay be more favorable for Internet voting, but we do notbelieve that the I-voting system can be made safe today.

AcknowledgmentsWe thank everyone in Estonia who shared their insights onthe e-voting system during our visit, including Sven Heiberg,Andres Hiie, Priit Kutser, Helger Lipmaa, Tarvi Martens,Arnis Parsovs, Jan Willemson, and many others. Of course,the views expressed here (and any errors) are ours alone.We also thank Brian Krebs, David Jefferson, and BarbaraSimons for helpful advice. We are particularly grateful to RopGonggrijp and Zakir Durumeric for numerous suggestionsand contributions.

We have not accepted any financial support from withinEstonia, except for travel and accommodations for the inter-national observers during the Oct. 2013 voting period, whichwere paid for by Tallinn City Council. The only requirementfor that arrangement was that we observe the election.

This material is based upon work supported by the U.S.National Science Foundation under Grant No. CNS-1255153and by the National Science Foundation Graduate ResearchFellowship under Grant No. DGE-1256260. Any opinions,findings, and conclusions or recommendations expressed inthis material are those of the authors and do not necessarilyreflect the views of the National Science Foundation.

About the AuthorsJ. Alex Halderman is an assistant professor of computerscience and engineering at the University of Michigan. Hisresearch focuses on computer security and privacy. A notedexpert on electronic voting security, Prof. Halderman helpeddemonstrate the first voting machine virus, took part in Cal-ifornia’s landmark “top-to-bottom” e-voting security audit,and helped lead the first independent review of election tech-nology in India, the world’s largest democracy. In 2010, heled a team that hacked into Washington D.C.’s Internet votingsystem as part of a public security test. He is the instruc-tor of Securing Digital Democracy, a massive online courseabout e-voting’s risks and promise. He is also well knownfor developing the “cold boot” attack against disk encryp-tion, which altered widespread security assumptions aboutthe behavior of RAM, influenced computer forensics prac-tice, and inspired the creation of a new subfield of theoreticalcryptography. His work has won numerous distinctions, in-cluding three best paper awards and the Election VerificationNetwork’s John Gideon Memorial Award. He received hisPh.D. in computer science from Princeton University.

Harri Hursti is a Finnish independent security researcher.He is well known for taking part in a series of e-voting se-curity studies in the U.S. and around the world. A memorycard hack he demonstrated against Diebold voting machinesis popularly known as "the Hursti Hack" and appeared in theHBO documentary, Hacking Democracy. Among other stud-ies, he participated in the EVEREST report commissioned bythe Ohio Secretary of State and a Princeton University studyordered by the New Jersey Superior Court. Hursti is a recip-ient of the EFF’s Pioneer Award for his work on electronicvoting. He has lived in the United States since 2009.

Jason Kitcat is the Leader of Brighton & Hove City Council,the UK’s first Green-led principal authority. Kitcat sits onthe Leading members group for the Key Cities associationof 23 mid-size cities and also has a number of roles at theLocal Government Association. He has been a Green citycouncillor in Brighton for six years. Before becoming Leaderin 2012 he was Lead member for Finance. His professionalbackground is in technology and online businesses such asNetmums and the Open Knowledge Foundation. Kitcat is a

11

prominent digital rights campaigner, particularly with regardsto electronic voting and counting. He wrote GNU.FREE, theFree Software Foundation’s e-voting software before con-cluding that Internet voting was too risky to be used forelections of any importance. Kitcat helped found the OpenRights Group and served as its e-voting coordinator. He led25 election observers in monitoring e-voting and e-countingacross the 2007 English & Scottish elections. The result-ing report helped set the agenda for electoral modernizationwith e-voting no longer a government priority. He holdsa BSc(Hons) from the University of Warwick in ComputerScience & Management Science and MSc Technology &Innovation Management from the University of Sussex.

Margaret MacAlpine is an independent advisor on post-election audits in the United States. Her most notable ad-visory role was the California Post-Election Risk-LimitingAudit Pilot Program of 2011–2012. There she advised theoffice of the Secretary of State on the topic of the logisticsand management of high-speed scanners in the processingand handling of ballots. MacAlpine is recognized as oneof the foremost experts in the country on the scanning andhandling of paper ballots for election auditing. Before theCalifornia pilot, she audited elections for several counties inFlorida and Connecticut.

Drew Springall is a Ph.D. student at the University of Michi-gan, where his research focuses on security and privacy. Heearned his bachelor’s in computer science from the Universityof Alabama after serving in the U.S. Marine Corps. He isa recipient of the National Science Foundation’s prestigiousgraduate research fellowship.

Travis Finkenauer is a Ph.D. student at the University ofMichigan, where his research focuses on security and privacy.Before entering the Ph.D. program, Finkenauer completed hisundergraduate degrees in computer science and mathematicsat the University of Maryland.

8. REFERENCES[1] ptrace(2): process trace. Linux Programmer’s Manual.[2] rsyslog: The rocket-fast system for log processing,

April 2014. http://www.rsyslog.com/.[3] Arne Ansper, Ahto Buldas, Mart Oruaas, Jaan Priisalu,

Anto Veldre, Jan Willemson, and Kaur Virunurm.E-voting concept security: analysis and measures.Technical Report EH-02-01, Estonian NationalElectoral Committee, 2003.

[4] Andrew W. Appel. Security seals on voting machines:A case study. ACM Trans. Inf. Syst. Secur.,14(2):18:1–18:29, September 2011.

[5] Jacob Applebaum, Judith Horchet, and ChristianStöcker. Shopping for spy gear: Catalog advertisesNSA toolbox. Der Spiegel, December 2013.http://www.spiegel.de/international/world/catalog-reveals-nsa-has-back-doors-for-numerous-devices-a-940994.html.

[6] Josh Benaloh, Mike Byrne, Philip T. Kortum, NealMcBurnett, Olivier Pereira, Philip B. Stark, and Dan S.Wallach. STAR-Vote: A secure, transparent, auditable,and reliable voting system. CoRR, abs/1211.1904,2012.

[7] Jennie Bretschneider, Sean Flaherty, SusannahGoodman, Mark Halvorson, Roger Johnston, MarkLindeman, Ronald L. Rivest, Pam Smith, and Philip B.Stark. Risk-limiting post-election audits: Why and how,October 2012. http://www.stat.berkeley.edu/~stark/Preprints/RLAwhitepaper12.pdf.

[8] Joseph A. Calandrino, J. Alex Halderman, andEdward W. Felten. Machine-assisted election auditing.In EVT’07: Proceedings of the USENIX Workshop onAccurate Electronic Voting Technology, pages 9–9,Berkeley, CA, USA, 2007. USENIX Association.

[9] D. Chaum. Secret-ballot receipts: True voter-verifiableelections. Security Privacy, IEEE, 2(1):38–47, Jan2004.

[10] D. Chaum, R.T. Carback, J. Clark, A. Essex, StefanPopoveniuc, R.L. Rivest, P. Y A Ryan, E. Shen, A.T.Sherman, and P.L. Vora. Scantegrity II: End-to-endverifiability by voters of optical scan elections throughconfirmation codes. Information Forensics and Security,IEEE Transactions on, 4(4):611–627, December 2009.

[11] Cybernetica AS. Internet voting solution, 2013.Accessed: May 13, 2014, http://cyber.ee/uploads/2013/03/cyber_ivoting_NEW2_A4_web.pdf.

[12] Dancho Danchev. Study finds the average price forrenting a botnet. ZDNet, May 2010.http://www.zdnet.com/blog/security/study-finds-the-average-price-for-renting-a-botnet/6528.

[13] Estonian Certification Authority. Avaleht. In Estonian.http://www.id.ee/.

[14] Estonian Certification Authority. Mis on ID-tarkvara?In Estonian. https://installer.id.ee/.

[15] Estonian Information System’s Authority. Public keyinfrastructure PKI, May 2012.https://www.ria.ee/public-key-infrastructure/.

[16] Estonian Internet Voting Committee. Dokumendid,2013. In Estonian. Accessed: March 2014,http://vvk.ee/valijale/e-haaletamine/e-dokumendid/.

[17] Estonian Internet Voting Committee. Ehk videos, 2013.In Estonian. Accessed: March 2014,https://www.youtube.com/channel/UCTv2y5BPOo-ZSVdTg0CDIbQ/videos.

[18] Estonian Internet Voting Committee. Using ID-cardand mobil-ID, May 2014.https://www.valimised.ee/eng/kkk.

[19] Estonian Ministry of Foreign Affairs. Estonia today,2012. http://www.euc.illinois.edu/estonia/documents/E-Estonia.pdf.

[20] Estonian National Electoral Committee. Kohalikuomavalitsuse volikogu valimised 2013. In Estonian.http://www.vvk.ee/kohalikud-valimised-2013/.

12

[21] Estonian National Electoral Committee. Vabariigivalimiskomisjon. In Estonian. Accessed: October 2013,http://www.vvk.ee/.

[22] Estonian National Electoral Committee. Elektroonilisehääletamise süsteemi üldkirjeldus, 2013. In Estonian.http://vvk.ee/public/dok/elektroonilise-haaletamise-systeemi-yldkirjeldus-EH-03-03-1_2013.pdf.

[23] Estonian National Electoral Committee. Valimised:Android Apps on Google Play, October 2013. InEstonian. Accessed: May 13, 2014,https://play.google.com/store/apps/details?id=ee.vvk.ivotingverification.

[24] Estonian National Electoral Committee. Statistics aboutInternet voting in Estonia, May 2014.http://www.vvk.ee/voting-methods-in-estonia/engindex/statistics.

[25] Estonian National Electoral Committee. Valimised onthe App Store on iTunes, April 2014. In Estonian.Accessed: May 15, 2014, https://itunes.apple.com/ee/app/valimised/id871129256.

[26] Estonian National Electoral Committee. Valimised:Windows Phone’i rakenduste+mängude pood (Eesti),April 2014. In Estonian. Accessed: May 15, 2014,https://www.windowsphone.com/et-ee/store/app/valimised/11c10268-343f-461a-9c73-630940d8234b.

[27] Estonian National Electoral Committee, EstonianInternet Voting Committee, and Cybernetica AS.Android based vote verification application forEstonian i-voting system, September 2013.https://github.com/vvk-ehk/ivotingverification.

[28] Estonian National Electoral Committee, EstonianInternet Voting Committee, and Cybernetica AS.e-hääletamise tarkvara, September 2013. Accessed:March 2014, https://github.com/vvk-ehk/evalimine.

[29] Estonian Public Broadcasting. Center Party petitionsEuropean human rights court over e-voting, September2013. Accessed: May 14, 2014, http://news.err.ee/v/politics/4ee0c8a2-b9c2-4d28-8ae4-061e7d9386a4.

[30] Jeremy Fleming. EU nations developing cyber‘capabilities’ to infiltrate government, private targets.EurActiv, December 2013.http://www.euractiv.com/infosociety/eu-nations-lack-common-approach-news-532294.

[31] Andy Greenberg. Shopping for zero-days: A price listfor hackers’ secret software exploits. Forbes, March2012. http://www.forbes.com/sites/andygreenberg/2012/03/23/shopping-for-zero-days-an-price-list-for-hackers-secret-software-exploits/.

[32] Sven Heiberg, Peeter Laud, and Jan Willemson. Theapplication of i-voting for Estonian parliamentaryelections of 2011. In VOTE-ID, pages 208–223, 2011.

[33] Greg Hoglund and James Butler. Rootkits: Subvertingthe Windows Kernel. Addison-Wesley, 2005.

[34] R.G. Johnston. The real deal on seals: Improvingtamper detection. Security Management, 41:93–100,September 1997.

[35] R.G. Johnston. Some comments on choosing seals &on PSA label seals. In Proceedings of the 7th SecuritySeals Symposium, 2006.http://www.ne.anl.gov/capabilities/vat/pdfs/choosing-seals-and-using-PSA-seals-2006.pdf.

[36] R.G. Johnston. Insecurity of New Jersey’s sealprotocols for voting machines, October 2010.http://www.cs.princeton.edu/~appel/voting/Johnston-AnalysisOfNJSeals.pdf.

[37] R.G. Johnston and Anthony R.E. Garcia. Vulnerabilityassessment of security seals. Journal of SecurityAdministration, 20:15–27, 1997.

[38] Douglas W. Jones and Barbara Simons. Broken Ballots:Will Your Vote Count? Stanford University Center forthe Study of Language and Information, 2012.

[39] Erik Kain. Report: NSA intercepting laptops orderedonline, installing spyware. Forbes, December 2013.Accessed: May 14, 2014, http://www.forbes.com/sites/erikkain/2013/12/29/report-nsa-intercepting-laptops-ordered-online-installing-spyware/.

[40] Jason Kitcat. Source availability and e-voting: Anadvocate recants. Commun. ACM, 47(10):65–67,October 2004.

[41] Benjamin Laxton, Kai Wang, and Stefan Savage.Reconsidering physical key secrecy: Teleduplicationvia optical decoding. In Proceedings of the 15th ACMConference on Computer and CommunicationsSecurity, CCS ’08, pages 469–478, New York, NY,USA, 2008. ACM.

[42] Mark Lindeman and Philip B. Stark. A gentleintroduction to risk-limiting audits. IEEE Security &Privacy, 10(5):42–49, 2012.

[43] Helger Lipmaa. Paper-voted (and why I did so). Blogpost, March 2011. http://helger.wordpress.com/2011/03/05/paper-voted-and-why-i-did-so/.

[44] Mandiant. APT1: Exposing one of China’s cyberespionage units, February 2013.http://intelreport.mandiant.com/Mandiant_APT1_Report.pdf.

[45] Nick Mediati. How to remotely install apps on yoursmartphone. TechHive, November 2013.http://www.techhive.com/article/2067005/how-to-remotely-install-apps-on-your-smartphone.html.

[46] Urmas Oja. Paavo Pihelgas: Elektroonilise hääletamisevaatlemine on lihtsalt võimatu, March 2011. InEstonian. http://forte.delfi.ee/news/digi/paavo-pihelgas-elektroonilise-haaletamise-vaatlemine-on-lihtsalt-voimatu.d?id=41933409.

[47] Francois Paget. Hacking summit names nations withcyberwarfare capabilities. McAfee Blog Central,October 2013.http://blogs.mcafee.com/mcafee-labs/hacking-summit-names-nations-with-cyberwarfare-capabilities.

[48] Arnis Parsovs. Practical issues with TLS clientcertificate authentication, February 2014. https://

13

www.internetsociety.org/sites/default/files/12_4_1.pdf.[49] Teet Raidma and Janno Kase. Kohaliku omavalitsuse

volikogu valimiste e-hääletamise protseduuridehindamise löpparuanne, January 2014. In Estonian.http://vvk.ee/public/KOV13/lopparuanne_2013.ddoc.

[50] Peter Y. A. Ryan, David Bismark, James Heather, SteveSchneider, and Zhe Xia. Prêt à voter: A voter-verifiablevoting system. Trans. Info. For. Sec., 4(4):662–673,December 2009.

[51] David E. Sanger. Obama order sped up wave ofcyberattacks against Iran. The New York Times, June2012. http://www.nytimes.com/2012/06/01/world/middleeast/obama-ordered-wave-of-cyberattacks-against-iran.html.

[52] Barbara Simons. Report on the Estonian Internet votingsystem. Verified Voting Blog, September 2011.https://www.verifiedvoting.org/report-on-the-estonian-internet-voting-system-2/.

[53] Philip B. Stark. Super-simple simultaneoussingle-ballot risk-limiting audits. In Proceedings of the2010 International Conference on Electronic VotingTechnology/Workshop on Trustworthy Elections,EVT/WOTE’10, pages 1–16, Berkeley, CA, USA,2010. USENIX Association.

[54] Ken Thompson. Reflections on trusting trust. Commun.ACM, 27(8):761–763, August 1984.

[55] Ian Traynor. Russia accused of unleashing cyberwar todisable Estonia. The Guardian, May 2007.http://www.theguardian.com/world/2007/may/17/topstories3.russia.

[56] Ian Traynor. GCHQ: EU surveillance hearing is told ofhuge cyber-attack on Belgian firm. The Guardian,October 2013.http://www.theguardian.com/uk-news/2013/oct/03/gchq-eu-surveillance-cyber-attack-belgian.

14