Rebecca Mercuri: Voting Advocates Roundtable Discussion (EAC)

[Wilms]

Written Testimony by Rebecca Mercuri

Representing: The BRAD BLOG
Voting Advocates Roundtable Discussion
EAC Offices, Washington, DC, April 24, 2008


The 2007 draft Voluntary Voting System Guidelines (VVSG) represents a significant departure from earlier Federal voting system guidelines (2005 EAC, 2002 and 1990 FEC), while still retaining much of the certification framework that has been increasingly demonstrated to be problematic. Within the guise of certification, the past few years have seen billions of Federal and State tax dollars squandered on the purchase of voting systems that were subsequently revealed as inappropriate for use, and then discarded. We now know that the VVSG, and its ITA testing program, provide no assurance of process or equipment correctness, either to those who are making procurement decisions, or to the citizens who must entrust their votes to these systems. Tragically, the net result of this false validation has led to further erosion of voter confidence in elections.

This draft VVSG continues to perpetrate this scam. Among other changes, it recognizes earlier shortcomings of the certification process (especially in the areas of voter verification, transparency, audibility and security) by introducing an innovation class that allows for the submission of novel voting system paradigms for certification, and provides for the (somewhat related) adoption of a software independence requirement. Unfortunately, both of these concepts are oxymorons in the context of voting system specifications. Here’s why. If a construct is truly innovative, the existing guidelines will not be able to appropriately address it, hence the resulting certification may be flawed or the implementation of the new design may necessarily be impeded by a lack of understanding as to how to properly perform certification. A system that contains software can never be software independent, even within the TGDC/NIST’s constrained definition that ties undetected changes or errors in software to election outcomes. Any software in the system necessarily affects a whole host of voting attributes that can affect election results, irrespective of undetected changes or errors.

Furthermore, neither the innovation class nor the software independence requirement are satisfiable due to legacy constraints imposed by the certification process. This is, at least in part, because the 2007 draft VVSG (like its predecessors) masquerades as a functional standard, while actually continuing to be predisposed to existing designs. Even the TGDC’s description of the innovation class makes design assumptions, such as its limiting “expect most technologies in this class be based on multiple mutually auditing components.” But even as a design specification, the draft VVSG falls short of achieving its goals of specifying “how voting systems should perform or be used in certain types of elections and voting environments.” This is because the guidelines repeatedly make the erroneous assumption that insiders (i.e. vendors, repair personnel, election officials, etc.) are trusted agents in the highly partisan process of US elections. In reality, insiders have both motive and opportunity to make changes and cover up the fact that they have done so. Where errors have been blatantly obvious, vendors go to great lengths (including lawsuit threats) to prevent independent examinations of equipment architecture and computer code. Some election officials have improperly conducted audits in order to avoid revelation that problems have occurred “on their watch.” In sum, virtually all of the checks and balances that are specified by the VVSG fail to take insider attacks into sufficient consideration. Voters believe that elections are inherently corrupt, and the VVSG does nothing to allay these fears.

Nor are the VVSG’s specified controls transparent enough to allow verification by the voter that the election system they are using has been configured properly. Production of a voter-verified paper ballot is utterly moot if vote totals are generated electronically and never checked against the original paper. Recent literature has suggested random audits (or spot-checks), but since these percentages are based on the computer-generated results, they grossly underestimate the amount of independent tallies that must be performed to sufficiently validate the election. These checks are not prescriptive as to what to do when anomalies are revealed. Courts have been reluctant to dismiss election results, even in the extreme, such as when over 80% of the precinct ballot counts differ from the number of signatures in the polling books and the vendor has admitted to deploying an uncertified configuration of voting system components in violation of State requirements (ref. The 2006 Franklin County, Ohio recount case of Carole R. Squire vs. Christopher J. Geer).

In these matters, it generally falls to the contestor to prove that anomalies affected the results in such extent that, had they not occurred, the outcome would have been different. And the contestor must make this proof in the absence of access to the voting equipment or test results, since vendors and ITAs are allowed to claim trade secrecy protection for their materials. The 2007 draft VVSG further perpetuates this trade secrecy loophole (as had prior versions of the guidelines) by continuing to exempt COTS (commercial-off-the shelf) products (including those with critical underpinnings such as device drivers and operating systems) from source code inspection and other standard reviews. This lax and dangerous view of COTS products is most evident in the fact that these are never required to be updated, even when new versions are issued to remedy known security risks.

One might think that, at least, if a voting system (or any of its components or modules) was found to be defective, or if the testing was discovered to have been improperly performed or deemed inadequate, there would be some process whereby the EAC would be required to withdraw certification. But the 2007 draft VVSG (like its predecessors) omits mention of any methodology whereby certification can be rescinded because of later-discovered flaws. The VVSG thus provides no protection to either the purchasers or the voters, since perversely, there is a disincentive for vendors to issue corrections to deployed systems, because any changes (even necessary ones) require costly recertification. The Catch-22 scenario is that you can continue to use defective voting machines, but you may not be able to obtain versions that have had the defect corrected. This situation must stop.

Most of the above issues are well-known and have been reported to the EAC in its various incarnations, by many people (including myself and Brad Friedman), numerous times. The 2007 draft VVSG continues the tradition of providing a set of straw hurdles that must be jumped over (or skirted around) in order to attain certification, while resulting in no true assurances. Another VVSG rewrite, novel designs, or more extensive testing cannot begin to solve these problems until the voters’ demands for transparency, reliability, security, accuracy and auditability requirements have first been appropriately defined and addressed. So long as the goal of certification trumps the need to ensure election integrity, the resulting systems, no matter whose imprimatur they bear, will be invalid and must be rejected.

http://www.eac.gov/News/docs/mercuritestimonyapr08/attachment_download/file (.pdf)

[FogerRox]

1. This is why Rebecca is the best... LOL.

[Wilms]

3. I still think so. But I'm now aware she was unaware of something when she wrote that.

There is a "methodology whereby certification can be rescinded because of later-discovered flaws".

It's just that it's not in the VVSG. It's in a separate EAC document, "Voting System Testing and Certification Manual"

Decertification
7.1. Overview.

Decertification is the process by which the EAC revokes a certification previously granted to a voting system. It is an important part of the Certification Program because it serves to ensure that the requirements of the program are followed and that certified voting systems fielded for use in Federal elections maintain the same level of quality as those presented for testing. Decertification is a serious matter. Its use will significantly affect Manufacturers, State and local governments, the public, and the administration of elections. As such, the process for Decertification is complex. It is initiated when the EAC receives information that a voting system may not be in compliance with the applicable voting system standard or the procedural requirements of this Manual. Upon receipt of such information, the Program Director may initiate an Informal Inquiry to determine the credibility of the information. If the information is credible and suggests the system is non-compliant, a Formal Investigation will be initiated. If the results of the Formal Investigation demonstrate non-compliance, the Manufacturer will be provided a Notice of Non-Compliance. Before a Final Decision on Decertification is made, the Manufacturer will have the opportunity to remedy any defects identified in the voting system and present information for consideration by the Decertification Authority. A Decertification of a voting system may be appealed in a timely manner.

http://www.eac.gov/voting%20systems/docs/certification-docs-certification-program-manual-omb-3265-0004-exp-6-30-2010.pdf/attachment_download/file

[FogerRox]

5. Nice little clause there. The Decertification clause, LOL

[Bill Bored]

2. Couple of comments:

"Recent literature has suggested random audits (or spot-checks), but since these percentages are based on the computer-generated results, they grossly underestimate the amount of independent tallies that must be performed to sufficiently validate the election."

Not the literature I've been reading. It says that the "computer-generated results" are only what is being independently TESTED by the audit -- not that the audit is "based on" them. Of course there is more than 1 piece of literature out there on this subject. Did Dr. Mercuri cite anything in particular in her testimony?

One more point: On the notion of the "innovation class" of software-independent voting systems, I tend to agree that this is mostly vapor ware at this point, which is why software independence for all practical purposes would require paper ballots for the foreseeable future. This is why it's probably a good idea to write some laws about this stuff to preclude whatever undesirable innovations might arise. But in the meantime, the requirement for software independence could achieve what no federal legislation or action has come close to achieving, in the foreseeable future: the end of federally certified paperless electronic voting systems.

[BradBlog]

6. Couple of responses...

What Mercuri was referring to is that the sample-size for the audits, are generally based on the computer-generated results. So, for example, where there is a close election (according to the computer results), the sample size of the audit would be larger. Not so close elections would lead to smaller audit samples, or no audit at all.

Thus, relying on the computer-generated (and perhaps manipulated and/or simply erroneous) computer-reported results to determine audit size is a huge flaw in such schemes.

As to the "Software Independence" nonsense, the point you make is inaccurate, as the supporters of the SI concept believe that unverifiable DRE voting with so-called VVPAT paper-trails, would conform with "Software Independence".

Thus, you're wrong when you suggest that "software independence for all practical purposes would require paper ballots for the foreseeable future." In fact, it would allow for the continued use -- and federal institutionalization of -- wholly unverifiable DRE voting systems.

Beyond that, good catch by Wilms re: the EAC's decert provision. Will have to look more into it, because they've never used it, despite the trigger many times occurring that should have led to its use. Their excuse is likely the same one they use when asked about this: "There are no currently federally certified voting systems," according to the EAC, since all the currently certified systems were done by NASED, not EAC, and so they consider them "federally qualified" but not "federally certified".

I wish I was kidding.

[Bill Bored]

7. Brad, you (and I guess Dr. M.) are incorrect about audits. Audits TEST outcomes, results, etc.....

...they do NOT "rely" on them. So by definition, they MUST take the reported outcomes, results, etc. into account in order to test them. The more incorrect a result is, the easier it is to detect with an audit. And like you, I would not support having "no audit at all", unless there are no opposing candidates at all. Even then, you might want to do a "spot check", but I wouldn't insist on it with only one candidate on the ballot.

As far as SI, I did say a law should be written to use paper ballots didn't I?

The point is that SI would do away with paperless DREs and if I remember correctly, in 2004 or so, these machines were seen as the end of our representative form of government. Now the federal government itself is asserting that they are not suitable voting devices, and you have a problem with that? I call it progress.

The reason the EAC won't decertify a voting system, they say, is that they have never certified a voting system in the first place. All systems certified so far were pre-EAC-certified, by the trade association known as NASED. I'm not saying the EAC will necessarily do a better job, but since they have yet to certify a voting system, perhaps they are attempting to.

[Wilms]

9. 2004 is sooooo 4 years ago, Bill.

These machines, we've since learned, are worse than we had realized. You say it's better to add a printer than not. Fine. Now don't put words in the mouth of others asking what problem they have with the fed saying paper-less DREs aren't any good. The problem for some is the possibility that DREs will be fitted with printers and grow roots in the nation's election management system.

I don't exactly agree with that perspective. Especially when words like institutionalize are used as if there's legal import. But I don't need to misrepresent it.

And while I appreciate, and often insist on accuracy, glibly repeating that the EAC not having certified is unable to decertify voting machines, while accurate, misses larger points we all know and agree on. HAVA is a mess.

[Bill Bored]

10. I said the EAC SAY they don't decertify because they don't certify.

That's not the same as saying they can't. And it's not repeating; it's stating their current position, in case there are some who are reading this board who don't know what the EAC's position is, who don't know that there was a voting system certification program and entity before the EAC, who don't know that this e-voting problem has been around since the 1980s, or earlier, who don't know that HAVA is just the latest manifestation of it, and who actually believe that before 2000, 2002 or 2004, this was not a problem or an issue in our democracy. And you know, judging from some of the posts on this board, that there are many who aren't aware of these facts. Maybe there are fewer of them now.

So now that the facts have been stated, anyone is free to attack the EAC for not getting the decertification job done, or whatever. But in doing so, I think it's important to know what their purported reason is, or doesn't that matter?

As far as the whole SI thing, here's how I see it:

Since the 1980s, long before HAVA, computer science types like Rebecca Mercuri and others have been saying that paperless electronic voting systems are unsuitable for use in democratic elections. Their detractors have said that they were on the fringe, disgruntled, etc.

Now, the federal government (NIST), and possibly the EAC itself (if the new VVSG is approved), are saying essentially the same thing, and may be prepared to consign this particular bit of madness to dust bin of history by making it impossible for such dangerous machines to be federally certified.

Now, if we can get together and write a simple paper ballot law to go along with this, and require some hand counts to see who really won our elections, then to a large extent, the e-vote counting threat will be neutralized. We can move on to other threats, being eternally vigilant and all that good stuff, and make sure this never happens again on our watch. (It certainly didn't start on "our watch" this time. I think a lot of us were in diapers when e-vote counting began.)

Now I could say some people just don't want to take "Yes" for an answer, but I'm sure that's not subtle enough because everyone's idea of "Yes" may be different and indeed, "Yes" may be a moving target. But when asked if they think that voting systems without a paper backup of some kind are a good idea, Uncle Sam is finally saying "No", and despite what others are saying, I think this is, on balance, a good thing.

[Wilms]

8. I have to ask.

What does "federal institutionalization" mean? If it means paying to help keep them around, that's not as impressive as HAVA paying for it all in the first place. I've heard this term used before in regard to DRE's in particular, and e-elections in general, but I don't quite follow the legal significance suggested by the complaints I've heard. I've asked and never had explained the terms use.

And actually Brad, are you kidding? In order to organize certification of equipment the following needs to occcur: EAC created, in part, to replace the private club, NASED; Voting machine standards written;
ITA standards written; ITA's certified; Equipment tested.

Not that I really wanted them to ever deploy equipment, but not deploying them prior to completing the above wasn't going to be.

There's no surprise, really. This was going to be SOME kind of mess.

[kster]

4. '--

http://www.votersunite.org/info/RandomSample.mov

OH SHIT!!