Top-To-Bottom Review
Secretary of State Debra Bowen will begin a thorough top-to-bottom review of the voting machines certified for use in California the week of May 14, 2007.
The review is designed to restore the public's confidence in the integrity of the electoral process and is designed to ensure that California voters are being asked to cast their ballots on machines that are secure, accurate, reliable, and accessible.
Following are the press release issued on May 9, 2007, announcing the review, a brief summary of the review process, and a longer "Frequently Asked Questions" (FAQ) document to help people understand how the review will work.
If you have additional questions or would like to see more information added to the FAQ please send an e-mail to votingsystems@sos.ca.gov.
snip
http://www.sos.ca.gov/elections/elections_vsr.htmFrequently Asked Questions About Secretary of State Debra Bowen’s Top-To-Bottom Review of California’s Voting Systems
May 9, 2007
Why is it necessary to conduct a top-to-bottom review of California’s voting systems? The top-to-bottom review is designed to give California’s voters an answer to one simple question: Are all of California’s voting systems secure, accurate, reliable and accessible?
What is a top-to-bottom review of California’s voting systems? The top-to-bottom review will consist of a thorough examination of all voting system documentation, procedures, and the equipment used to record and tally votes. The review will have four components:
A document review will examine manufacturer documentation, testing reports from federal Independent Testing Authorities (ITAs), reports from prior state certification testing, and reports of independent examinations and testing of voting systems.
A source code review will examine the human-readable instructions that are converted into machine-readable code to run the voting systems. The primary focus will be to identify any security vulnerabilities that could be exploited to alter vote recording, vote results, critical election data such as audit logs, or to conduct a “denial of service” attack that prevents people from voting.
Red team penetration testing will involve open-ended, hands-on efforts to identify and document any potential for tampering or error in any part of the voting system’s hardware, storage devices or software. The red team testers and source code review teams will coordinate their efforts, so potential vulnerabilities identified in red team testing can be further explored in the source code review, and vice versa.
The accessibility of the voting systems will be assessed and will include test voting on each of the voting systems by volunteer voters representing a broad range of disabilities.
The document review teams, source code review teams and red teams will interact regularly to learn from one another and to ensure the review of all systems is even-handed.
How will the voting systems be evaluated and does that differ from the draft criteria published on March 22? The draft criteria relied in part on a set of standards that voting system vendors were not explicitly required by state or federal law to meet. Based on the comments received from interested parties, the final project plan being used to evaluate the voting systems doesn’t include those draft standards. Instead, the top-to-bottom review teams will provide an independent technical evaluation of the voting systems that the Secretary of State will use to carry out her statutory duty with respect to voting systems, as required by Division 19 of the State Elections Code.
The standards and definitions for security, accuracy, reliability and protection of ballot secrecy governing the top-to-bottom review are set forth in the 2002 Voluntary Voting System Standards, which may be found at
http://www.eac.gov/election_resources/vss.html. California Elections Code Section 19250 requires voting systems to comply with these standards as a condition of being certified for use in the state.
With respect to accessibility for voters with disabilities and with alternative language requirements, the standards and definitions governing the top-to-bottom review are set forth in the 2005 federal Voluntary Voting System Guidelines, which may be found at
http://www.eac.gov/VVSG%20Volume_I.pdf and in California Elections Code Sections 19227, 19250 and 19251.
The red team penetration testing will be conducted in accordance with Resolution # 17-05 of the Technical Guidelines Development Committee (hereafter “TGDC”) of the U.S. Election Assistance Commission, adopted at the TGDC plenary meeting on January 18-19, 2005, which calls for:
“. . . testing of voting systems that includes a significant amount of open-ended research for vulnerabilities by an analysis team supplied with complete source code and system documentation and operational voting system hardware. The vulnerabilities sought should not exclude those involving collusion between multiple parties (including vendor insiders) and should not exclude those involving adversaries with significant financial and technical resources.”
Who will conduct the review? The Secretary of State is contracting with the University of California (UC) to assemble three top-to-bottom review teams that rely on specialists from UC, as well as from public and private universities and private sector companies throughout the United States. To ensure a fresh look at the voting systems, scientists with specific experience in voting system technology and security experts from other fields who may have no experience with voting system technology will be asked to participate. Each review team will consist of seven members and will include three components – document review, source code review, and red team penetration testing.
While all of the team members have not yet been identified, the two Principal Investigators for the project are Matthew Bishop, Professor in the Department of Computer Science and Co-Director of the Computer Security Laboratory at UC Davis, and David Wagner, Associate Professor in the Computer Science Division at UC Berkeley, with extensive experience in computer security, cryptography and electronic voting. He is a founding member of the ACCURATE center, which is funded by the National Science Foundation to research ways that technology can be used to improve voting.
The three source code review teams will include:
** Matt Blaze, Associate Professor of Computer Science, University of Pennsylvania, an internationally recognized expert in computer security, cryptography and the interplay of technology and public policy.
** Ed Felten, Professor of Computer Science and Public Affairs, Princeton University; Director of Center for Information Technology Policy, Princeton University; an internationally recognized expert in computer security and in information technology policy.
** Eric Rescorla, Chief Scientist of Network Resonance, Inc., a network security research and development company located in Palo Alto, California. His research interests focus on communications security and evidence-based analysis of security strategies. He is active in the standards community, serving as Internet Engineering Task Force (“IETF”) Transport Layer Security (“TLS”) working group chair, the editor of the TLS and HTTP over TLS specifications as well as numerous other IETF documents. He has served on the Internet Architecture Board since 2002.
The three red team penetration teams will include:
** Mark McLarnon, RABA Technologies, Columbia, Maryland, Co-author of Trusted Agent Report, Diebold AccuVote TS Voting System for the Department of Legislative Services, Maryland General Assembly (2004).
** Harri Hursti, Independent Computer Security Consultant; Member, Task Force of The Brennan Center For Justice Voting Technology Assessment Project; Co-founder and chairman of the board of ROMmon, Finland.
** Giovanni Vigna, Associate Professor, Computer Security Group, Department of Computer Science, UC Santa Barbara.
The three document review teams will include:
** Deirdre K. Mulligan, Director of the Samuelson Law, Technology & Public Policy Clinic, a Clinical Professor of Law at the UC Berkeley School of Law (Boalt Hall) and a member of the ACCURATE center. Before coming to Boalt, she was staff counsel at the Center for Democracy & Technology in Washington, D.C.
** Candice Hoke, Associate Professor of Law and Director, Center for Election Integrity, Cleveland State University.
** Joseph Lorenzo Hall, MA, MIMS, Ph.D. candidate in the Department of Information Management and Systems, UC Berkeley.
In addition to the teams described above, the accessibility of the voting systems will be assessed by a single team of two accessibility experts, headed by Noel Runyan of Campbell, California. Mr. Runyan is an electrical engineer and computer scientist with over 33 years experience in design and manufacturing of access technology systems for people with disabilities. For the last four years, he has concentrated on the accessibility of voting systems. The accessibility assessment will include test voting on each of the voting systems by volunteer voters representing a broad range of disabilities.
How will the review be conducted, how long will it take, and what will happen when the review is completed? The Secretary of State will determine the order in which the voting systems are reviewed by a random selection that will be conducted in public using the same system that’s used to randomly select the order candidates appear on the ballot.
Once the order of testing has been determined, if the system that has been randomly selected first has not been provided to the Secretary of State, the Secretary of State will move to the system that has been randomly selected second. That process will continue until there are three systems for the reviewers to begin testing the week of May 14. The Secretary of State reserves the right to begin decertification proceedings for any system that hasn’t been provided to the Secretary of State’s office.
Each top-to-bottom review team will evaluate at least two voting systems. If more than one version of a vendor’s voting system is subject to review, the different versions will be assigned to the same team.
Each team will devote a minimum of three weeks to examine, test and prepare a draft report of findings for the Secretary of State to review and evaluate.
The entire top-to-bottom review process is designed to be completed by the end of July. The Secretary of State will hold a public hearing and invite public comments on the results of the review before making any final decisions on whether currently certified systems will continue to be certified for use in California and if so, what, if any, new conditions will be attached to their use.
What happens if a voting system vendor chooses not to participate in the review? If a vendor chooses not to have its voting system reviewed, the Secretary of State has the option of initiating a decertification process immediately.
What will happen with new voting systems that receive federal approval? If a system receives federal approval and is submitted to the Secretary of State by July 1, 2007, for certification in California, the Secretary of State will fully review that system using the same standards that will be applied in the top-to-bottom review.
What if a vendor chooses to opt out of having its existing system tested in anticipation of federal approval later this year for a replacement system? Any system that isn’t federally certified and submitted to the Secretary of State by July 1, 2007, will not be able to make it through the state certification process in time to be used in the 2008 elections. Therefore, if a vendor opts out of the top-to-bottom review but does not submit a replacement system for certification by July 1, 2007, the Secretary of State may either decertify or conditionally recertify the existing system for 2008 elections with additional restrictions, which may include the following:
For direct recording electronic (DRE) machines:
o A requirement that these machines shall only be used to provide accessibility for disabled voters.
o A mandatory 100% manual audit of the voter-verified paper audit trail to verify DRE voting results.
o Additional chain of custody controls and other security measures.
For paper-based systems:
o A requirement that all paper ballots must be centrally counted.
o A mandatory 10% manual audit of machine tallies.
o Additional chain of custody controls and other security measures.
Is the top-to-bottom review going to test entire voting systems or only the voting machines used in polling places? The only way to make sure a voting system is properly recording and counting votes is to review a voting system from top to bottom. That’s why the review will include all of the various machines used to cast ballots, as well as the systems used to count ballots, including vote tabulating devices, election management and tabulation programs, and associated firmware, software and peripheral devices.
What are the systems being tested? The following certified voting systems are subject to examination and testing under the top-to-bottom review:
Diebold GEMS 1.18.24/AccuVote
** GEMS software, version 1.18.24
** AccuVote-TSX with AccuView Printer Module and Ballot Station firmware version 4.6.4
** AccuVote-OS (Model D) with firmware version 1.96.6
** AccuVote-OS Central Count with firmware version 2.0.12
** AccuFeed
** Vote Card Encoder, version 1.3.2
** Key Card Tool software, version 4.6.1
** VC Programmer software, version 4.6.1
ES&S Unity 2.4.3.1/AutoMARK
** Unity 2.4.3.1
• Audit Manager v. 7.0.2.0
• EDM v. 7.2.1.0
• ESSIM v. 7.2.0.0
• HPM v. 5.0.3.0
• ERM v. 6.4.3.3
** Model 100 Precinct Scanner, version 5.0.0.0
** Model 550 Central Scanner, version 2.1.1.0
** Model 650 Central Scanner, version 1.2.0.0
** AutoMARK Information Management System (AIMS), version 1.0
** AutoMARK Voter Assist Terminal, version 1.0
ES&S City and County of San Francisco Voting System
** Optech III-P Eagle version HPS 1.30/APS 1.52
** Optech IV-C, Model 400 version 1.07(a) (or version 1.08(c))
** Unity version 2.4.3
ES&S InkaVote Plus Precinct Ballot Counter Voting System, version 2.1
** InkaVote Plus Precinct Ballot Counter with ADA unit, firmware version 1.10
** Unisyn Election Management System, version 1.1
• Ballot Generation, version 1.1
• Election Converter, version 1.1
• Election Loader, version 1.1
• Vote Converter, version 1.1
• Vote Tabulation, version 1.1
Hart Intercivic System 6.1
** Ballot Now software, version 3.2.4
** BOSS software, version 4.2.13
** Rally software, version 2.2.4
** Tally software, version 4.2.8
** SERVO, version 4.1.6
** JBC, version 4.1.3
** eSlate/DAU, version 4.1.3
** eScan, version 1.2.0
** VBO, version 1.7.5
** eCM Manager, version 1.1.7
Hart Intercivic System 6.2.1
** Ballot Now software, version 3.3.11
** BOSS software, version 4.3.13
** Rally software, version 2.3.7
** Tally software, version 4.3.10
** SERVO, version 4.2.10
** JBC, version 4.3.1
** eSlate/DAU, version 4.2.13
** eScan, version 1.3.14
** VBO, version 1.8.3
** eCM Manager, version 1.1.7
Sequoia WinEDS version 3.1.012/Edge/Insight/400-C
** WinEDS, version 3.1.012
** AVC Edge Model I, firmware version 5.0.24
** AVC Edge Model II, firmware version 5.0.24
** VeriVote Printer
** Optech 400-C/WinETP firmware version 1.12.4
** Optech Insight, APX K2.10, HPX K1.42
** Optech Insight Plus, APX K2.10, HPX K1.42
** Card Activator, version 5.0.21
** HAAT Model 50, version 1.0.69L
** Memory Pack Reader (MPR), firmware version 2.15
County of Los Angeles InkaVote Optical Scan Voting System
** Microcomputer Tally System (MTS) version 1.3.1
** LRC 1000 CPM Card Reader
** InkaVote Vote Recorder Device
Are any systems not scheduled to be reviewed at this point? The DFM Mark-A-Vote system used by Lake, Madera, and Sonoma counties will not be reviewed in this round of testing, but the Secretary of State reserves the right to conduct a review of this system at a later date. The reason to exclude the DFM system at this point is based on the fact that there is a limited amount of time and money to conduct the voting system reviews, and the Mark-A-Vote is a paper-based system that is fully auditable.
Where will the top-to-bottom review be conducted? Testing, examination and review activities, and analysis will be conducted onsite at the Secretary of State’s facilities in Sacramento under secure conditions, with one exception. The review of documentation and source code may, upon express written authorization of the Secretary of State, be conducted at secure facilities of UC or other secure locations designated by UC.
Will this review be open to the public? Given the proprietary nature of the systems being reviewed and laws that preclude the Secretary of State from releasing anything that may be considered proprietary in nature or contain trade secret information, it’s unlikely that much of the actual physical red team testing and source code examination will be conducted in front of the public. A select number of county representatives who agree to sign the same non-disclosure agreements that each of the reviewers will be required to sign may be permitted to observe portions of the review process. Furthermore, as necessary and appropriate, the Secretary of State may, during the review process, ask top-to-bottom team members to publicly replicate any findings or discoveries they make on a voting system. In addition, the Secretary of State intends to maintain a website with up-to-date information on the status and progress of the review.
How much will the review cost and where will the funding come from to pay for it? Approximately $450 million has been spent or set aside to upgrade California’s voting equipment over the past several years.
The total cost of the top-to-bottom review is estimated to be $1.8 million, though that number could drop depending on the number of systems that are ultimately reviewed and the length of each review. Of the total amount, $760,000 will come from the federal Help America Vote Act (HAVA) funding that was provided by the Legislature for voting machine source code review as part of the 2006-07 state budget. The remaining funding for the review will come from the voting system vendors that have voting systems certified for use in California and potentially from HAVA funding that may be apportioned as a part of the 2007-08 state budget. It’s estimated the review of each system will cost approximately $262,000, with the costs being split equally between the vendor and California’s HAVA funding allocation. California law, as well as the certification agreements many of the voting system vendors signed with the former
Secretary of State as a condition of certification, allow the Secretary of State to review voting systems at any time and allow the Secretary of State to require vendors to pay for the cost of conducting the review. If a vendor chooses not to have its system reviewed as part of the top-to-bottom review, the overall project cost would drop, though the per system cost will rise slightly, since certain baseline costs (such as hiring a project manager and accessibility consultants) won’t be reduced solely because the number of systems being reviewed is reduced.
What if a vendor wants a new system to be tested instead of or in addition to having its currently certified system tested? All future testing and examination of voting systems submitted to the Secretary of State’s office for certification will incorporate the security and accessibility testing used in the top-to-bottom review.
The costs for complete state examination of a voting system are estimated to be approximately $350,000 to $375,000. Should problems be discovered in testing that require system modification and retesting, these costs can increase. Vendors will be required to submit adequate funds to the Secretary of State to be held in an escrow account for the payment of the state’s examination costs before any testing begins.
Vendors must complete all applicable federal testing and receive federal certification from the Election Assistance Commission (EAC) before they can submit an application for California certification of a voting system.
This examination plan and the testing protocols will be updated and further refined to incorporate lessons learned from the top-to-bottom review.
It is worth noting that other states have also begun a thorough state-level testing program, partially in response to the major flaws in the federal testing regime.
For example, New York has also instituted comprehensive state testing, and voting system vendors will be expected to pay a comparable or greater amount of money to have their systems certified for use there..pdf
http://www.sos.ca.gov/elections/voting_systems/ttbr/qa.pdf