EAC and NASED have to certify equipment AFTER the ITA...then the states

[Wilms]

I started a thread about the Election Technology Council
http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=203x410640

A pdf on their site covered certification procedure. I'd rather get all this from the EAC (whose comments are posted below) but their's seems not as thorough.
http://www.itaa.org/es/docs/FINAL_ETC_Version_Infographic.pdf

What interested me the most is that the EAC and NASED have to certify the equipment after the ITA has done so. Then, of course, the states will probably have their own process.


Voting System Independent

Testing and Certification Process:
Comprehensive, Rigorous, and Objective

Prepared by:
The Election Technology Council

November 2005

1. Standards Development:
Current and evolving state and federal law, regulations, and standards define requirements that must
be met before voting equipment may be used in an election. Today, the federal 2002 Voting Systems
Standards provide guidance to vendors and the system-testing bodies. New standards - the Voluntary
Voting System Guidelines - are currently under development.

1a. Product Development & Quality Assurance
Election systems manufacturers continually conduct new product development to enhance current
voting equipment and innovate the next generation of voting technology. This development process is
driven by state and federal laws/standards that establish specific voting system requirements. It is
also responsive to the needs of election administrators and voters.

2. To ITA Testing:
After development, documentation, and quality assurance, to be certified to federal voting systems
standards, a voting system and its component parts must go through extensive testing conducted by
accredited Independent Testing Authorities (ITAs). This process begins with submission of a detailed
Technical Date Packet (TDP) from the vendor.

2a. Source Code
ITAs review every line of software code to ensure compliance with standards and overall integrity.
Once complete, an ITA will perform and witness the compilation of the source code into program
executable files.

2b: Functionality (End-to-End System Testing)
ITAs test the functionality of the voting equipment using compiled code to ensure it operates
accurately - that votes are properly captured, results are properly reported, and data is properly
retained. To pass, a system must tabulate 1.5 million votes with 100% accuracy.

2b. Durability
ITAs test the operation of the voting equipment to ensure it can withstand extreme environmental
conditions and intensive human handling.

3. ITA Reports
If, at any point in the testing process, an ITA identifies an issue that must be addressed, a product or
component part is sent back to the vendor for additional development and resubmission through the
whole ITA testing process. Only after the system or component has passed every test is it deemed
qualified for federal certification. An ITA compiles the results of the entire testing process into a report
which is delivered to the EAC/NASED for further review.

4. EAC/NASED Review
The NASED technical committee assesses the results of the ITA tests to ensure compliance with
federal law and standards. If necessary, NASED requests clarification from the ITA and/or additional
development by the vendor. Only if a system passes this review does NASED issue a number
indicating formal federal certification of the specific version of the voting system.

5. State Testing & Certification
In a majority of states, federal certification is only a first step before a voting system can achieve state
certification. In some cases, the state will carry out its own independent testing of the accuracy,
security, and reliability of a system. State testing (which varies state-to-state) expands upon and
enhances testing at the federal level. A state also will compare a product's features and functionality
against state law and standards to ensure it complies.

5a. State Software Escrow
Many states require the vendor to escrow a copy of the certified system software.

6. Local Testing
After production testing and upon delivery from a vendor, local election authorities conduct
acceptance testing to ensure the voting system equipment performs properly and is certified. Further
still, prior to every election, local authorities test the logic and accuracy of the equipment and
procedures with election-specific ballots to confirm it functions properly and is secure.

7. National Software Reference Library
After software is federally certified, election system vendors can voluntarily submit the executable
code to the National Software Reference Library, which archives a validation code for future reference. This allows jurisdictions to verify the delivered system software against the archived validation code to ensure it is the certified version.

Here's what the EAC says...



United States Election Assistance Commission
1225 New York Avenue N.W.
Washington, DC 20005
Testing and Certification Process for Voting Systems

The Help America Vote Act of 2002 (HAVA) directs the U.S. Election Assistance Commission (EAC) to issue Voluntary Voting System Guidelines to update and augment the 2002 Voting System Standards. The updated guidelines will address advancements in information and computer technologies. EAC released the proposed guidelines for public comment on June 29, which marked the beginning of a 90-day comment period. EAC will consider comments and then adopt final guidelines in the fall of 2005. HAVA also directs EAC to develop a national program for accrediting voting system testing laboratories and to oversee the certification of voting systems.

Currently, the National Association of State Election Directors (NASED) oversees the qualification process for voting systems, working with independent testing authorities (ITA) accredited by NASED. HAVA mandates that EAC assume responsibility for the accreditation of testing labs and the certification of voting systems. NASED is currently working with EAC to transfer this process, which will be completed in 2005.

The evolution of the testing and certification process for voting systems follows.

NASED History

* NASED is a non-partisan trade association consisting of election directors from the 50 states, the territories and the District of Columbia.
* Until HAVA was passed in 2002, there was no federal agency that oversaw the certification and/or qualification process. National or federal certification/qualification did not exist.
* NASED members agreed to fill the gap and took responsibility for the “qualification” process to assist the states.
* NASED began performing this function after members adopted the 1990 Voting System Standards and accredited an ITA to make sure voting systems met the requirements.
* NASED members and staff have always performed this function as unpaid volunteers.
* The entire qualification process is a cooperative effort among the states, not a federal function.

Current Qualification Process

* Currently no federal agency certifies/qualifies voting systems or accredits independent testing authorities. NASED is a non-partisan trade association.
* Today, voting systems are tested by ITAs against the 2002 Voting System Standards.
* ITAs are non-federal labs. These labs apply to the NASED Voting Systems Board to become accredited.
* A National Voluntary Laboratory Accreditation Program (NVLAP) accredited lab assessor reviews the workmanship and performance of these labs. The process follows the NASED Accreditation Handbook, modeled after similar National Institute of Standards and Technology (NIST)/NVLAP handbooks for lab accreditation in other industries. Once a lab has received accreditation, it may begin testing voting equipment.
* After the ITA determines through testing that a voting system meets the 2002 VSS, the Technical Subcommittee of the NASED Voting System Board reviews the test report to validate ITA’s findings.
* Only the Technical Subcommittee members see the test reports, and they have standing "non disclosure agreements" with the vendors.
* States are then notified that a voting system meets the requirements of 2002 VSS. The NASED qualification number and system information are made public on the NASED web site.
* Qualification is based solely on the ITA test reports and validation by the NASED Technical Sub Committee of the Voting Systems Board.

National Certification Mandated by HAVA

* HAVA mandated that EAC update the 2002 VSS.
* EAC, NIST and the Technical Guidelines Development Committee (TGDC) authored the Voluntary Voting System Guidelines to update and augment the 2002 Voting System Standards.
* EAC issued proposed guidelines for public comment on June 29, beginning a 90-day comment period. EAC will consider all comments, then vote to adopt a final version in the fall of 2005.
* These guidelines are voluntary. States can chose to adopt all, some or none of the guidelines. States can also set their own timeline for adoption of the guidelines.
* HAVA gives EAC the authority to accredit testing laboratories and federally certify voting systems – the first time both functions will be conducted by a federal entity. "The Commission shall provide for the testing, certification, decertification, and recertification of voting system hardware and software by accredited laboratories." -- HAVA, Section 231(a)(1)
* HAVA uses the word “certification” not “qualification” to describe the process.
* EAC will assume responsibility for entire certification process in 2005.
* Voting systems will be tested against the 2002 VSS or the 2005 Guidelines, depending on the requirements of individual states.
* NVLAP will evaluate testing labs and recommend to EAC which should be accredited.
* EAC will vote to accredit recommended testing labs.
* Upon the recommendation of the testing labs, EAC will issue federal certification to voting systems that meet the requirements.

http://www.eac.gov/testing_certification.htm

[Boredtodeath]

1. NASED only ISSUES certification numbers

They do no testing, no qualification.

If the ITA recommends the system for certification, NASED issues the number.

NASED is the lapdog of the ITA.

The ITA is the lapdog of the vendor.

[Einsteinia]

2. And now I thought this number issuing

authority was in the process of being handed over to the EAC (presumably because the NASED house of cards has been blow to smithereens).

[onthebench]

3. From what I remember...

The ITA is just a test lab. You send them a machine, they test it to a series of spelled out tests. They create a report listing results. If something fails in test, the vendor fixes problem and resubmits for test. A separate body then is the "certifying" agent. HAVA was supposed to put that in the hands of a technical committee that reported to the EAC. This is just how a missile system gets to the military. The missile is tested by a lab for a battery of tests. A government agency reads the test results and inspects the test lab and stamps approval. A certifying agent can question any test result. I know for a fact that NASED has questioned results in the past pre-HAVA. I know that certifications were delayed due to the questions being raised.

The other component is the state then can require a series of extra tests by other labs or other state consultants. The state uses the NASED testing as a filter to avoid having to listen to the Joe Inventor. I know that after 2000, the California SOS office saw hundreds of inventors. Each one was told to contact the ITAs first. This is how they weeded out the nut jobs. There actually was someone that created a voting machine using a gun to make the selection.

The real problem is that each state has very different rules for certification. Some are full of ambigous requirements. Some have no requirements. Some are full of old not applicable requirements. In the military, it is easy to standardize the process as there are not 50 different armies buying equipment. The Feds can not demand each state to have a uniform testing requirement since elections are state run. HAVA was an attempt to entice the states with money tied to rules but that did not work. It was underfunded and watered down.

[Wilms]

4. kick

[FogerRox]

5. KICKIN